Olive groves are one of the most important agro-systems in the Mediterranean basin, and the Andalusia region produces the highest quantity of olive oil in Europe. The aim of this work was to evaluate the long-term (15 years) influence of two management practices in olive orchards—conventional tillage (CT) and no tillage with bare soil and herbicide application (NT + H)—on soil physical properties, infiltration capacity, erosion rates, and soil productivity. In addition, the short-term (2 years) influence of no tillage with cover crop management (NT-CC) on these parameters was also assessed. In the study area, CT and NT + H management practices showed unsustainable erosion values, 9.82 and 13.88 Mg ha−1 year−1, respectively, while NT-CC inclusion decreased the erosion rates (2.06 Mg ha−1 year−1). The implementation of NT-CC not only reduced erosion rates but also caused a change in the trend of soil productivity loss observed under CT and NT + H. In this sense, NT-CC showed a positive influence on soil quality. However, tillage removal led to a significant reduction in the infiltration capacity of soils under NT + H and NT-CC, which will be a serious handicap for water storage in an environment with continuous processes of water deficit.