Experimentation by farmers with conservation agriculture (CA) is increasing in southern Africa, but local longer term data on these new production systems are scarce. This study focuses on CA research at two contrasting on‐farm sites and one on‐station long‐term trial in Zimbabwe. The on‐farm trials were conducted at Chikato village on a sandy soil at Zimuto Communal Area with low rainfall and at Hereford farm near Bindura on a clay‐rich soil in a high rainfall area. The on‐station trial was at Henderson Research Station near Mazowe where more in‐depth soil studies were possible. Results of CA systems from the on‐station site show on average 38 and 65% greater water infiltration on ripline‐seeded (RS) and direct‐seeded CA treatments compared with conventionally ploughed control treatments. Results from on‐farm sites show a 123 and 168% greater aggregate stability at Hereford and 11 and 24% lower dispersion ratio at Chikato on the two CA compared with the conventionally ploughed control treatments. Soil carbon increased by 46% in the first 20 cm on the sandy soils at Chikato in RS and by 104% in direct‐seeded CA treatments in four cropping seasons from 2004 to 2008, while it stayed at low levels on the conventionally tilled control treatment. Yields on CA plots were higher on the sandy soils in dry seasons, but lower in very wet seasons because of waterlogging. Yields on clay soils were less affected by the rainfall season. Crop productivity from CA systems increased at all sites over time owing to better management although significant differences between CA and conventional treatments on the three sites were apparent only after several cropping seasons. Conservation agriculture offers practical solutions to small‐scale farmers threatened by future soil degradation and fertility loss, but its successful use will depend on weed control and adequate application of fertilizers. The results indicate that there is no immediate increase in maize (Zea mays L.) yield when changing from a tilled to a CA system, but there is gradual improvement in some soil quality indicators over time.