Fertilization of soil is needed to fulfill the growing demand for livestock feed and human food requirements. However, fertilization has short and long-term impacts on the soil microbiota. These, in turn, may influence plant viability and growth. We investigated the soil microbiota of a 27-year field trial, focusing on the influences of mineral nitrogen (N) fertilization, different composts and combinations of compost plus mineral N as soil amendments. Two N rates (0 and 80 kg per ha) and four different composts (urban organic waste compost (OWC) green waste compost (GC), farmyard manure (MC) compost and sewage sludge compost (SSC)) were used. Soil samples for this study were taken in 2018 after the growing season of maize. In addition to maize yield, the effects on soil physicochemical properties and the soil microbiota were analyzed. There was a trend for increased maize yields for all fertilizers; however, only the application of GC and SSC in combination with mineral N fertilizer showed significant effects. The different organic amendments influenced physicochemical soil properties. Phosphorus concentrations were three times higher in plots receiving SSC (≈312 mg kg−1) and SSC + N (≈297 mg kg−1) than control (≈89 mg kg−1) or mineral N fertilizer (≈97 mg kg−1) alone. Magnesium concentrations in plots treated with SSC (≈74 mg kg−1) were lower compared to soils treated with GC and MC, respectively (≈135 mg kg−1 and 126 mg kg−1). Bacteria exceeded the fungal community in terms of both richness and diversity. While the bacterial community composition differed significantly among the treatments, the fungal community composition was rather unaffected. Our conclusion is that composts produced from various substrates serve as valuable nutrient sources for plants and can partially substitute mineral N. In addition, composts increased soil microbial biomass and modulated the composition of the soil’s microbial community.