Abstract Background It is documented that married women do not utilize contraceptive methods, because of the fear of adverse effects, no or seldom sexual interaction; perception that they should not use contraception during breastfeeding, postpartum amenorrhea, or dissatisfaction with a specific method of contraception. The current study aimed to examine the socio-economic inequalities associated with the non-use of modern contraceptive methods among young (15-24 years) and non-young (25-49 years) married women and the contributing factors in those inequalities. Methods The present study utilized the cross-sectional data from the fourth round of the National Family Health Survey (NFHS-4) with a sample of 499,627 women who were currently married. The modern methods of family planning include sterilization, injectables, intrauterine devices (IUDs/PPIUDs), contraceptive pills, implants, the standard days method, condoms, diaphragm, foam/jelly, the lactational amenorrhea method, and emergency contraception. Multivariable logistic regression analysis was used to estimate the odds of non-use of modern contraceptive methods according to different age groups after controlling for various confounding factors. Additionally, concentration curve and Wagstaff decomposition method were used in the study. Results The prevalence of non-use of modern contraceptive use was higher among women from young category (79.0%) than non-young category (45.8%). The difference in prevalence was significant (33.2%; p < 0.001). Women from non-young age group had 39% significantly lower odds of non-use of modern contraceptive use than women from young age group (15–24 years) [AOR: 0.23; CI: 0.23, 0.23]. The value of concentration quintile was -0.022 for young and -0.058 for non-young age groups which also confirms that the non-use of modern contraceptives was more concentrated among women from poor socio-economic group and the inequality is higher among non-young women compared to young women. About 87.8 and 55.5% of the socio-economic inequality was explained by wealth quintile for modern contraceptive use in young and non-young women. A higher percent contribution of educational status (56.8%) in socio-economic inequality in non-use of modern contraceptive use was observed in non-young women compared to only -6.4% in young women. Further, the exposure to mass media was a major contributor to socio-economic inequality in young (35.8%) and non-young (43.2%) women. Conclusion Adverse socioeconomic and cultural factors like low levels of education, no exposure to mass media, lack of or limited knowledge about family planning, poor household wealth status, religion, and ethnicity remain impediments to the use of modern contraceptives. Thus, the current findings provide evidence to promote and enhance the use of modern contraceptives by reducing socioeconomic inequality.