Abstract Background Volunteer community health workers (CHWs) are a key approach to improving community-based maternal and child health services in developing countries. BRAC, a large Bangladeshi non-governmental organization (NGO), has employed female volunteer CHWs in its community-based health programs since 1977, recently including its Manoshi project, a community-based maternal and child health intervention in the urban slums of Bangladesh. A case–control study conducted in response to high dropout rates in the first year of the project showed that financial incentives, social prestige, community approval and household responsibilities were related to early retention in the project. In our present prospective cohort study, we aimed to better understand the factors associated with retention of volunteer CHWs once the project was more mature. Methods We used a prospective cohort study design to examine the factors affecting retention of volunteer CHWs who remained in the project after the initial start-up period. We surveyed a random sample of 542 CHWs who were working for BRAC Manoshi in December 2008. In December 2009, we revisited this cohort of CHWs and interviewed those who had dropped out about the main reasons for their dropping out. We used a multivariable generalized linear model regression analysis with a log link to estimate the relative risk (RR) of independent factors on retention. Results Of the 542 CHWs originally enrolled, 120 had dropped out by the end of one year, mainly because they left the slums. CHWs who received positive community appraisal (adjusted RR = 1.45, 95% confidence interval (CI) = 1.10 to 1.91) or were associated with other NGOs (adjusted RR = 1.13, 95% CI = 1.04 to 1.23) were more likely to have been retained in the project. Although refresher training was also associated with increased retention (adjusted RR = 2.25, 95% CI = 1.08 to 4.71) in this study, too few CHWs had not attended refresher training regularly to make it a meaningful predictor of retention that could be applied in the project setting. Conclusion Factors that affect retention of CHWs may change over time, with some factors that are important in the early years of a project losing importance as the project matures. Community health programs operating in fragile urban slums should consider changing factors over program duration for better retention of volunteer CHWs.