We explored experts’ perceived challenges and success factors in the recruitment of adults with a low socioeconomic position (SEP) for participation in community-based lifestyle modification programs. We conducted semi-structured interviews with 11 experienced project coordinators, based on a topic list that included experiences with recruitment, perceived barriers and success factors, and general views on recruitment strategies. Results revealed challenges related to the context of the program (e.g., limited program resources), psychosocial barriers of the participants (e.g., mistrust or skepticism), practical barriers (e.g., low literacy or having other priorities), and reasons to decline participation (e.g., lack of interest or motivation). Success factors were related to securing beneficial contextual and program-related factors (e.g., multi-layered recruitment strategy), establishing contact with the target group (e.g., via existing networks, community key-members), methods to increase engagement (e.g., personal approach and involvement of the target group in the program process) and making participation easier (e.g., providing transport), and providing various types of incentives. Concluding, the group of participants with low SEP covers a wide spectrum of individuals. Therefore, multiple recruitment strategies at multiple layers should be employed, and tailored. The lessons learned of those with hands-on experiences will help to enhance recruitment in future programs.