A review of reviews following a scoping review study design was conducted in order to deconstruct the black box of interactions between the built environment and human behaviors pertaining to physical activity and/or diet. In the qualitative analysis 107 records were included, 45 of which were also coded. Most review papers confirmed the influence of the built environment on the behaviors of interest with some noting that a same built environment feature could have different behavioral outcomes. The conceptual model developed sheds light on these mixed results and brings out the role of several personal and behavioral factors in the shift from the measured to the perceived built environment. This shift was found to shape individuals’ behaviors critically and to have the power of redefining the strength of every interaction. Apart from its theoretical relevance, this model has high practical relevance especially for the design and implementation of interventions with a behavioral component. Intervention researchers can use the model developed to identify and label the built environment and individual factors that can be measured objectively or perceived as facilitators, concurrent options and barriers, in order to develop comprehensive and multi-component intervention strategies.