AirRater is a free smartphone app developed in 2015, supporting individuals to protect their health from environmental hazards. It does this by providing (i) location-specific and near real-time air quality, pollen and temperature information and (ii) personal symptom tracking functionality. This research sought to evaluate user perceptions of AirRater’s usability and effectiveness. We collected demographic data and completed semi-structured interviews with 42 AirRater users, identified emergent themes, and used two frameworks designed to understand and support behavior change—the Behavior Change Wheel (BCW) and the Protective Action Decision Model (PADM)—to interpret results. Of the 42 participants, almost half indicated that experiencing symptoms acted as a prompt for app use. Information provided by the app supported a majority of the 42 participants to make decisions and implement behaviors to protect their health irrespective of their location or context. The majority of participants also indicated that they shared information provided by the app with family, friends and/or colleagues. The evaluation also identified opportunities to improve the app. Several study limitations were identified, which impacts the generalizability of results beyond the populations studied. Despite these limitations, findings facilitated new insights into motivations for behavior change, and contribute to the existing literature investigating the potential for smartphone apps to support health protection from environmental hazards in a changing climate.