Background: The point of care in many health systems is increasingly a point of health data generation, data which may be shared and used in a variety of ways by a range of different actors. Aim: We set out to gather data about the perspectives on health data-sharing of people living in North East England who have been underrepresented within other public engagement activities and who are marginalized in society. Methods: Multi-site ethnographic fieldwork was carried out in the Teesside region of England over a 6-month period in 2019 as part of a large-scale health data innovation program called Connected Health Cities. Organizations working with marginalized groups were contacted to recruit staff, volunteers, and beneficiaries for participation in qualitative research. The data gathered were analyzed thematically and vignettes constructed to illustrate findings. Results: Previous encounters with health and social care professionals and the broader socio-political contexts of people’s lives shape the perspectives of people from marginalized groups about sharing of data from their health records. While many would welcome improved care, the risks to people with socially produced vulnerabilities must be appreciated by those advocating systems that share data for personalized medicine or other forms of data-driven care. Conclusion: Forms of innovation in medicine which rely on greater data-sharing may present risks to groups and individuals with existing vulnerabilities, and advocates of these innovations should address the lack of trustworthiness of those receiving data before asking that people trust new systems to provide health benefits.