Abstract Background International migration leads to an increasingly fluctuating and superdiverse migrant population in reception countries. In Germany, structural, legal, social and political barriers to adequate care complicate the ability of health care professionals to respond to the challenges associated with caring for such a dynamic patient population. In this analysis, we explore the views of physicians working with asylum seekers and refugees in Germany, and aim to identify the barriers and enablers they face in providing appropriate care. Methods We performed a secondary qualitative analysis of 16 semi-structured interviews conducted for two studies on the delivery of medical care to asylum seekers in Germany. To examine physicians' views towards their work with asylum seekers, we analysed evaluative judgements on interpersonal and workplace factors, the external environment, the physician's own self, and individual medical conduct. Analysis was conducted by identifying cross-cutting themes through thematic analysis and mapping these onto a framework matrix. Results We found that evaluative judgements made by physicians providing care to asylum seekers emphasized one of three conceptualisations of “difference”: regarding personal or cultural attributes of the patient group; the spaces of care provision; or the context of asylum seeking patients. Emphasis on differences of the patient population and the spaces of care provision to asylum seekers was found to impede appropriate care, while physicians emphasizing contextual factors reported more responsive medical practices. Discussion Our findings suggest that processes of “othering” asylum seekers as a patient group as well as the heterotopic spaces of care provision endanger the doctor-patient relationship and responsiveness of care. Appropriate training, physical and human resources and encouraging support between physicians working with asylum seeking patients could alleviate these processes. Key messages Physicians’ conceptualisations of “difference” in providing care to asylum seekers focus on the patient group, workplace, or structural context; and affect the way in which medical care is provided. Structural competency training, resource investments, and clear structures could reduce physicians’ experiences of “difference” of the patient group and workplace, and facilitate responsive care.