Imaging is becoming increasingly important for the diagnosis of large vessel vasculitis (LVV). Atherosclerosis may be difficult to distinguish from LVV on imaging as both are inflammatory conditions of the arterial wall. Differentiating atherosclerosis from LVV is important to enable optimal diagnosis, risk assessment, and tailored treatment at a patient level. This paper reviews the current evidence of ultrasound (US), 2-deoxy-2-[18F]fluoro-D-glucose positron emission tomography (FDG-PET), computed tomography (CT), and magnetic resonance imaging (MRI) to distinguish LVV from atherosclerosis. In this review, we identified a total of eight studies comparing LVV patients to atherosclerosis patients using imaging—four US studies, two FDG-PET studies, and two CT studies. The included studies mostly applied different methodologies and outcome parameters to investigate vessel wall inflammation. This review reports the currently available evidence and provides recommendations on further methodological standardization methods and future directions for research.