(1) Background: The study investigated whether the ankle–brachial index (ABI) and pulse wave velocity (baPWV) could reflect the severity of small vessel disease (SVD) and large artery atherosclerosis (LAA). (2) Methods: A total of 956 consecutive patients diagnosed with ischemic stroke were prospectively enrolled from July 2016 to December 2017. SVD severity and LAA stenosis grades were evaluated via magnetic resonance imaging and carotid duplex ultrasonography. Correlation coefficients were calculated between the ABI/baPWV and measurement values. Multinomial logistic regression analysis was performed to determine predictive potential. (3) Results: Among the 820 patients included in the final analysis, the stenosis grade of extracranial and intracranial vessels was inversely correlated with the ABI (p < 0.001, respectively) and positively correlated with the baPWV (p < 0.001 and p = 0.004, respectively). Abnormal ABI, not baPWV, independently predicted the presence of moderate (adjusted odds ratio, aOR: 2.18, 95% CI: 1.31–3.63) to severe (aOR: 5.59, 95% CI: 2.21–14.13) extracranial vessel stenosis and intracranial vessel stenosis (aOR: 1.89, 95% CI: 1.15–3.11). Neither the ABI nor baPWV was independently associated with SVD severity. (4) Conclusions: ABI is better than baPWV in screening for and identifying the existence of cerebral large vessel disease, but neither test is a good predictor of cerebral SVD severity.