Background and Purpose— If the diagnostic and prognostic significance of brain microbleeds (BMBs) are to be investigated and used for these purposes in clinical practice, observer variation in BMB assessment must be minimized. Methods— Two doctors used a pilot rating scale to describe the number and distribution of BMBs (round, low-signal lesions, <10 mm diameter on gradient echo MRI) among 264 adults with stroke or TIA. They were blinded to clinical data and their counterpart’s ratings. Disagreements were adjudicated by a third observer, who informed the development of a new Brain Observer MicroBleed Scale (BOMBS), which was tested in a separate cohort of 156 adults with stroke. Results— In the pilot study, agreement about the presence of ≥1 BMB in any location was moderate (κ=0.44; 95% CI, 0.32–0.56), but agreement was worse in lobar locations (κ=0.44; 95% CI, 0.30–0.58) than in deep (κ=0.62; 95% CI, 0.48–0.76) or posterior fossa locations (κ=0.66; 95% CI, 0.47–0.84). Using BOMBS, agreement about the presence of ≥1 BMB improved in any location (κ=0.68; 95% CI, 0.49–0.86) and in lobar locations (κ=0.78; 95% CI, 0.60–0.97). Conclusion— Interrater reliability concerning the presence of BMBs was moderate to good, and could be improved with the use of the BOMBS rating scale, which takes into account the main sources of interrater disagreement identified by our pilot scale.