This study presents data on the development and preliminary validation of an observational scale assessing neighborhood disorder. Independent observations by trained raters of neighborhood disorder were conducted in 552 census block groups in the city of Valencia (Spain). Intraclass correlation coefficients assessing inter-rater reliability indicated fair to substantial levels of agreement among raters. Confirmatory factor analyses supported a final three-factor model scale measuring physical disorder, social disorder, and physical decay. Results for the internal consistency showed large composite reliability indices indicating good reliability for all neighborhood disorder factors. Evidence of criterion-related validity was found by exploring associations between neighborhood disorder factors and three neighborhood characteristics: neighborhood socioeconomic status, immigrant concentration, and residential instability. Also for criterion-related validity, Moran's I test results for spatial correlation showed that the three types of neighborhood disorder tend to cluster in space and are not randomly distributed across the city. In general, this paper provides evidence of a reliable and valid observational measure to assess neighborhood disorder.