Reliable information on building stock and its vulnerability is important for understanding societal exposure to floods. Unfortunately, developing countries have less access to and availability of this information. Therefore, calculations for flood damage assessments have to use the scarce information available, often aggregated on a national or district level. This study aims to improve current assessments of flood damage by extracting individual building characteristics and estimate damage based on the buildings' vulnerability. We carry out an object-based image analysis (OBIA) of high-resolution (11 cm ground sample distance) unmanned aerial vehicle (UAV) imagery to outline building footprints. We then use a support vector machine learning algorithm to classify the delineated buildings. We combine this information with local depth–damage curves to estimate the economic damage for three villages affected by the 2019 January river floods in the southern Shire Basin in Malawi and compare this to a conventional, pixel-based approach using aggregated land use to denote exposure. The flood extent is obtained from satellite imagery (Sentinel-1) and corresponding water depths determined by combining this with elevation data. The results show that OBIA results in building footprints much closer to OpenStreetMap data, in which the pixel-based approach tends to overestimate. Correspondingly, the estimated total damage from the OBIA is lower (EUR 10 140) compared to the pixel-based approach (EUR 15 782). A sensitivity analysis illustrates that uncertainty in the derived damage curves is larger than in the hazard or exposure data. This research highlights the potential for detailed and local damage assessments using UAV imagery to determine exposure and vulnerability in flood damage and risk assessments in data-poor regions.