Context. Small binary asteroid systems and pairs are thought to form through fission induced by spin up via the Yarkovsky-O’Keefe-Radzievskii-Paddack (YORP) effect. This process is expected to depend on their structural strength and therefore composition. Aims. We aim to determine how taxonomic classes – used as a proxy for composition – are distributed amongst binary asteroids and asteroid pairs compared to the general population. Methods. We compared the distribution of taxonomic classes of binary systems and pairs with that of a reference sample of asteroids. We built this sample by selecting asteroids in a way that reproduces the orbital and size distribution of the binaries and pairs. We did this in order to minimize potential biases between samples. Results. A strong deficit of primitive compositions (C, B, P, D types) among binary asteroids and asteroid pairs is identified, as well as a strong excess of asteroids with mafic-silicate-rich surface compositions (S, Q, V, A types). Conclusions. Amongst low-mass, rapidly rotating asteroids, those with mafic-silicate-rich compositions are more likely to form multiple asteroid systems than their primitive counterparts.