(1) Background: The use of complementary and alternative medicine (CAM) has seen a notable increase in popularity. However, there is an absence of data regarding the prevalence of CAM use in patients with rare bone diseases (RBDs). (2) Methods: This monocentric, cross-sectional study was carried out in a reference hospital for RBDs. RBD patients included individuals with osteogenesis imperfecta, hypophosphatasia and X-linked hypophosphatemia, and their data were compared with those of patients with osteoporosis (OPO) and of healthy controls (CON). This study utilized the German version (I-CAM-G) of the I-CAM questionnaire. (3) Results: This study comprised 50 RBD patients [mean age (SD) of 48.8 (±15.9), 26% male], 51 OPO patients [66.6 (±10.0), 9.8% male] and 52 controls [50.8 (±16.3), 26.9% male]. Treatments by naturopaths/healers were more prevalent in the RBD group (11.4%) compared with OPO (0%) and CON (5.8%) (p = 0.06). More than half of the OPO (60.8%) and CON (63.5%) patients and 46% of the RBD patients reported vitamin/mineral intake within the past 12 months (p = 0.16). Individuals with tertiary education had a significantly higher odds ratio of 2.64 (95% CI: 1.04–6.70, p = 0.04) for visiting any CAM provider. Further, OPO patients were significantly less likely to use self-help techniques compared with the CON group (OR = 0.42, 95% CI: 0.19–0.95; p = 0.04). (4) Conclusions: Herbal medicine, vitamin and mineral supplements, and self-help techniques were the most common forms of CAM reported by patients with RBDs. However, the use of CAM was generally low.