Background: A balanced diet rich in calcium and protein is recommended for bone-healthy people and osteoporosis patients, but it may also be important for rare bone disease (RBD). Little data is available on RBD and diet. Therefore, the aim of this study was to evaluate the nutritional behavior of patients with RBD. Methods: This single-center, cross-sectional, questionnaire-based study assessed the nutritional behavior of RBD patients (X-linked hypophosphatemia (XLH), osteogenesis imperfecta (OI), hypophosphatasia (HPP)), osteoporosis (OPO) patients and healthy controls (CTRL). The nutritional questionnaire comprised 25 questions from seven nutritional areas. The associations between socioeconomic factors and BMI were assessed by age-adjusted univariate analysis of covariance (ANCOVA). Results: Fifty patients with RBD (17 OI, 17 HPP, 16 XLH; mean age of 48.8 ± 15.9, 26.0% male, mean BMI 26.2 ± 5.6), 51 with OPO (mean age 66.6 ± 10.0, 9.8% male, mean BMI 24.2 ± 3.9) and 52 CTRL (mean age 50.8 ± 16.3, 26.9% male, mean BMI 26.4 ± 4.7) participated. Twenty-six (52.0%) RBD, 17 (33.4%) OPO and 24 (46.1%) CTRL were overweight or obese according to BMI. Only a minority of RBD, OPO and CTRL had a daily intake of at least three portions of milk or milk products (17.3% RBD, 15.6% OPO, 11.6% CTRL, p = 0.453). In general, similar nutritional behavior was observed between the three subgroups. However, significant differences were found in caffeine consumption (p = 0.016), fruit/vegetable juice consumption (p = 0.034), portions of fish per week (p = 0.044), high-fat meals per week (p = 0.015) and consumption of salty snacks (p = 0.001). Conclusion: Nutritional counseling, controlling BMI and ensuring sufficient calcium and protein intake are crucial in patients with osteoporosis as well as in rare bone diseases. Vitamin D does not appear to be sufficiently supplied by the diet, and therefore supplementation should be considered in patients with bone diseases.