Vitamin D insufficiency/deficiency is prevalent worldwide. We investigated the effect of vitamin D intake and ultraviolet ray (UV) exposure on serum vitamin D concentration in Japan. A total of 107 healthy adult participants were recruited from Hokkaido (43° N) and Kumamoto (33° N) prefectures. All participants undertook surveys in both summer and winter. Serum 25-hydroxyvitamin D (25(OH)D3) was examined, and vitamin D intake was assessed with a diet history questionnaire. UV exposure was measured with a wearable UV dosimeter. Regression analysis was performed to investigate the relationship between these factors, with covariates such as sun avoidance behavior. The prevalence of vitamin D insufficiency (serum 25(OH)D3; 12 ng/mL (30 nmol/L) ≤ and <20 ng/mL (50 nmol/L))/deficiency (<12 ng/mL) was 47.7% in summer and 82.2% in winter. UV exposure time was short in Kumamoto (the urban area), at 11.6 min in summer and 14.9 min in winter. In Hokkaido (the rural area), UV exposure time was 58.3 min in summer and 22.5 min in winter. Vitamin D intake was significantly associated with serum 25(OH)D3, and a 1 μg/1000kcal increase in intake was necessary to increase 25(OH)D3 by 0.88 ng/mL in summer and by 1.7 ng/mL in winter. UV exposure time was significantly associated with serum 25(OH)D3 in summer, and a 10 min increase in UV exposure time was necessary to increase 25(OH)D3 by 0.47 ng/mL. Although consideration of personal occupation and lifestyle is necessary, most Japanese may need to increase both vitamin D intake and UV exposure.