Background and Objectives: Brain atrophy is related to cognitive decline. However, the heritability of brain atrophy has not been fully investigated in the Eastern Asian population. Materials and Methods: Brain imaging of 74 Japanese twins registered in the Osaka University Twin Registry was conducted with voxel-based morphometry SPM12 and was processed by individual voxel-based morphometry adjusting covariates (iVAC) toolbox. The atrophy of the measured lobes was obtained by comparing the focal volume to the average of healthy subjects. Classical twin analysis was used to measure the heritability of its z-scores. Results: The heritability of brain atrophy ranged from 0.23 to 0.97, depending upon the lobes. When adjusted to age, high heritability was reported in the frontal, frontal-temporal, and parietal lobes, but the heritability in other lobes was lower than 0.70. Conclusions: This study revealed a relatively lower heritability in brain atrophy compared to other ethnicities. This result suggests a significant environmental impact on the susceptibility of brain atrophy the Japanese. Therefore, environmental factors may have more influence on the Japanese than in other populations.