A sizable fraction of colorectal cancer (CRC) is expected to be explained by heritable factors, with heritability estimates ranging 12-35% from twin and family studies. Genome-wide association studies (GWAS) have successfully identified a number of common single-nucleotide polymorphisms (SNPs) associated with CRC risk. Although it has been shown that these CRC susceptibility SNPs only explain a small proportion of the genetic risk, it is not clear how much of the heritability these SNPs explain and how much is left to be detected by other, yet to be identified, common SNPs. Therefore, we estimated the heritability of CRC under different scenarios using Genome-wide Complex Trait Analysis (GCTA) in the Genetics and Epidemiology of Colorectal Cancer Consortium (GECCO) including 8,025 cases and 10,814 controls. We estimated that the heritability explained by known common CRC SNPs identified in GWAS was 0.65% (95% CI:0.3%- 1%; p= 1.11x10(-16)) while the heritability explained by all common SNPs was at least 7.42% (95% CI: 4.71%-10.12%; p=8.13x10(-8)); suggesting that many common variants associated with CRC risk remain to be detected. Comparing the heritability explained by the common variants with that from twin and family studies, a fraction of the heritability may be explained by other genetic variants, such as rare variants. In addition, our analysis showed that the gene x smoking interaction explained a significant proportion of the CRC variance (p=1.26x10(-2)). In summary, our results suggest that known CRC SNPs only explain a small proportion of the heritability and more common SNPs have yet to be identified.