In the present study, we perform the first direct analysis on how the composition of the prokaryotic soil community differs depending on whether high-throughput sequencing or fluorescent in situ hybridization (FISH) coupled with catalyzed reporter deposition (CARD) is used. Soil samples were collected along short (<3 m) tundra vegetation gradients from Northern Sweden. Relative abundances of Acidobacteria and Bacteroidetes estimated by the high-throughput sequencing were higher than those estimated by CARD–FISH, while relative abundances of Archaea and α-Proteobacteria estimated by high-throughput sequencing were lower than those estimated by CARD–FISH. The results indicated that the high-throughput sequencing overestimates/underestimates the relative abundance of some microbial taxa if we assume that CARD–FISH can provide potentially more quantitative data. Great caution should be taken when interpreting data generated by molecular technologies (both of high-throughput sequencing and CARD–FISH), and supports by multiple approaches are necessary to make a robust conclusion.