The microbial community within the rumen can be changed and shaped by heat stress. Accumulating data have suggested that different breeds of dairy cows have differential heat stress resistance; however, the underlying mechanism by which nonanimal factors contribute to heat stress are yet to be understood. This study is designed to determine changes in the rumen microbiome of Holstein and Jersey cows to normal and heat stress conditions. Under heat stress conditions, Holstein cows had a significantly higher respiration rate than Jersey cows. Heat stress increased the rectal temperature of Holstein but not Jersey cows. In the Kyoto encyclopedia of genes and genomes (KEGG) pathway analysis, Jersey cows had a significantly higher proportion of genes associated with energy metabolism in the normal condition than that with other treatments. Linear discriminant analysis effect size (LEfSe) results identified six taxa as distinguishing taxa between normal and heat stress conditions in Holstein cows; in Jersey cows, 29 such taxa were identified. Changes in the rumen bacterial taxa were more sensitive to heat stress in Jersey cows than in Holstein cows, suggesting that the rumen mechanism is different in both breeds in adapting to heat stress. Collectively, distinct changes in rumen bacterial taxa and functional gene abundance in Jersey cows may be associated with better adaptation ability to heat stress.