Contemporary in-depth sequencing of environmental samples has provided novel insights into microbial community structures, revealing that their diversity had been previously underestimated. Communities in marine environments are commonly composed of a few dominant taxa and a high number of taxonomically diverse, low-abundance organisms. However, studying the roles and genomic information of these “rare” organisms remains challenging, because little is known about their ecological niches and the environmental conditions to which they respond. Given the current threat to coral reef ecosystems, we investigated the potential of corals to provide highly specialized habitats for bacterial taxa including those that are rarely detected or absent in surrounding reef waters. The analysis of more than 350,000 small subunit ribosomal RNA (16S rRNA) sequence tags and almost 2,000 nearly full-length 16S rRNA gene sequences revealed that rare seawater biosphere members are highly abundant or even dominant in diverse Caribbean corals. Closely related corals (in the same genus/family) harbored similar bacterial communities. At higher taxonomic levels, however, the similarities of these communities did not correlate with the phylogenetic relationships among corals, opening novel questions about the evolutionary stability of coral-microbial associations. Large proportions of OTUs (28.7–49.1%) were unique to the coral species of origin. Analysis of the most dominant ribotypes suggests that many uncovered bacterial taxa exist in coral habitats and await future exploration. Our results indicate that coral species, and by extension other animal hosts, act as specialized habitats of otherwise rare microbes in marine ecosystems. Here, deep sequencing provided insights into coral microbiota at an unparalleled resolution and revealed that corals harbor many bacterial taxa previously not known. Given that two of the coral species investigated are listed as threatened under the U.S. Endangered Species Act, our results add an important microbial diversity-based perspective to the significance of conserving coral reefs.