The Baculoviridae, a family of insect-specific large DNA viruses, is widely used in both biotechnology and biological control. Its applied value stems from millions of years of evolution influenced by interactions with their hosts and the environment. To understand how ecological interactions have shaped baculovirus diversification, we reconstructed a robust molecular phylogeny using 217 complete genomes and ~580 isolates for which at least one of four lepidopteran core genes was available. We then used a phylogenetic-concept-based approach (mPTP) to delimit 165 baculovirus species, including 38 species derived from new genetic data. Phylogenetic optimization of ecological characters revealed a general pattern of host conservatism punctuated by occasional shifts between closely related hosts and major shifts between lepidopteran superfamilies. Moreover, we found significant phylogenetic conservatism between baculoviruses and the type of plant growth (woody or herbaceous) associated with their insect hosts. In addition, we found that colonization of new ecological niches sometimes led to viral radiation. These macroevolutionary patterns show that besides selection during the infection process, baculovirus diversification was influenced by tritrophic interactions, explained by their persistence on plants and interactions in the midgut during horizontal transmission. This complete eco-evolutionary framework highlights the potential innovations that could still be harnessed from the diversity of baculoviruses.