In recent years, it has been suggested that host cells exert intrinsic mechanisms to control nuclear replicating DNA viruses. This cellular response involves nuclear antiviral factors targeting incoming viral genomes. Herpes simplex virus-1 (HSV-1) is the best-studied model in this context, and it was shown that upon nuclear entry HSV-1 genomes are immediately targeted by components of promyelocytic leukemia nuclear bodies (PML-NBs) and the nuclear DNA sensor IFI16 (interferon gamma inducible protein 16). Based on HSV-1 studies, together with limited examples in other viral systems, these phenomena are widely believed to be a common cellular response to incoming viral genomes, although formal evidence for each virus is lacking. Indeed, recent studies suggest that the case may be different for adenovirus infection. Here we summarize the existing experimental evidence for the roles of nuclear antiviral factors against incoming viral genomes to better understand cellular responses on a virus-by-virus basis. We emphasize that cells seem to respond differently to different incoming viral genomes and discuss possible arguments for and against a unifying cellular mechanism targeting the incoming genomes of different virus families.