The plant endomembrane system is an elaborate collection of membrane-bound compartments that perform distinct tasks in plant growth and development, and in responses to abiotic and biotic stresses. Most plant viruses are positive-strand RNA viruses that remodel the host endomembrane system to establish intricate replication compartments. Their fundamental role is to create optimal conditions for viral replication, and to protect replication complexes and the cell-to-cell movement machinery from host defenses. In addition to the intracellular antiviral defense, represented mainly by RNA interference and effector-triggered immunity, recent findings indicate that plant antiviral immunity also includes membrane-localized receptor-like kinases that detect viral molecular patterns and trigger immune responses, which are similar to those observed for bacterial and fungal pathogens. Another recently identified part of plant antiviral defenses is executed by selective autophagy that mediates a specific degradation of viral proteins, resulting in an infection arrest. In a perpetual tug-of-war, certain host autophagy components may be exploited by viral proteins to support or protect an effective viral replication. In this review, we present recent advances in the understanding of the molecular interplay between viral components and plant endomembrane-associated pathways.