The visual systems of vertebrates and many other bilaterian clades consist of complex neural structures guiding a wide spectrum of behaviors. Homologies at the level of cell types and even discrete neural circuits have been proposed, but many questions of how the architecture of visual neuropils evolved among different phyla remain open. In this review we argue that the profound conservation of genetic and developmental steps generating the eye and its target neuropils in fish and fruit flies supports a homology between some core elements of bilaterian visual circuitries. Fish retina and tectum, and fly optic lobe, develop from a partitioned, unidirectionally proliferating neurectodermal domain that combines slowly dividing neuroepithelial stem cells and rapidly amplifying progenitors with shared genetic signatures to generate large numbers and different types of neurons in a temporally ordered way. This peculiar 'conveyor belt neurogenesis' could play an essential role in generating the topographically ordered circuitry of the visual system.