Early in vivo embryonic retinal development is a well-documented and evolutionary conserved process. The specification towards eye development is temporally controlled by consecutive activation or inhibition of multiple key signaling pathways, such as the Wnt and hedgehog signaling pathways. Recently, with the use of retinal organoids, researchers aim to manipulate these pathways to achieve better human representative models for retinal development and disease. To achieve this, a plethora of different small molecules and signaling factors have been used at various time points and concentrations in retinal organoid differentiations, with varying success. Additions differ from protocol to protocol, but their usefulness or efficiency has not yet been systematically reviewed. Interestingly, many of these small molecules affect the same and/or multiple pathways, leading to reduced reproducibility and high variability between studies. In this review, we make an inventory of the key signaling pathways involved in early retinogenesis and their effect on the development of the early retina in vitro. Further, we provide a comprehensive overview of the small molecules and signaling factors that are added to retinal organoid differentiation protocols, documenting the molecular and functional effects of these additions. Lastly, we comparatively evaluate several of these factors using our established retinal organoid methodology.