Although the left and right hemispheres of our brains develop with a high degree of symmetry at both the anatomical and functional levels, it has become clear that subtle structural differences exist between the two sides and that each is dominant in processing specific cognitive tasks. As the result of evolutionary conservation or convergence, lateralization of the brain is found in both vertebrates and invertebrates, suggesting that it provides significant fitness for animal life. This widespread feature of hemispheric specialization has allowed the emergence of model systems to study its development and, in some cases, to link anatomical asymmetries to brain function and behavior. Here, we present some of what is known about brain asymmetry in humans and model organisms as well as what is known about the impact of environmental and genetic factors on brain asymmetry development. We specifically highlight the progress made in understanding the development of epithalamic asymmetries in zebrafish and how this model provides an exciting opportunity to address brain asymmetry at different levels of complexity.