Neuronal circuit asymmetries are important components of brain circuits, but the molecular pathways leading to their establishment remain unknown. Here we found that the mutation of FRMD7, a gene that is defective in human congenital nystagmus, leads to the selective loss of the horizontal optokinetic reflex in mice, as it does in humans. This is accompanied by the selective loss of horizontal direction selectivity in retinal ganglion cells and the transition from asymmetric to symmetric inhibitory input to horizontal direction-selective ganglion cells. In wild-type retinas, we found FRMD7 specifically expressed in starburst amacrine cells, the interneuron type that provides asymmetric inhibition to direction-selective retinal ganglion cells. This work identifies FRMD7 as a key regulator in establishing a neuronal circuit asymmetry, and it suggests the involvement of a specific inhibitory neuron type in the pathophysiology of a neurological disease.