We compared six cases of congenital prosopagnosia to unimpaired participants using standardized test batteries, tailor-made experimental paradigms, and clinical questionnaires. Every prosopagnosic participant displayed deficits in recognizing famous faces and retaining novel faces over short periods of time. Other aspects of face perception such as judgment of emotional expression, speech reading and memory for faces and names were impaired to a lesser degree or only in single cases. No evidence was found for general visual deficits or social dysfunctions. Two of our six cases are first order relatives, and a further three report first-order relatives suffering from prosopagnosic symptoms. The results are in line with the idea of a genetic component to congenital prosopagnosia.