The contributions of the hippocampal formation and adjacent regions of the medial temporal lobe (MTL) to memory are still a matter of debate. It is currently unclear, to what extent discrepancies between previous human lesion studies may have been caused by the choice of distinct patient models of MTL dysfunction, as disorders affecting this region differ in selectivity, laterality and mechanisms of post-lesional compensation. Here, we investigated the performance of three distinct patient groups with lesions to the MTL with a battery of visuo-spatial short-term memory tasks. Thirty-one subjects with either unilateral damage to the MTL (postsurgical lesions following resection of a benign brain tumor, 6 right-sided lesions, 5 left) or bilateral damage (10 post-encephalitic lesions, 10 post-anoxic lesions) performed a series of tasks requiring short-term memory of colors, locations or color-location associations. We have shown previously that performance in the association task critically depends on hippocampal integrity. Patients with postsurgical damage of the MTL showed deficient performance in the association task, but performed normally in color and location tasks. Patients with left-sided lesions were almost as impaired as patients with right-sided lesions. Patients with bilateral post-encephalitic lesions showed comparable damage to MTL sub-regions and performed similarly to patients with postsurgical lesions in the association task. However, post-encephalitic patients showed additional impairments in the non-associative color and location tasks. A strikingly similar pattern of deficits was observed in post-anoxic patients. These results suggest a distinct cerebral organization of associative and non-associative short-term memory that was differentially affected in the three patient groups. Thus, while all patient groups may provide appropriate models of medial temporal lobe dysfunction in associative visuo-spatial short-term memory, additional deficits in non-associative memory tasks likely reflect damage of regions outside the MTL. Importantly, the choice of a patient model in human lesion studies of the MTL significantly influences overall performance patterns in visuo-spatial memory tasks.