The automation of segmentation of medical images is an active research area. However, there has been criticism of the standard of evaluation of methods. We have comprehensively evaluated four novel methods of automatically segmenting subcortical structures using volumetric, spatial overlap and distance-based measures. Two of the methods are atlas-based - classifier fusion and labelling (CFL) and expectation-maximisation segmentation using a dynamic brain atlas (EMS), and two model-based - profile active appearance models (PAM) and Bayesian appearance models (BAM). Each method was applied to the segmentation of 18 subcortical structures in 270 subjects from a diverse pool varying in age, disease, sex and image acquisition parameters. Our results showed that all four methods perform on par with recently published methods. CFL performed significantly better than the other three methods according to all three classes of metrics. © 2008 Springer-Verlag Berlin Heidelberg.