We present the results from adaptive optics (AO) assisted imaging in the Ks band of an area of 15 arcmin^2 for SWAN (Survey of a Wide Area with NACO). We derive the high resolution near-IR morphology of ~400 galaxies up to Ks~23.5 in the first 21 SWAN fields around bright guide stars, carefully taking into account the survey selection effects and using an accurate treatment of the anisoplanatic AO PSF. The detected galaxies are sorted into two morphological classes according to their Sersic index. The extracted morphological properties and number counts of the galaxies are compared with the predictions of different galaxy formation and evolution models, both for the whole galaxy population and separately for late-type and early-type galaxies. This is one of the first times such a comparison has been done in the near-IR, as AO observations and accurate PSF modeling are needed to obtain reliable morphological classification of faint field galaxies at these wavelengths. For early-type galaxies we find that a pure luminosity evolution model, without evidence for relevant number and size evolution, better reproduces the observed properties of our Ks-selected sample than current semi-analytic models based on the hierarchical picture of galaxy formation. In particular, we find that the observed flattening of elliptical galaxy counts at Ks~20 is quantitatively in good agreement with the prediction of the pure luminosity evolution model that was calculated prior to the observation. For late-type galaxies, while both models are able to reproduce the number counts, we find some hints of a possible size growth. These results demonstrate the unique power of AO observations to derive high resolution details of faint galaxies' morphology in the near-IR and drive studies of galaxy evolution. Comment: 15 pages, 10 figures. A&A, in press. Final version with corrected typos