The use of toolkits and reference frameworks for the design and evaluation of learning activities enables the systematic application of pedagogical criteria in the elaboration of learning resources and learning designs. Pedagogical classification as described in such frameworks is a major criterion for the retrieval of learning objects, since it serves to partition the space of available learning resources depending either on the pedagogical standpoint that was used to create them, or on the interpreted pedagogical orientation of their constituent learning contents and activities. However, pedagogical classification systems need to be evaluated to assess their quality with regards to providing a degree of inter-subjective agreement on the meaning of the classification dimensions they provide. Without such evaluation, classification metadata, which is typically provided by a variety of contributors, is at risk of being fuzzy in reflecting the actual pedagogical orientations, thus hampering the effective retrieval of resources. This paper describes a case study that evaluates the general pedagogical dimensions proposed by Conole et al. to classify learning resources. Rater agreement techniques are used for the assessment, which is proposed as a general technique for the evaluation of such kind of classification schemas. The case study evaluates the degree of coherence of the pedagogical dimensions proposed by Conole et al. as an objective instrument to classify pedagogical resources. In addition, the technical details on how to integrate such classifications in learning object metadata are provided.