Nowadays, a wide range of advanced techniques provides accurate and detailed 3D data about patients anatomy, as captured by medical scans (MRI, CT, MicroCT, etc.). While medical imaging assists daily clinical practice, 3D patient-specific models (3D-PSMs) of anatomy have still a quite limited use. We consider part-based semantic annotation beneficial to bring 3D-PSMs into clinical practice. To this end, tools are needed to extract clinically relevant information from 3D models, to associate such knowledge with their corresponding parts, and to support the storage, sharing and searching of annotated 3D-PSMs in a structured manner. In this context, we present the SemAnatomy3D framework, which demonstrates the idea of ontology-driven annotation and indexing of 3D-PSMs and their Parts-of-Relevance, characterized by anatomical landmarks and pathological markers (e.g. articular and non-articular facets, ligament insertion sites, erosions). The key functionaity is to offer services for part-base annotation of 3D-PSMs which enables search or browse the 3D-PSM according to the annotation attached to its Parts-of-Relevance. The paper describes the results in terms of methods to support the part-based annotation of 3D-PSM, and the formalization of the data model to store and manage global and part-based annotation to improve search and analysis of 3D patient-specific anatomical models and subparts. Finally, we specialized our framework to support the diagnosis of rheumatoid arthritis in the carpal bones, but, in principle, it can support similar tasks in other clinical applications.