An important task in connectomics studies is the classification of connectivity graphs coming from healthy and pathological subjects. In this paper, we propose a mathematical framework based on Riemannian geometry and kernel methods that can be applied to connectivity matrices for the classification task. We tested our approach using different real datasets of functional and structural connectivity, evaluating different metrics to describe the similarity between graphs. The empirical results obtained clearly show the superior performance of our approach compared with baseline methods, demonstrating the advantages of our manifold framework and its potential for other applications.