Purely data driven approaches for machine learning present diculties when data is scarce relative to the complexity of the model or when the model is forced to extrapolate. On the other hand, purely mechanistic ap- proaches need to identify and specify all the interactions in the problem at hand (which may not be feasible) and still leave the is- sue of how to parameterize the system. In this paper, we present a hybrid approach us- ing Gaussian processes and dierential equa- tions to combine data driven modelling with a physical model of the system. We show how dierent, physically-inspired, kernel func- tions can be developed through sensible, sim- ple, mechanistic assumptions about the un- derlying system. The versatility of our ap- proach is illustrated with three case studies from computational biology, motion capture and geostatistics.