If an extended source, such as a galaxy, is gravitationally lensed by a massive object in the foreground, the lensing distorts the observed image. It is straightforward to simulate what the observed image would be for a particular lens and source combination. In practice, one observes the lensed image on the sky, but blurred by atmospheric and telescopic effects and also contaminated with noise. The question that then arises is, given this incomplete data, what combinations of lens mass distribution and source surface brightness profile could plausibly have produced this image? This is a classic example of an inverse problem, and the method for solving it is given by the framework of Bayesian inference. In this paper we demonstrate the application of Bayesian inference to the problem of gravitational lens reconstruction, and illustrate the use of Markov Chain Monte Carlo simulations which can be used when the analytical calculations become too difficult. Previous methods for performing gravitational lens inversion are seen in a new light, as special cases of the general approach presented in this paper. Thus, we are able to answer, at least in principle, lingering questions about the uncertainties in the reconstructed source and lens parameters, taking into account all of the data and any prior information we may have. ; Comment: Accepted for publication in ApJ