We report a study of a spin-down impurity strongly coupled to a spin-up Fermi sea (a so-called Fermi polaron) with the diagrammatic Monte Carlo (DiagMC) technique. Conditions of zero temperature and three dimensions are considered for an ultracold atomic gas with resonant interactions in the zero-range limit. A Feynman diagrammatic series is developed for the one-body and two-body propagators providing information about the polaron and molecule channel, respectively. The DiagMC technique allows us to reach diagram orders that are high enough for extrapolation to infinite order. The robustness of the extracted results is examined by checking various resummation techniques and by running the simulations with various choices for the propagators and vertex functions. It turns out that dressing the lines in the diagrams as much as possible is not always the optimal choice. We also identify classes of dominant diagrams for the one-body and two-body self-energy in the region of strong interaction. These dominant diagrams turn out to be the leading processes of the strong-coupling limit. The quasiparticle energies and Z factor are obtained as a function of the interaction strength. We find that the DiagMC results for the molecule and polaron properties are very similar to those obtained with a variational ansatz. Surprisingly, this variational ansatz gives very good predictions for the quasiparticle residue even when this residue is significantly less than 1.