We study the dipole layer formed at metal-organic interfaces by means of first-principles calculations. Interface dipoles are monitored by calculating the work function change of Au, Ag, Al, Mg and Ca surfaces upon adsorption of a monolayer of PTCDA (3,4,9,10-perylene-tetra-carboxylic-di-anhydride), perylene or benzene molecules. Adsorption of PTCDA leads to pinning of the work function for a range of metal substrates. It gives interface dipoles that compensate for the difference in the clean metal work functions, leading to a nearly constant work function. In contrast, adsorption of benzene always results in a decrease of the work function, which is relatively constant for all metal substrates. Both effects are found in perylene, where adsorption on low work function metals gives work function pinning, whereas adsorption on high work function metals gives work function lowering. The work function changes upon adsorption are analyzed and interpreted in terms of two competing effects. If the molecule and substrate interact weakly, the molecule pushes electrons into the surface, which lowers the work function. If the metal work function is sufficiently low with respect to the unoccupied states of the molecule, electrons are donated into these states, which increases the binding and the work function. Comment: 18 pages, 12 figures, 5 tables