We examine the influence of the adsorption geometry and electronic coupling between azobenzene-based molecular switches and metal surfaces on their photoisomerization ability. Using the normal-incidence X-ray standing wave technique and large-scale density functional theory (DFT) calculations we determine adsorption geometries for azobenzene and 3,3´,5,5´-tetra-tert-butyl-azobenzene (TBA) adsorbed on Ag(111). Comparing the experimental determined and calculated vertical bonding distances between the photochemically active diazo (-N=N-) moiety of both molecules, reveals that the photoisomerization ability is rather insensitive on the adsorption height, as the N-Ag adsorption distance in TBA/Ag(111) is only 0.14 Å (0.13 Å for the calculated value) larger than the corresponding value for azobenzene. Our DFT calculations predict also similar adsorption heights of the diazo-bridge for azobenzene and TBA adsorbed on Au(111) even though TBA undergoes a photoinduced isomerization while these process is suppressed in azobenzene/Au(111). The photoisomerization ability of TBA/Au(111) and its suppression for azobenzene on Au(111), Ag(111) as well as for TBA on Ag(111) thus demonstrate that a purely geometrical argumentation explaining the isomerization properties fails. Thus the electronic structure of the complete adsorbate/substrate complex has to be taken into account in order to control molecular functionality at surfaces.