Abstract In this contribution, we investigate second harmonic generation (SHG) in periodically poled lithium niobate (LN) on insulator waveguides and examine under what conditions such waveguides suffer from undesirable loss due to lateral leakage. We investigate the lateral leakage losses in X-cut and Z-cut LN for the fundamental (1550 nm) and second harmonic (775 nm) wavelengths. Our findings show that Z-cut lithium niobate on insulator (LNOI) is more likely to suffer from lateral leakage and has a lower SHG efficiency. We further provide design guidelines for highly efficient nonlinear optical waveguides in LNOI and show how lateral leakage can be avoided.