Systematic strategies from applied mathematics for stochastic modelling in climate are reviewed here. One of the topics discussed is the stochastic modelling of mid-latitude low-frequency variability through a few teleconnection patterns, including the central role and physical mechanisms responsible for multiplicative noise. A new low-dimensional stochastic model is developed here, which mimics key features of atmospheric general circulation models, to test the fidelity of stochastic mode reduction procedures. The second topic discussed here is the systematic design of stochastic lattice models to capture irregular and highly intermittent features that are not resolved by a deterministic parametrization. A recent applied mathematics design principle for stochastic column modelling with intermittency is illustrated in an idealized setting for deep tropical convection; the practical effect of this stochastic model in both slowing down convectively coupled waves and increasing their fluctuations is presented here.