We demonstrate simultaneous control of both the phase and amplitude of light using a conjugate gradient minimisation-based hologram calculation technique and a single phase-only spatial light modulator (SLM). A cost function which incorporates the inner product of the light field with a chosen target field within a defined measure region is efficiently minimised to create high fidelity patterns in the Fourier plane of the SLM. A fidelity of $F=0.999997$ is achieved for a pattern resembling an $LG^{0}_{1}$ mode with a calculated light-usage efficiency of $41.5\%$. Possible applications of our method in optical trapping and ultracold atoms are presented and we show uncorrected experimental realisation of our patterns with $F = 0.97$ and $7.8\%$ light efficiency.