We show that optical memories arranged along an optical path can perform arbitrary unitary transformations on frequency domain optical states. The protocol offers favourable scaling and can be used with any quantum memory that uses an off-resonant Raman transition to reversibly transfer optical information to an atomic spin coherence.