In large-scale quantum-chemical calculations, the electron-repulsion integral (ERI) tensor rapidly becomes the bottleneck in terms of memory and disk space. When an external finite magnetic field is employed, this problem becomes even more pronounced because of the reduced permutational symmetry and the need to work with complex integrals and wave function parameters. One way to alleviate the problem is to employ a Cholesky decomposition (CD) to the complex ERIs over gauge-including atomic orbitals. The CD scheme establishes favorable compression rates by selectively discarding linearly dependent product densities from the chosen basis set while maintaining a rigorous and robust error control. This error control constitutes the main advantage over conceptually similar methods such as density fitting, which relies on employing pre-defined auxiliary basis sets. We implemented the use of the CD in the framework of finite-field (ff) Hartree–Fock and ff second-order Møller–Plesset perturbation theory (MP2). Our work demonstrates that the CD compression rates are particularly beneficial in calculations in the presence of a finite magnetic field. The ff-CD-MP2 scheme enables the correlated treatment of systems with more than 2000 basis functions in strong magnetic fields within a reasonable time span.