We study the holographic field theory dual of a probe SU(2) Yang-Mills field in a background $(4+1)$-dimensional asymptotically Anti-de Sitter space. We find a new ground state when a magnetic component of the gauge field is larger than a critical value. The ground state forms a triangular Abrikosov lattice in the spatial directions perpendicular to the magnetic field. The lattice is composed of superconducting vortices induced by the condensation of a charged vector operator. We perform this calculation both at finite temperature and at zero temperature with a hard wall cutoff dual to a confining gauge theory. The study of this state may be of relevance to both holographic condensed matter models as well as to heavy ion physics. The results shown here provide support for the proposal that such a ground state may be found in the QCD vacuum when a large magnetic field is present. ; Comment: 24 pages, 6 figures. v2: Added references, fixed typos, gave a clearer interpretation and more explicit details of the calculations