Finite-range numerical atomic orbitals are the basis functions of choice for several first principles methods, due to their flexibility and scalability. Generating and testing such basis sets, however, remains a significant challenge for the end user. We discuss these issues and present a new scheme for generating improved polarization orbitals of finite range. We then develop a series of high-accuracy basis sets for the water molecule, and report on their performance in describing the monomer and dimer, two phases of ice, and liquid water at ambient and high density. The tests are performed by comparison with plane-wave calculations, and show the atomic orbital basis sets to exhibit an excellent level of transferability and consistency. The highest-order bases (quadruple-ζ) are shown to give accuracies comparable to a plane-wave kinetic energy cutoff of at least ∼1000 eV for quantities such as energy differences and ionic forces, as well as achieving significantly greater accuracies for total energies and absolute pressures.