Measurements of Zn and Cr abundances in damped Lyman alpha systems at absorption redshifts between 0.692 and 3.390 show that metals and dust are much less abundant in high redshift galaxies than in the Milky Way today. We conclude that the overall degree of metal enrichment of DLA galaxies approximately 13.5 Gyr ago is 1/15 solar. The depletion of Cr is approximately 2, significantly less than in local interstellar clouds. We propose this reflects an overall lower abundance of dust, and deduce a typical dust-to-gas ratio of 1/30 of the Milky Way value, still sufficient to explain the weakness of Lyman alpha emission from star-forming regions. We show that, despite claims to the contrary, these conclusions are not inconsistent with recent high resolution observations of DLAs with the Keck Telescope. Comment: 23 pages, LaTeX type, 1 Postscript figure Submitted to Astrophysical Journal