We use the ACS BViz data from the HUDF and all other deep HST ACS fields (including the GOODS fields) to find large samples of star-forming galaxies at z~4 and z~5 and to extend our previous z~6 sample. These samples contain 4671, 1416, and 627 B, V, and i dropouts, respectively, and reach to extremely low luminosities (0.01-0.04 L* or M(UV)~-16 to -17), allowing us to determine the rest-frame UV luminosity function (LF) and faint-end slope alpha at z~4-6 to high accuracy. We find faint-end slopes alpha of -1.73+/-0.05 at z~4, -1.66+/-0.09 at z~5, and -1.74+/-0.16 at z~6 -- suggesting that the faint-end slope is very steep and shows little evolution with cosmic time. We find that M*(UV) brightens considerably in the 0.7 Gyr from z~6 to z~4 (by ~0.7 mag from M*=-20.24+/-0.19 to M*=-20.98+/-0.10). The observed increase in the characteristic luminosity over this range is almost identical to that expected for the halo mass function -- suggesting that the observed evolution is likely due to the hierarchical coalescence and merging of galaxies. The evolution in phi* is not significant. The UV luminosity density at z~6 is modestly lower (0.45+/-0.09 times) than that at z~4 (integrated to -17.5 AB mag) though a larger change is seen in the dust-corrected star-formation rate density. We thoroughly examine published LF results and assess the reasons for their wide dispersion. We argue that the results reported here are the most robust available. The extremely steep faint-end slopes alpha found here suggest that lower luminosity galaxies play a significant role in reionizing the universe. Finally, we consider recent search results for galaxies at z~7-8 and use them to extend our estimates of the evolution in M* from z~7-8 to z~4. ; Comment: 33 pages, 18 figures, accepted for publication in the Astrophysical Journal, updated to match version in press