We report the detection of water maser emission from four nearby galaxies hosting ultradense HII (UDHII) regions, He 2-10, the Antennae galaxies (NGC 4038/4039), NGC 4214, and NGC 5253, with the Green Bank Telescope. Our detection rate is 100%, and all of these H2O "kilomasers" (L(H2O) < 10 L_sun) are located toward regions of known star formation as traced by UDHII regions and bright 24 micron emission. Some of the newly discovered H2O masers have luminosities 1-2 orders of magnitude less than previous extragalactic studies and the same order of magnitude as those typical of Galactic massive star-forming regions. The unusual success of this minisurvey suggests that H2O maser emission may be very common in starburst galaxies, and the paucity of detections to date is due to a lack of sufficient sensitivity. While the galaxy sample was selected by the presence of UDHII regions, and the UDHII regions lie within the telescope beam, in the absence of H2O spectral line maps the connection between H2O masers and UDHII regions has not yet been demonstrated. Comment: 4 pages, 2 figures, accepted by ApJ Letters