The prostate is a luminal secretory tissue whose function is regulated by male sex hormones. Castration produces involution of the prostate to a reversible basal state, and as the majority of prostate cancers also have a luminal phenotype, drug-induced castration is a front line therapy. It has therefore been assumed that the tumor arises from transformation of a luminal progenitor cell. Here, we demonstrate that a minority basal "cancer stem cell" (CSC) population persists in primary human prostate cancers, as in normal prostate, serving as a reservoir for tumor recurrence after castration therapy. While the CSCs exhibit a degree of phenotypic fluidity from different patients, the tumor-initiating cells in immunocompromised mice express basal markers (such as p63), but do not express androgen receptor (AR) or markers of luminal differentiation (PSA, PAP) when freshly fractionated from human tissues or following culture in vitro. Estrogen receptors α and β and AR are transcriptionally active in the transit amplifying (TA) cell (the progeny of SC). However, AR protein is consistently undetectable in TA cells. The prostate-specific TMPRSS2 gene, while upregulated by AR activity in luminal cells, is also transcribed in basal populations, confirming that AR acts as an expression modulator. Selected cells with basal phenotypes are tumor initiating, but the resultant tumors are phenotypically intermediate, with focal expression of AR, AMACR, and p63. In vitro differentiation experiments, employing lentivirally transduced SCs with a luminal (PSA-probasin) promoter regulating a fluorescent indicator gene, confirm that the basal SCs are the source of luminal progeny.