Yam (Dioscorea spp.) plants are mostly dioecious and sometimes monoecious. Low, irregular, and asynchronous flowering of the genotypes are critical problems in yam breeding. Selecting suitable pollen parents and preserving yam pollen for future use are potential means of controlling these constraints and optimizing hybridization practice in yam breeding programs. However, implementing such procedures requires a robust protocol for pollen collection and viability testing to monitor pollen quality in the field and in storage. This study, therefore, aimed at optimizing the pollen germination assessment protocol for yam. The standard medium composition was stepwisely modified, the optimal growth condition was tested, and in vivo predictions were made. This study showed that the differences in yam pollen germination percentage are primarily linked to the genotype and growing conditions (i.e., medium viscosity, incubation temperature, and time to use) rather than the medium composition. The inclusion of polyethylene glycol (PEG) in the culture medium caused 67–75% inhibition of germination in D. alata. Although the in vivo fertilization was dependent on female parents, the in vitro germination test predicted the percentage fruit set at 25.2–79.7% and 26.4–59.7% accuracy for D. rotundata and D. alata genotypes, respectively. This study provides a reliable in vitro yam pollen germination protocol to support pollen management and preservation efforts in yam breeding.