We found, through extensive experimental studies, that the physical aging effects are absent in the relaxation of rubbing-induced birefringence (RIB) in polystyrene (PS), and the relaxation involves very small length scale. A phenomenological model based on individual birefringence elements is proposed for the RIB relaxation. The relaxation times (RTs) of the elements are found to be independent of the thermal or stress history of the samples, either before or after the formation of the birefringence. The RTs are also independent of the molecular weight, rubbing conditions, and film thickness, while the RTs distribution function does depend on the molecular weight and rubbing conditions. The model provides quantitative interpretations that agree very well with all the reported experimental results, and sheds important light on the novel behaviors of the RIB relaxation. The absence of physical aging effects is probably due to the combined effects of small length scale of the RIB relaxation, and the accelerated aging speed in the near surface region in which the RIB concentrates.