Semiconductor-based bulk optical modulators and switches demand material homogeneity and lack of residual birefringence, to guarantee low attenuation and noise performances. The fulfillment of such requirements strongly depends on crystal growth technique and machinery process. We have developed a polarimetric two-dimensional mapping technique at λ=1.5 μm, that comparatively characterizes the residual stress-induced birefringence of semiconductor modulator rods from a functional point of view, by evaluating the distribution of effective birefringence and estimating the overall noise figure of merit (cross-talk) for optical switching applications. We have examined high-resistivity CdTe rods grown by the horizontal Bridgman technique, and compared results to data from GaAs single crystals grown by horizontal Bridgman, liquid encapsulated Czochralski and vapor pressure controlled Czochralski. The birefringence has been measured with a resolution in the Δn≈10−7 range; spatial resolution can be adjusted, reaching 10 μm. The performance of CdTe samples is strongly affected by the presence of dislocations and related localized stress fields, and acceptable performance is reached only for large diameter optical beams, which average birefringent effects on the cross-sectional area of the rod. High volume density of dislocations is intrinsic to single crystals grown by the Bridgman method. Growing techniques from the vapor phase usually result in more homogeneous samples. Therefore, we think that the development of a suitable vapor phase growth method is the key to obtain semi-insulating CdTe bulk crystals to be used for high-performance optical modulation and switching.