For a floating three-dimensional (3D) display system using a prism type retroreflector, non-retroreflected light and a blurred 3D image source are two key causes of the deterioration in image quality. In the present study, ray tracing is used to analyze the light distribution of a retroreflector at different incident angles. Based on this analysis, a telecentric retroreflector (TCRR) is proposed to suppress non-retroreflected light without sacrificing the viewing angle. A contrast transfer function (CTF) is used to evaluate the optical performance of the TCRR. To improve the 3D image source, the relationship between the root mean square (RMS) of the voxels and the 3D image quality is discussed, and an aspheric lens array is designed to reduce aberrations. Computational simulation results reveal that the structural similarity (SSIM) of the 3D image source increased to 0.9415. An experimental prototype system combining the TCRR and optimized 3D image source is then built. Experimental analysis demonstrates that the proposed method suppresses non-retroreflected light and improves the 3D image source. In particular, a clear floating 3D image with a floating distance of 70 mm and a viewing angle of 50° can be achieved