We experimentally studied the three-dimensional imaging of the microspheres by using confocal and conventional scanning polarization microscopes. Because of the field amplitude averaging effect of the confocal system, the polarization of the detected signals is mainly parallel to the initial polarization. As a result, the signal intensity from the microspheres in the confocal polarization microscope with a crossed analyzer was found to be weaker than that in the conventional system. Based on a vector approach that takes the polarization into account and on the image formations of the two systems, theoretical expressions are given that agree well with the experimental results.