This paper presents an investigation of the nonlinear least-squares sphere fitting algorithm (TLSA). The work concentrates on investigating the reliability of the TLSA algorithm when applied to a small segment angle of a sphere. The definition of small segment angle is discussed in the paper and taken to be below 1° (in both x and y directions) of the spherical surface. This application of the TLSA method is important when it is used on data from optical scanning systems where the measurements are limited by the gauge range and the angular tolerance of the sensor. The TLSA algorithm has been first compared with the TLSD algorithm suggested by Forbes for this application. The results show that the TLSA algorithm can be used in small surface segment angles. The main study is focused on testing the algorithm on a sphere superimposed with surface irregularities (sensor/measurement noise or roughness). Two properties of the TLSA algorithm are covered: the bias and the uncertainty of the estimated radius. Both simulation and theoretical approaches have been attempted. A new algorithm to estimate the bias of the TLSA algorithm has been derived in this paper based on Box's method. Together with uncertainty estimation, which can be produced by using either a conventional method or Zhang's error propagation function (EPF), a comprehensive understanding of the TLSA algorithm in this application is thus achieved, and used to develop a number of recommendations for the precision metrology of spherical and near spherical surfaces.