Single scattering by randomly oriented Saharan sand, silt, and clay particles is studied at 441.6 nm wavelength. Numerical simulations using the ray optics approximation and the Lorenz-Mie theory are compared with laboratory measurements of scattering matrix elements reported in the literature. The ray optics approximation is modified with ad hoc simple schemes of Lambertian surface elements and internal screens to study the effect of small-scale surface roughness and internal structures, respectively. Two different Lambertian reflection/refraction matrices with varying depolarization characteristics are applied. Model particle shapes are based on a tentative shape analysis of real Saharan particles. It is found that the traditional ray optics approximation agrees well with measurements only if unrealistically spiky particle shapes and an imaginary part of the refractive index (Im(m)) that is rather small compared with the typical values given in the literature are used. When the Lambertian schemes are applied, the agreement with measurements clearly improves. More importantly, good agreement can then be achieved using realistic particle shapes; this too requires a rather small Im(m). If Im(m) values corresponding to typical values in the literature are used, good fits can be achieved, but unrealistically spiky particle shapes have to be used. Our findings also indicate that sophisticated single-scattering modeling is important even below the ray optics domain. Finally, we demonstrate the importance of sophisticated single-scattering modeling for radiative flux and radiance calculations.