Albedo, or average solar reflectance, is an important property affecting the development of temperature profiles within asphalt concrete (AC) pavements. It influences material durability and has an urban heat island effect. The albedo of new AC pavements ranges from 0.05 to 0.10 but rises rapidly to a stable value ranging from 0.15 to 0.25 within the first year of service. The mechanism of this rapid increase has typically been associated with oxidation of the asphalt binder, loosely referred to as “aging,” but has never been verified in the laboratory. To test this hypothesis, AC mixtures, prepared using PG 64-22 unmodified asphalt binder and aggregates, were subjected to accelerated aging using various techniques. The albedo of the AC mixtures was found to have statistically the same value of about 0.05 regardless of the extent and method of aging. An alternate hypothesis introduced is that the albedo increases as a result of abrasion loss of the surface binder, which has been tested in the laboratory. When this new abrasion procedure was applied to four field cores, the albedo increased in the same way observed in the field. This laboratory study confirmed that abrasion and loss of surface binder is the primary mechanism for the rapid increase in albedo of AC pavements.