We systematically analyzed the Al composition dependences of the structural properties of (AlxGa1−x)2O3 thin films grown on β-Ga2O3 (010) substrates. The crystal structure was characterized by x-ray diffraction, and the surface morphology was observed by reflection high-energy electron diffraction and atomic force microscopy. In the 100-nm-thick thin films, the crystallinity began to degrade and defects appeared on the surface when the Al composition x exceeded about 0.16. The defects developed mainly along the [201] direction and slightly along the [001] direction as x increased. The boundary where the thin film quality changed was close to a critical thickness curve calculated using the Matthews–Blakeslee model assuming the slip system of ⟨201⟩{102¯}.