Recent morphological studies of galaxies by the Hubble Space Telescope (HST) have revealed that actively star-forming galaxies at intermediate and high redshifts (z= 0.5–2.0) have very clumpy and irregular distributions of stars. It is however unclear whether and how these clumpy galaxies evolve into the present spiral and elliptical galaxies with regular shapes. We here propose that spatial distributions of asymptotic giant branch (AGB) stars, probing the different mean age and metallicity of the underlying stellar population, can provide vital clues to the evolution of these clumpy galaxies, in particular, those at intermediate redshifts. In order to demonstrate this proposal to be quite promising, we show the results of test-particle simulations on the long-term dynamical evolution of unbound groups of AGB stars (‘stellar clumps’), which correspond to the successors of star-forming clumps at intermediate redshifts, in isolated and interacting galaxies. We particularly show that azimuthal distributions of AGB stars dispersed from stellar clumps as a result of gravitational interaction with their host galaxies can still be inhomogeneous several Gyr after stellar clump formation for some models. We also show that the inhomogeneities in the azimuthal distributions of dispersed AGB stars can disappear more quickly in stellar clumps with larger sizes and higher velocity dispersions. These results suggest that if apparently clumpy structures of galaxies at intermediate redshifts are due to stars in unbound or weakly bound clusters, spatial distributions of AGB stars can have fossil records on past clumpy structures of galaxies. As an example, we compare the latest observations on the spatial distributions of AGB stars in the Large and Small Magellanic Clouds (LMC and SMC, respectively) with the corresponding simulations, and thereby discuss whether the LMC and the SMC had massive star-forming clumps in their outer discs a few to several Gyr ago.