Well-dispersed hierarchical straw–sheaf-like architectures of [Gd(1,4-BDC)1.5(H2O)2] (BDC2− = benzene-1,4-dicarboxylate) have been successfully synthesized via an ultrasonic-assisted solution-phase method in the presence of polyvinylpyrrolidone (PVP, 30 K). The as-obtained products were characterized by X-ray powder diffraction (XRD), energy dispersive spectrometry (EDS), field emission scanning electron microscopy (FE-SEM), transmission electron microscopy (TEM) and thermogravimetric analysis (TGA). Systematic investigations have been performed on the factors influencing the morphology of [Gd(1,4-BDC)1.5(H2O)2] nanostructures, such as the concentration of Na2BDC and Gd(NO3)3·6H2O, the ultrasonic time and power, as well as the amount of PVP. A possible mechanism responsible for the formation of hierarchical architectures was proposed. The as-obtained Eu3+- and Tb3+-doped [Gd(1,4-BDC)1.5(H2O)2] products show strong characteristic red and green emissions under ultraviolet excitation. Furthermore, the straw–sheaf shaped Gd2O3 can be obtained via a thermal decomposition method using the Gd-1,4-BDC complex as a precursor.