The structure of small (2–5 nm) Ge quantum dots prepared by the colloidal synthesis route is examined. Samples were synthesized using either GeO₂or GeCl₄as precursor. As-prepared samples were further annealed under Ar or H₂/Ar atmosphere at different temperatures in order to understand the effect of annealing on their structure. It was found that as-prepared samples possess distinctly different structures depending on their synthesis route as indicated by their long-range ordering. An appreciable amount of oxygen was found to be bound to Ge in samples prepared with GeO₂as a precursor; however, not for GeCl₄. Based on combined transmission electron microscope, Raman, X-ray diffraction and X-ray absorption measurements, it is suggested that as-prepared samples are best described by the core-shell model with a small nano-crystalline core and an amorphous outer layer terminated either with oxygen or hydrogen depending on the synthesis route. Annealing in an H₂Ar atmosphere leads to sample crystallization and further nanoparticle growth, while at the same time reducing the Ge—O bonding. X-ray diffraction measurements for as-prepared and annealed samples indicate that diamond-type and metastable phases are present.