Lithium-stuffed garnet oxides are promising candidates of lithium ion electrolyte materials due to their high ionic conductivities and good chemical stability. In this study, time-of-flight (TOF) neutron diffraction experiments and the Rietveld refinement were performed on four garnet compounds in the Li7-xLa3Zr2-xTaxO12 (x = 0–2) series to study their average structures. Different structural models (split-site and anisotropic displacement) for lithium disorder at tetrahedral (Td) and octahedral (Oh) sites were compared and results suggested possible lithium disorder around both ideal Oh and ideal Td sites. To study the local lithium distribution, atomistic simulations based on static energy minimization with interatomic potentials were carried out on a large number of configurations of eight compositions. We demonstrated that energy probability distribution plots were helpful in understanding the lithium disorder/order in these materials. Nuclear density maps provided a visual presentation of structural disorder at both Td and Oh lithium sites. Our simulation results suggested that Td lithium occupancy is generally lower than that obtained from the average structure (Rietveld refinement) and the exclusion principle (no nearest-neighbor Td–Oh–Td lithium triplets). In addition, the nearest-neighbor Td–Oh lithium pairs and (Oh,□)–Td–(Oh,□) lithium clusters appear to be the characteristic local features contributing to the structure and conduction of these lithium-stuffed garnet oxides.