AbstractThe [111]-oriented InAs/GaSb and GaSb/InAs core-shell nanowires have been studied by the 8 × 8 Luttinger-Kohn "Equation missing" Hamiltonian to search for non-vanishing fundamental gaps between inverted electron and hole bands. We focus on the variations of the band-inverted fundamental gap, the hybridization gap, and the effective gap with the core radius and shell thickness of the nanowires. The evolutions of all the energy gaps with the structural parameters are shown to be dominantly governed by the effect of quantum confinement. With a fixed core radius, a band-inverted fundamental gap exists only at intermediate shell thicknesses. The maximum band-inverted gap found is ~4.4 meV for GaSb/InAs and ~3.5 meV for InAs/GaSb core-shell nanowires, and for the GaSb/InAs core-shell nanowires the gap persists over a wider range of geometrical parameters. The intrinsic reason for these differences between the two types of nanowires is that in the shell the electron-like states of InAs is more delocalized than the hole-like state of GaSb, while in the core the hole-like state of GaSb is more delocalized than the electron-like state of InAs, and both favor a stronger electron-hole hybridization.