This Letter proposes a method that balances miniaturization and high performance of fiber optic speckle spectrometers. The periodically tapered coreless fiber is used as the scattering element to excite more higher-order modes in the coreless fiber. As a result, a remarkable spectral resolution of 0.03 nm in the near-infrared spectrum can be achieved with a 5-cm-long fiber. Narrow linewidth and broadband spectra in the wavelength of 1540–1560 nm are reconstructed separately, demonstrating the excellent performance of the designed all-fiber spectrometer. The spectral resolution of our proposed spectrometer is comparable to that of a 2-m multimode fiber spectrometer and has a significant improvement in miniaturization.