Molybdenum ditelluride, MoTe2, is a versatile material where the topological phase can be readily tuned by manipulating the associated structural phase transition. The fine details of the band structure of MoTe2, key to understanding its topological properties, have proven difficult to disentangle experientially due to the multi-band character of the material. Through experimental optical conductivity spectra, we detect two strong low-energy interband transitions. Both are linked to excitations between spin-orbit split bands. The lowest interband transition shows a strong thermal shift, pointing to a chemical potential that dramatically decreases with temperature. With the help of ab initio calculations and a simple two-band model, we give qualitative and quantitative explanation of the main features in the temperature-dependent optical spectra up to 400 meV.