The extent and conditions of storage may affect the stability and quality of extra virgin olive oil (EVOO). This study aimed at evaluating the effects of different storage conditions (ambient, 4 °C and −18 °C temperatures, and argon headspace) on three EVOOs (low, medium, and high phenols) over 18 and 36 months, analyzing the main metabolites at six time points. The results showed that low temperatures are able to maintain all three EVOOs within the legal limits established by the current EU regulations for most compounds up to 36 months. Oleocanthal, squalene, and total phenols were affected by storage temperatures more than other compounds and degradation of squalene and α-tocopherol was inhibited only by low temperatures. The best temperature for 3-year conservation was 4 °C, but −18 °C represented the optimum temperature to preserve the organoleptic properties. The present study provided new insights that should guide EVOO manufacturers and traders to apply the most efficient storage methods to maintain the characteristics of the freshly extracted oils for a long conservation time.