Photonics has the potential for processing information at unprecedented speeds and accommodates massively parallel data processing algorithms. To compete against electronics, photonic switches must allow for dense integration and surpass the gigahertz performance of electronic components. This requires a nonlinear switching medium with an as-yet unavailable ultrafast giant nonlinear response. Here, we show that a suitable switching material can be engineered by harnessing the nanoscale confinement of light and the nonlinearity of metal. Using a nanostructured gold film, we demonstrate resonant switching, the performance of which is at least one order of magnitude faster and stronger than designed materials reported so far. It exceeds electronic component speeds by a factor of a thousand and achieves control of light with light in a film only 50 nm thick at an average light power level of only a few milliwatts, thus providing a ground-breaking solution for all-optical data processing.