New sensory stimuli can be learned with a single or a few presentations. Similarly, the responses of cortical neurons to a stimulus have been shown to increase reliably after just a few repetitions. Long-term memory is thought to be mediated by synaptic plasticity, butin vitroexperiments in cortical cells typically show very small changes in synaptic strength after a pair of presynaptic and postsynaptic spikes. Thus, it is traditionally thought that fast learning requires stronger synaptic changes, possibly because of neuromodulation. Here we show theoretically that weak synaptic plasticity can, in fact, support fast learning, because of the large number of synapsesNonto a cortical neuron. In the fluctuation-driven regime characteristic of cortical neuronsin vivo, the size of membrane potential fluctuations grows only asN, whereas a single output spike leads to potentiation of a number of synapses proportional toN.Therefore, the relative effect of a single spike on synaptic potentiation grows asN. This leverage effect requires precise spike timing. Thus, the large number of synapses onto cortical neurons allows fast learning with very small synaptic changes.SIGNIFICANCE STATEMENTLong-term memory is thought to rely on the strengthening of coactive synapses. This physiological mechanism is generally considered to be very gradual, and yet new sensory stimuli can be learned with just a few presentations. Here we show theoretically that this apparent paradox can be solved when there is a tight balance between excitatory and inhibitory input. In this case, small synaptic modifications applied to the many synapses onto a given neuron disrupt that balance and produce a large effect even for modifications induced by a single stimulus. This effect makes fast learning possible with small synaptic changes and reconciles physiological and behavioral observations.