Coherent radiation with frequencies ranging from 0.3 to 30 THz has recently become accessible using femtosecond laser technology. These terahertz (THz) waves have already been applied in spectroscopy and imaging and can be manipulated using static optical elements such as lenses, polarizers and filters. However, ultrafast modulation of THz radiation is required as well, for instance, in short-range wireless communication or for preparing shaped THz transients for the coherent control of numerous material excitations. Here, we demonstrate an all-optically created transient metamaterial that allows us to manipulate the polarization of THz waveforms with subcycle switch-on times. The polarization-modulated pulses are potentially interesting for controlling elementary motions such as the vibration of crystal lattices, the rotation of molecules and the precession of spins.