Mechanisms of molecular self-replication have the potential to shed light on the origins of life. In particular, self-replication through RNA-catalysed templated RNA synthesis is thought to have supported a primordial 'RNA world'. However, existing polymerase ribozymes lack the capacity to synthesize RNAs approaching their own size. Here, we report the in vitro evolution of such catalysts directly in the RNA-stabilizing medium of water ice, which yielded RNA polymerase ribozymes specifically adapted to sub-zero temperatures and able to synthesize RNA in ices at temperatures as low as -19 °C. The combination of cold-adaptive mutations with a previously described 5' extension operating at ambient temperatures enabled the design of a first polymerase ribozyme capable of catalysing the accurate synthesis of an RNA sequence longer than itself (adding up to 206 nucleotides), an important stepping stone towards RNA self-replication.