The evolution of new function in living organisms is slow and fundamentally limited by their critical mutation rate. Here, we established a stable orthogonal replication system in Escherichia coli. The orthogonal replicon can carry diverse cargos of at least 16.5 kilobases and is not copied by host polymerases but is selectively copied by an orthogonal DNA polymerase (O-DNAP), which does not copy the genome. We designed mutant O-DNAPs that selectively increase the mutation rate of the orthogonal replicon by two to four orders of magnitude. We demonstrate the utility of our system for accelerated continuous evolution by evolving a 150-fold increase in resistance to tigecycline in 12 days. And, starting from a GFP variant, we evolved a 1000-fold increase in cellular fluorescence in 5 days.