Toxin-Antitoxin modules are small operons involved in stress response and persister cell formation that encode a “toxin” and its corresponding neutralizing “antitoxin”. Regulation of these modules involves a complex mechanism known as conditional cooperativity, which is supposed to prevent unwanted toxin activation. Here we develop mathematical models for their regulation, based on published molecular and structural data, and parameterized using experimental data for F-plasmid ccdAB, bacteriophage P1 phd/doc and E. coli relBE. We show that the level of free toxin in the cell is mainly controlled through toxin sequestration in toxin-antitoxin complexes of various stoichiometry rather than by gene regulation. If the toxin translation rate exceeds twice the antitoxin translation rate, toxins accumulate in all cells. Conditional cooperativity and increasing the number of binding sites on the operator serves to reduce the metabolic burden of the cell by reducing the total amounts of proteins produced. Combining conditional cooperativity and bridging of antitoxins by toxins when bound to their operator sites allows creation of persister cells through rare, extreme stochastic spikes in the free toxin level. The amplitude of these spikes determines the duration of the persister state. Finally, increases in the antitoxin degradation rate and decreases in the bacterial growth rate cause a rise in the amount of persisters during nutritional stress.