T cells engineered to express chimeric antigen receptors (CARs) targeting CD19 have demonstrated impressive activity against relapsed or refractory B cell cancers yet fail to induce durable remissions for nearly half of patients treated. Enhancing the efficacy of this therapy requires detailed understanding of the molecular circuitry that restrains CAR-driven anti-tumor T cell function. We developed and validated an in vitro model that drives T cell dysfunction through chronic CAR activation and interrogated how CAR costimulatory domains, central components of CAR structure and function, contribute to T cell failure. We found that chronic activation of CD28-based CARs results in activation of classical T cell exhaustion programs and development of dysfunctional cells that bear the hallmarks of exhaustion. In contrast, 41BB-based CARs activate a divergent molecular program and direct differentiation of T cells into a novel cell state. Interrogation of CAR T cells from a patient with progressive lymphoma confirmed activation of this novel program in a failing clinical product. Further, we demonstrate that 41BB-dependent activation of the transcription factor FOXO3 is directly responsible for impairing CAR T cell function. These findings identify that costimulatory domains are critical regulators of CAR-driven T cell failure and that targeted interventions are required to overcome costimulation-dependent dysfunctional programs.