Cancer therapy has entered a new era, transitioning from unspecific chemotherapeutic agents to increasingly specific immune-based therapeutic strategies. Among these, chimeric antigen receptor (CAR) T cells have shown unparalleled therapeutic potential in treating refractory hematological malignancies. In contrast, solid tumors pose a much greater challenge to CAR T cell therapy, which has yet to be overcome. As this novel therapeutic modality matures, increasing effort is being invested to determine the optimal structure and properties of CARs to facilitate the transition from empirical testing to the rational design of CAR T cells. In this review, we highlight how individual CAR domains contribute to the success and failure of this promising treatment modality and provide an insight into the most notable advances in the field of CAR T cell engineering.