Adoptive transfer of T cells gene-engineered with antigen-specific T cell receptors (TCRs) has proven its feasibility and therapeutic potential in the treatment of malignant tumors. To ensure further clinical development of TCR gene therapy, it is necessary to target immunogenic epitopes that are related to oncogenesis and selectively expressed by tumor tissue, and implement strategies that result in optimal T cell fitness. In addition, in particular for the treatment of solid tumors, it is equally necessary to include strategies that counteract the immune-suppressive nature of the tumor micro-environment. Here, we will provide an overview of the current status of TCR gene therapy, and redefine the following three challenges of improvement: "choice of target antigen"; "fitness of T cells"; and "sensitization of tumor milieu." We will categorize and discuss potential strategies to address each of these challenges, and argue that advancement of clinical TCR gene therapy critically depends on developments toward each of the three challenges.