Adoptive cellular immunotherapy using chimeric antigen receptor (CAR)-modified T cells and Natural Killer (NK) cells are common immune cell sources administered to treat cancer patients. In detail, whereas CAR-T cells induce outstanding responses in a subset of hematological malignancies, responses are much more deficient in solid tumors. Moreover, NK cells have not shown remarkable results up to date. In general, immune cells present high plasticity to change their activity and phenotype depending on the stimuli they receive from molecules secreted in the tumor microenvironment (TME). Consequently, immune cells will also secrete molecules that will shape the activities of other neighboring immune and tumor cells. Specifically, NK cells can polarize to activities as diverse as angiogenic ones instead of their killer activity. In addition, tumor cell phagocytosis by macrophages, which is required to remove dying tumor cells after the attack of NK cells or CAR-T cells, can be avoided in the TME. In addition, chemotherapy or radiotherapy treatments can induce senescence in tumor cells modifying their secretome to a known as “senescence-associated secretory phenotype” (SASP) that will also impact the immune response. Whereas the SASP initially attracts immune cells to eliminate senescent tumor cells, at high numbers of senescent cells, the SASP becomes detrimental, impacting negatively in the immune response. Last, CAR-T cells are an attractive option to overcome these events. Here, we review how molecules secreted in the TME by either tumor cells or even by immune cells impact the anti-tumor activity of surrounding immune cells.