The magnitude and functional quality of antiviral CD8 T cell responses are critical for the efficacy of T cell based vaccines. Here, we investigate the influence of two popular viral vectors, adenovirus type 5 (Ad5) and modified vaccinia Ankara (MVA), on expansion, contraction and memory differentiation of HIV-1 Gag insert-specific CD8 T cell responses following immunization and show different patterns for the two recombinant viral vectors. The Ad5 vector primed 6-fold higher levels of insert-specific CD8 effector T cells than the MVA vector. The Ad5-primed effector cells also underwent less contraction (<2-fold) than the MVA-primed cells (>5-fold). The Ad5-primed memory cells were predominantly CD62L negative (effector memory) whereas the MVA-primed memory cells were predominantly CD62L positive (central memory). Consistent with their memory phenotype, MVA-primed CD8 T cells underwent higher fold expansion than Ad5-primed CD8 T cells following a homologous or heterologous boost. Impressively, the Ad5 boost changed the quality of MVA-primed memory response such that they undergo less contraction with effector memory phenotype. However, the MVA boost did not influence the contraction and memory phenotype of Ad5-primed response. In conclusion, our results demonstrate that vaccine vector strongly influences the expansion, contraction and the functional quality of insert-specific CD8 T cell responses and have implications for vaccine development against infectious diseases.