In stark contrast to the rapid development of vaccines against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), an effective human immunodeficiency virus (HIV) vaccine is still lacking. Furthermore, despite virologic suppression and CD4 T-cell count normalization with antiretroviral therapy (ART), people living with HIV (PLWH) still exhibit increased morbidity and mortality compared to the general population. Such differences in health outcomes are related to higher risk behaviors, but also to HIV-related immune activation and viral coinfections. Among these coinfections, cytomegalovirus (CMV) latent infection is a well-known inducer of long-term immune dysregulation. Cytomegalovirus contributes to the persistent immune activation in PLWH receiving ART by directly skewing immune response toward itself, and by increasing immune activation through modification of the gut microbiota and microbial translocation. In addition, through induction of immunosenescence, CMV has been associated with a decreased response to infections and vaccines. This review provides a comprehensive overview of the influence of CMV on the immune system, the mechanisms underlying a reduced response to vaccines, and discuss new therapeutic advances targeting CMV that could be used to improve vaccine response in PLWH.