Rapid and inexpensive immunodiagnostic assays to monitor severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) seroconversion are essential for conducting large-scale COVID-19 epidemiological surveillance and profiling humoral responses against SARS-CoV-2 infections or immunizations. Herein, a colorimetic serological assay to detect SARS-CoV-2 IgGs in patients' plasma was developed using short antigenic epitopes conjugated to gold nanoparticles (AuNPs). Four immunodominant linear B-cell epitopes, located on the spike (S) and nucleocapsid (N) proteins of SARS-CoV-2, were characterized for their IgG binding affinity and used as highly specific biological motifs on the nanoparticle to recognize target antibodies. Specific bivalent binding between SARS-CoV-2 antibodies and epitope-functionalized AuNPs trigger nanoparticle aggregation, which manifests as a distinct optical transition in the AuNPs' plasmon characteristics within 30 min of antibody introduction. Co-immobilization of two epitopes improved the assay sensitivity relative to single-epitope AuNPs with a limit of detection of 3.2 nM, commensurate with IgG levels in convalescent COVID-19-infected patients. A passivation strategy was further pursued to preserve the sensing response in human plasma medium. When tested against 35 clinical plasma samples of varying illness severity, the optimized nanosensor assay can successfully identify SARS-CoV-2 infection with 100% specificity and 83% sensitivity. As the epitopes are conserved within the circulating COVID-19 variants, the proposed platform holds great potential to serve as a cost-effective and highly specific alternative to classical immunoassays employing recombinant viral proteins. These epitope-enabled nanosensors further expand the serodiagnostic toolbox for COVID-19 epidemiological study, humoral response monitoring, or vaccine efficiency assessment.