The coronavirus disease 2019 (COVID-19), caused by SARS-CoV-2, has spread around the world with high mortality. To diagnose promptly and accurately is the vital step to effectively control its pandemic. Dynamic characteristics of SARS-CoV-2-specific antibodies which are important for diagnosis of infection have not been fully demonstrated. In this retrospective, single-center, observational study, we enrolled the initial 131 confirmed cases of COVID-19 at Jin-Yin-Tan Hospital who had at least one-time antibody tested during their hospitalization. The dynamic changes of IgM and IgG antibodies to SARS-CoV-2 nucleocapsid protein in 226 serum samples were detected by ELISA. The sensitivities of IgM and IgG ELISA detection were analyzed. Result showed that the sensitivity of the IgG ELISA detection (92.5%) was significantly higher than that of the IgM (70.8%) (P < 0.001). The meantimes of seroconversion for IgM and IgG were 6 days and 3 days, respectively. The IgM and IgG antibody levels peaked at around 18 days and 23 days, and then IgM fell to below the baseline level at about day 36, whereas IgG maintained at a relatively high level. In conclusion, antibodies should be detected to aid in diagnosis of COVID-19 infection. IgG could be a sensitive indicator for retrospective diagnosis and contact tracing, while IgM could be an indicator of early infection.