Compared to COVIDneg patients at the time of diagnostic testing, COVIDpos patients tended to have higher plasma fibrinogen levels and lower platelet counts. However, as the infection evolves, COVIDpos patients distinctively show declining fibrinogen, increasing platelet counts, and lower white blood cell counts. Augmented curation of EHRs suggests that only a minority of COVIDpos patients develop thromboembolism, and rarely, disseminated intravascular coagulopathy (DIC), with patients generally not displaying platelet reductions typical of consumptive coagulopathies. These temporal trends provide fine-grained resolution into COVID-19 associated coagulopathy (CAC) and set the stage for personalizing thromboprophylaxis.Temporal inference from laboratory testing results and triangulation with clinical outcomes extracted from unstructured electronic health record (EHR) provider notes is integral to advancing precision medicine. Here, we studied 246 SARS-CoV-2 PCR-positive (COVID There is a growing body of evidence suggesting that severe COVID-19 outcomes may be associated with dysregulated coagulation , includipos) and 72,354 confirmed SARS-CoV-2 negative (COVIDneg) patients over a retrospectively defined 2-month observation period straddling the date of the PCR test. For the COVIDpos cohort, we center the 2-month observation period around the date of the first positive PCR test for SARS-CoV-2, and for the COVIDneg cohort, we center the 2-month observation period around the date of the first PCR test for SARS-CoV-2 . It is important to note that not all individuals infected by SARS-CoV-2 develop symptoms of COVID-19, but rather that a majority of patients are either asymptomatic or have mild-to-moderate symptoms not requiring hospitalization for COVID-19  clinical notes and associated 3.2 million lab tests from 2232 SARS-CoV-2 positive , as well as the 30 days following the diagnostic testing date, and triangulating this information with medications and clinical outcomes, we were able to identify laboratory abnormalities significantly associated with the COVIDpos and 13,666 COVIDneg patients met study the inclusion criteria. The need for longitudinal data on the testing results, while constraining the study population size greatly, enables us to provide a fine-grained temporal resolution of CAC for the first time.In order to hone into longitudinal lab test trends that would apply at the individual patient level, we restricted our analysis to patients with available serial testing data, which had at least three test results of the same type during the observation period. After applying these inclusion criteria, 246 COVIDpos and COVIDneg groups were 60.8 years and 64.1 years, respectively  and 3901 (29%), respectively. These counts of coagulopathies include the following phenotypes identified in the clinical notes from day −365 to day −31 relative to the PCR testing date: deep vein thrombosis, pulmonary embolism, myocardial infarction, venous thromboembolism, thrombotic stroke, cerebral venous thrombosis, and disseminated intravascular coagulation , compared to 1247 (9.1%) for the COVIDneg cohort.After filtering the patients with the available longitudinal testing data, the median age in the COVIDneg cohort to use for the final statistical analysis . In particular, the general categories of covariates considered for balancing included: demographics, anticoagulant/antiplatelet medication use, medical history of pre-existing coagulopathies, and hospital admission status. Population-level characteristics of the COVIDpos, COVIDneg, and the final propensity score-matched COVIDneg (matched) cohorts are summarized in pos and COVIDneg (matched) cohorts are well-balanced along these covariates which are potential confounding variables for thrombotic events and coagulopathy-related lab tests during the study period.To balance these clinical covariates and others between the two cohorts, we applied 1:10 propensity score matching to define a subset of 2460 patients from the COVIDpos and COVIDneg (matched) patients, we analyzed longitudinal trends of 194 laboratory test results in the 30 days before and after the day of PCR testing (designated as day 0). As most patients did not undergo laboratory testing for each assay on a daily basis, we grouped the measurements into nine time windows reflecting potential stages of infection as follows: pre-infection (days −30 to −11), pre-PCR (days −10 to −2), time of clinical presentation (days −1 to 0), and post-PCR phases 1 (days 1 to 3), 2 (days 4 to 6), 3 (days 7 to 9), 4 (days 10 to 12), 5 (days 13 to 15), and 6 (days 16 to 30). We only considered test-time window pairs in which at least three patients contributing to laboratory test results in both groups. During each time window, we then compared the distribution of results from COVIDpos versus COVIDneg (matched) patients, allowing us to identify any lab tests which were significantly altered in COVIDpos patients during any time of disease acquisition, onset, and/or progression.To identify laboratory test results that differ between COVIDpos patients were observed  which met our thresholds for statistical significance  cohorts significantly above the normal range; however, there was no differentiation between the cohorts. We observe extended activated partial thromboplastin times (aPTT) in the COVIDpos significantly above normal levels from day 7 onward  cohorts using different time window definitions including daily trends  were positively diagnosed with one or more of the above-mentioned thrombotic phenotypes in the 30 days after PCR testing, with the majority of these patients (53 of 101) experiencing a deep vein thrombosis. Interestingly, we found that after creating subsets of the patients with longitudinal lab testing data (i.e. the patients meeting the criteria for inclusion in our study), 76 of the 246 patients (31%) had at least one EHR-derived clot diagnosis, including 47 patients with deep vein thrombosis (−50) for patients experiencing thrombotic events compared to the overall COVIDpos cohort.We found that 101 of the total 2232 COVIDrombosis . Thus, tpos patients with longitudinal lab testing data, 81 were serially tested starting at clinical presentation for fibrinogen versus 245 tested for platelets. As such, we first analyzed whether associations exist between platelet counts  and clotting propensity in this cohort. Among these 245, there were 169 patients without thrombosis after PCR-based diagnosis (non-thrombotic) and 76 patients with thrombosis (thrombotic). There is a statistically significant difference between the COVIDpos and COVIDneg cohorts in the platelet count at clinical presentation (9/L) was observed in 29% (46 out of 154) COVIDpos and 21% (346 of 1661) COVIDneg patients at the time of diagnosis , but we were not able to do so given the lack of longitudinal testing available for a large majority of non-thrombotic COVIDpos patients  despite the population-level trend of increasing platelets over time. Indeed, we found that nine of 74 thrombotic patients showed absolute platelet counts below 100 × 109/L during at least one post-diagnosis time window  criteria for disseminated intravascular coagulation  to have low-grade DIC, and as expected we found this patient’s longitudinal lab test pattern to be consistent with consumptive coagulopathy . Over the next 10 days, this patient’s fibrinogen levels consistently decreased, reaching a minimum of 110 mg/dL on day 9. Similarly, after an initial recovery to 190 × 109/L the platelet counts consistently declined starting on day 2 post-diagnosis, reaching a minimum of 117 × 109/L on day 11. D-dimer levels exponentially increased after 5 days, reaching a maximum of 41,300 ng/mL on day 10. Phenotypically, this patient experienced both thrombotic  and hemorrhagic  events. This combination of lab results and clinical manifestations is consistent with the diagnosis of DIC-like consumptive coagulopathy during the first week after COVID-19 diagnosis.This patient-level analysis indeed revealed multiple distinct trajectories with respect to fibrinogen and other coagulation parameters in COVIDLab test results from three other non-DIC thrombotic patients with longitudinal fibrinogen testing confirm the presence of alternative forms of coagulopathy in the COVID-19 population. Patient 23 developed a clot on day 4 post-diagnosis in the context of a declining fibrinogen level and increasing D-dimers but steady platelet counts, which actually increased shortly thereafter . Patientpos cohort. This hypothesis may warrant further analysis in cohorts with more longitudinal fibrinogen data, but again it is important to note that several COVIDpos patients who presented with hyperfibrinogenemia did not go on to develop thromboses (One hypothesis is that early elevations in plasma fibrinogen contribute to the clotting observed in the non-DIC like COVIDromboses . This emromboses .pos patients and emphasizes that consumptive coagulopathy is an exception rather than the rule as it pertains to thrombotic phenotypes in COVID-19 patients. These results should be considered as a preliminary characterization of COVID-associated coagulopathies (CAC) and will be updated as patient counts increase with the continued evolution of the COVID-19 pandemic.Taken together, this analysis affirms that a DIC-like coagulopathy resulting in a combination of hemorrhage and thrombosis can develop in the setting of COVID-19 infection. However, the observations that DIC was formally diagnosed in only five of 2232 COVIDpos patients showed leukopenia and lymphopenia, and prothrombin time and D-dimer levels were higher in ICU patients than in non-ICU patients  is the predominant cause of death. A recent study suggested that DIC was associated with MODS during the early stage of ARDS and that persistent DIC may also have a role in this association . Our stuThe ability to derive this longitudinal understanding of COVID-19 progression, including laboratory abnormalities and their associated clinical manifestations, mandates the synthesis of structured and unstructured EHR data  at a large scale. The fact that tens of thousands of patients have undergone SARS-CoV-2 testing at major academic medical centers (AMCs) provides an abundance of potential data to perform this analysis but also poses significant challenges from a practicality standpoint. Manual review and curation of patient trajectories and associated testing results is not practical. It is not likely to provide comprehensive or even entirely accurate individual patient records. Rather, triangulation across datasets, including lab measurements, clinical notes, and prescription information, using a scalable digitized approach to extract structured data along with sentiment-surrounded clinical phenotypes and outcomes enables us to efficiently perform this analysis in a timely fashion.pos patients from COVIDneg patients. The abnormalities in coagulation-related tests, including fibrinogen and platelets, were intriguing in the context of literature reporting the occurrence of various clotting phenotypes in COVID-19 patients, including DIC-like consumptive coagulopathies along with more isolated clotting events in the lungs, central nervous system, and other tissues  after the PCR date in few COVIDpos patients with serial measurements, and the development of such outright thrombocytosis was observed with similar frequencies in the thrombotic and non-thrombotic cohorts  or patients who are on specific medication .It is important to note that while we center the study period around the PCR testing date, this date may not correspond to the same disease state of COVID-19 for each individual in the COVIDntervals . AdditioIn summary, this work demonstrates significant progress toward enabling scaled and digitized analyses of longitudinal unstructured and structured EHRs to identify variables (e.g. laboratory results) which are associated with relevant clinical phenotypes (e.g. COVID-19 diagnosis and outcomes). In doing so, we identified trends in lab test results which may be relevant to monitor in COVID-19 patients and warrant both clinical and mechanistic follow-up in more targeted and explicitly controlled prospective analyses.www.mayo.edu/research/institutional-review-board/overview.This is a retrospective study of patients who underwent polymerase chain reaction (PCR) testing for suspected SARS-CoV-2 infection at the Mayo Clinic and hospitals affiliated to the Mayo health system. This research was conducted under IRB 20–003278, ‘Study of COVID-19 patient characteristics with augmented curation of Electronic Health Records (EHR) to inform strategic and operational decisions’. For further information regarding the Mayo Clinic Institutional Review Board (IRB) policy, and its institutional commitment, membership requirements, review of research, informed consent, recruitment, vulnerable population protection, biologics, and confidentiality policy, please refer to pos cohort, we used the date of the first positive PCR test result as the reference date (day = 0). For patients with all negative PCR tests, we used the date of the first PCR test result as the reference date (day = 0). We defined the study period for each patient to be 30 days before and after the PCR testing date. Patients with contradictory PCR test results were excluded for the purpose of this analysis; for example, a positive PCR test result and a negative PCR test result on the same day, or a positive PCR test result followed immediately by several negative PCR test results.We analyzed data from 74,586 patients who received PCR tests from the Mayo Clinic between February 15, 2020 to May 28, 2020. Among this population, 2232 patients had at least one positive SARS-CoV-2 PCR test result, and 72,354 patients had all negative PCR test results. In order to align the data for the analysis of aggregate longitudinal trends, we selected a reference date for each patient. For patients in the COVIDpos cohort among the patients with PCR testing on or before May 8, 2020. In addition, we considered different subsets of the COVIDpos cohort for the analysis of each of the 194 lab tests, due to differences in availability of testing results. For each lab test, we consider the results from patients with three or more observations during the study period.Over 4 million test results from 6298 different types of lab tests were recorded for the patients who received PCR tests in the 60-day window surrounding their PCR testing dates at the Mayo Clinic campuses in Minnesota, Arizona, and Florida. Among these lab tests, we restricted our analysis to 194 tests with at least 1000 observations total and at least 10 observations from the COVIDpos cohort. In order to construct the COVIDneg cohort from the 13,666 COVID-19 negative patients, we apply propensity score matching, which is described in the next section.In the end, there are 246 SARS-COV-2 positive and 13,666 SARS-CoV-2 negative patients that had three or more test results during the study period for at least one of the assays among the 194 lab tests considered. We take this set of 246 COVID-19 positive patients to be the COVIDneg cohort similar in baseline clinical covariates to the COVIDpos cohort, we employ 1:10 propensity score matching , pre-PCR (days −10 to −2), time of clinical presentation (days −1 to 0), and post-PCR phases 1 (days 1 to 3), 2 (days 4 to 6), 3 (days 7 to 9), 4 (days 10 to 12), 5 (days 13 to 15), and 6 (days 16 to 30).We conduct a systematic statistical analysis to identify tests that show significant differentiation among the COVIDpos cohort over that time interval to the mean lab test value in the COVIDneg (matched) cohort over that time window. We only considered  pairs in which there were at least three patients contributing to laboratory test results in both groups. Specifically, for each  pair, we conducted the following procedure:For each lab test and for each of each of our nine pre-specified time intervals, we compared the mean lab test value among patients who underwent at least one such lab test in the COVIDpos and COVIDneg (matched) cohorts during the specified time interval.Compute  averages: We compute the average lab test values for each patient in the COVIDU test in order to test the null hypothesis that the average lab test results for each of the  pairs from the COVIDpos and COVIDneg (matched) cohorts come from the same distribution. In addition, we compute the Cohen’s D statistic as a measure of the effect size.Statistical hypothesis testing: We conduct a Mann-Whitney Once we have the statistics and p-values for each  pair, in order to account for multiple hypotheses, we apply the Benjamini-Hochberg (BH) procedure with FDR controlled at 0.05. The results from the systematic comparisons which met our thresholds for effect size and statistical significance  are shown in We perform a sensitivity analysis to assess whether or not the key findings from the systematic statistical assessment remain the same if we perturb the considered time intervals. In particular, we repeat the statistical analysis with the time intervals shifted forward or backward 1 day for all patients. For the forward shifted sensitivity analysis, the new time intervals under consideration are: pre-infection (days −30 to −10), pre-PCR (days −9 to −1), time of clinical presentation (days 0 to 1), and post-PCR phases 1 (days 2 to 4), 2 (days 5 to 7), 3 (days 8 to 10), 4 (days 11 to 13), 5 (days 14 to 16), and 6 (days 17 to 30). For the backward shifted sensitivity analysis, the new time intervals under consideration are: pre-infection (days −30 to −12), pre-PCR (days −11 to −3), time of clinical presentation (days −2 to −1), and post-PCR phases 1 (days 0 to 2), 2 (days 3 to 5), 3 (days 6 to 8), 4 (days 9 to 11), 5 (days 12 to 14), and 6 (days 15 to 30). For both the forward and backward sensitivity analyses, we apply the same thresholds of effect size and significance , and we compare the results to the original time intervals.From this analysis, we observe consistent results (i.e. comparisons meeting same criteria of significance and effect) on (i) both perturbations in 83 out of 130 (64%) lab test trends identified in pos and 13,666 COVIDneg patients’ clinical notes, structuring this information so that it could be compiled with longitudinal lab measurement and medication information.A state-of-the-art BERT-based neural network  was prevIn particular, we used the BERT model to identify the seven coagulopathy phenotypes mentioned in clinical notes in the Mayo Clinic EHR database, including: deep vein thrombosis, pulmonary embolism, myocardial infarction, venous thromboembolism, thrombotic stroke, cerebral venous thrombosis, and disseminated intravascular coagulation. We validated the performance of this model for these phenotypes on a set of 1000 randomly selected sentences from the clinical notes of the patients in the study population. In  In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.Acceptance summary:The paper analyzes a large EHR-based dataset of coagulation tests in COVID-19 patients to obtain an understanding of the kinetics of COVID-19 associated coagulopathy. By using machine learning to extract patterns data are provided which support that the majority of thrombotic events in COVID-19 patients are not the result of a DIC-like consumptive coagulopathy, and that this only occurs in a small subset.Decision letter after peer review:eLife. Your article has been reviewed by four peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Jos van der Meer as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Coen Maas (Reviewer #1); Jinbo Chen (Reviewer #4).Thank you for submitting your article "Longitudinal laboratory testing tied to PCR diagnostics in COVID-19 patients reveals temporal evolution of coagulopathy" for consideration by The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.eLife but that some conclusions require a modest amount of additional new data, as they do with your paper, we are asking that the manuscript be revised to either limit claims to those supported by data in hand, or to explicitly state that the relevant conclusions require additional supporting data.We would like to draw your attention to changes in our revision policy that we have made in response to COVID-19 (https://elifesciences.org/articles/57162). Specifically, when editors judge that a submitted work as a whole belongs in eLife, either of which would be linked to the original paper.Our expectation is that the authors will eventually carry out the additional experiments and report on how they affect the relevant conclusions either in a preprint on bioRxiv or medRxiv, or if appropriate, as a Research Advance in Summary:The authors analyze a large EHR-based dataset of coagulation tests in COVID-19 patients to obtain an understanding of the kinetics of COVID-19 associated coagulopathy. It is a clear approach and the data are important for the clinics. They used machine learning to extract patterns. Overall the conclusions were drawn in many directions. Nonetheless, the paper asks a potentially important question and is adequately written and presented. The data support that the majority of thrombotic events in COVID-19 patients are not the result of a DIC-like consumptive coagulopathy, this only occurs in a small subset. This should also be the focus of the Discussion.Essential revisions:Patient characteristics:– Please clinically define "pre-existing coagulopathies".– It is a unclear what the comparator cohort adds: In part it is not clear who these patients are. Perhaps the controls should be age/sex-matched patients with other causes of hypoxemic respiratory failure/pneumonia? Or different COVID-19 disease severity subgroups? Otherwise what are we inferring from comparing these two groups?– Ascertained rates of venous thromboembolism are much lower than what others have observed . Could this relate to ascertainment bias in this cohort? Perhaps a push not to perform diagnostic imaging studies lead to lower objective identification of venous thromboembolism than has been generally described in COVID-19?– Patients are identified as COVID-19 positive or negative based on PCR testing. Actually, using the PCR test a positive/negative infection by SARS-CoV-2 is assessed; COVID-19 is the disease that can follow upon infection. This distinction should be made clear, as not all patients tested positive for SARS-CoV-2 infection necessarily develop COVID-19. What was the guideline followed for PCR-testing: displaying COVID-19 symptoms/ contact with infected persons/ other? This should also be explained and included.– Introduction: "… straddling the date of the PCR test.…" Do the authors refer to the first PCR test here, as it is mentioned that multiple PCR tests may have been performed? Please include a definition.Analysis:– The analysis was restricted to patients who had serial (>/= 3) tests done. This could have led to a survival bias which should be acknowledged.pos patients. Why was there such a strong emphasis on longitudinal markers?– This restriction also led to the exclusion of the vast majority of the cohort: 1,192 -> 181 COVID-19– The potential significance of these longitudinal trends are assessed with these tests as far as I can understand, which both only looks to see whether mean/median change was different between the groups, but moreover does not allow for adjustment. Why not use multilevel regression mixture models, longitudinal regression, etc?neg cohort”) is anticoagulant/antiplatelet medication use. Is this covariate positively or negatively correlated to infection and what is the rationale for this? Table 1 comprises more details; however, these add to further confusion. The Table 1 legend states "anticoagulant/antiplatelet use within 30 days/1 year of PCR testing date" i.e. after the PCR test, while the table itself mentions "medication use in the preceding 30 days/1yr", so before the PCR test. Please clarify and adapt.– One of the three covariates used in the regularized logistic regression model to predict the likelihood of a positive infection  and dosing (prophylactic vs. therapeutic)?– Figure 6 is very hard to understand; please find another way to graphically display the findings in a clear manner.Specific statistical comments:1) Cohort identification and description. (a) For the 1.3 million lab tests on 194 assays over the 60-day window: provide the mean/range of number of tests separately for positive and negative patients, and also provide the mean/range of the number of tests for pre- vs. post-index "0" date; (b) For positive patients, the proportion of those whose PCR positive test is the first test in the 60-day window; (c) If at all possible, COVID-19 related info at the first PCR test for all patients; (4) If possible, reasons for hospitalizations at the day=0;2) It is not clear why it is important to assess lab tests in relation to diagnosis. For studying association between lab tests with diagnosis, ideally, the positive cases should use the "time of infection" as time zero which of course is intractable. But before variation in time from infection to diagnosis by PCR tests, the relevance of test results for diagnosis is unclear. Further, because of the propensity score matching, the cohort may not be suitable for assess the diagnosis: if some matching variables are associated with any of the test results, matching will artificially deflate the association. We therefore suggest the focus of this paper on prognosis only. The Abstract indeed focused on the prognosis, but diagnosis was mentioned multiple times in the manuscript.3) BERT method: The accuracy of BERT was established in an unpublished manuscript developed by the same research group. What is the proportion of patients who were classified as "Maybe", "Yes" AND "No"? It is helpful if chart review is performed on 100, say, patients, to validate this algorithm in the study context;4) Tables 3 and 4 indicate that the endpoints were significantly enriched for positive cases who had longitudinal data. But it is possible that patients received more tests because of indications of more negative outcomes. As a result, patients with longitudinal lab tests are representative of all positive patients as indicated by the enrichment. This is an important weakness of the study as this may be a source of bias, making the study results not generalizable. It is important to provide additional information on the comparing characteristics of patients with and without longitudinal tests. It may be worthwhile to provide information on reasons of repeating tests for negative patients. Summary:The authors analyze a large EHR-based dataset of coagulation tests in COVID-19 patients to obtain an understanding of the kinetics of COVID-19 associated coagulopathy. It is a clear approach and the data are important for the clinics. They used machine learning to extract patterns. Overall the conclusions were drawn in many directions. Nonetheless, the paper asks a potentially important question and is adequately written and presented. The data support that the majority of thrombotic events in COVID-19 patients are not the result of a DIC-like consumptive coagulopathy, this only occurs in a small subset. This should also be the focus of the Discussion.We thank the editors for their advice to focus the key conclusion of the paper. Accordingly, we have revised the manuscript’s Discussion section to reflect this suggestion. We find that the updated data for this manuscript also support this conclusion that DIC-like consumptive coagulopathy is relatively rare among COVID-19 patients, as shown in updated Table 3.Essential revisions:Patient characteristics:– Please clinically define "pre-existing coagulopathies".We consider the following phenotypes to be coagulopathies: deep vein thrombosis, pulmonary embolism, myocardial infarction, venous thromboembolism, thrombotic stroke, cerebral venous thrombosis, and disseminated intravascular coagulation. We have added a definition of pre-existing coagulopathies to the Introduction, which includes all patients who have at least one of the phenotypes from days -365 to -31 relative to the PCR testing date. In Table 1 of the revised manuscript, we have now added in the medical history of thrombotic events for the patients in the COVID-19 positive, negative, and negative (matched) cohorts.– It is a unclear what the comparator cohort adds: In part it is not clear who these patients are. Perhaps the controls should be age/sex-matched patients with other causes of hypoxemic respiratory failure/pneumonia? Or different COVID-19 disease severity subgroups? Otherwise what are we inferring from comparing these two groups?pos cohort on a variety of clinical characteristics which may be confounding variables for thrombotic events and related lab tests that we observe during the study period. The clinical covariates that we controlled for initially included demographics , hospitalization status, and anticoagulant/antiplatelet use, which can be viewed as a proxy variable for medical history of thrombotic events. In this revision, we have explicitly added in medical history of thrombotic events as clinical covariates to balance on, as shown in the updated Table 1 of the revised manuscript. We have added a few statements describing the updated COVIDneg (matched) cohort to the last paragraph of the Introduction.The comparator cohort was selected to match the COVID– Ascertained rates of venous thromboembolism are much lower than what others have observed . Could this relate to ascertainment bias in this cohort? Perhaps a push not to perform diagnostic imaging studies lead to lower objective identification of venous thromboembolism than has been generally described in COVID-19?pos patients. In particular, our study is not restricted to severe MODS or ARDS patients, as has been the focus of previous studies such as Helms ICM 2020, Middeldorp preprints, or https://pubmed.ncbi.nlm.nih.gov/32353745/ etc. We expect that this is the reason for the differences in venous thromboembolism rate reported across the different studies.Our study presents, for the first time, fine grained temporal resolution into lab results change in all COVID– Patients are identified as COVID-19 positive or negative based on PCR testing. Actually, using the PCR test a positive/negative infection by SARS-CoV-2 is assessed; COVID-19 is the disease that can follow upon infection. This distinction should be made clear, as not all patients tested positive for SARS-CoV-2 infection necessarily develop COVID-19. What was the guideline followed for PCR-testing: displaying COVID-19 symptoms/ contact with infected persons/ other? This should also be explained and included.As recommended by the reviewers, the distinction between SARS-CoV-2 infection and the COVID-19 disease onset and progression has been clarified in the revised paper. The guidelines followed for PCR-testing included all of the following – displaying COVID-19 symptoms, contact with infected persons, and other factors such as potential work-place/community exposure as well as underlying conditions predisposing to SARS-CoV-2 infection. This has been noted in the second paragraph of the Introduction section of the manuscript.– Introduction: "… straddling the date of the PCR test.…" Do the authors refer to the first PCR test here, as it is mentioned that multiple PCR tests may have been performed? Please include a definition.pos cohort, we center the two month observation period around the date of the first positive PCR test for SARS-CoV-2, and for the COVIDneg cohort, we center the two month observation period around the date of the first PCR test for SARS-CoV-2.We have added a statement to the Introduction, clarifying this point. For the COVIDAnalysis:– The analysis was restricted to patients who had serial (>/= 3) tests done. This could have led to a survival bias which should be acknowledged.We thank the reviewers for raising this point. We have added in a statement on potential survival bias to the Discussion section. In particular, we now provide a summary of the clinical characteristics of the study population vs. all patients with PCR tests during the same time period in Table 6. We have also included an additional table characterizing the key lab tests and their availability for the study population vs. all patients (see Tables 7-8).pos patients. Why was there such a strong emphasis on longitudinal markers?– This restriction also led to the exclusion of the vast majority of the cohort: 1,192 -> 181 COVID-19The need for longitudinal data on the testing results, while constraining the study population size greatly, enables us to provide a fine-grained temporal resolution of COVID-associated coagulopathy for the first time. In addition, we consider this study population to be of high clinical interest because these patients are highly enriched for severe thrombotic events during the study period (see Table 4). We have added this in as a point in the seventh paragraph of the Discussion section.– The potential significance of these longitudinal trends are assessed with these tests as far as I can understand, which both only looks to see whether mean/median change was different between the groups, but moreover does not allow for adjustment. Why not use multilevel regression mixture models, longitudinal regression, etc?pos population prior to Day 0, we were unable to train an accurate predictive model using the available data. However, we expect that as additional data becomes available this is a promising area for future research, and we have added a note on this in the Discussion section.Thank you for the suggestion. We considered logistic regression for this analysis, to predict the presence/absence of thrombotic events using the longitudinal lab testing information available for each patient prior to the PCR testing date Day 0. However, due to large heterogeneity in lab tests conducted for the study population, and the relative sparsity of lab testing data available for the COVIDneg cohort”) is anticoagulant/antiplatelet medication use. Is this covariate positively or negatively correlated to infection and what is the rationale for this? Table 1 comprises more details; however, these add to further confusion. The Table 1 legend states "anticoagulant/antiplatelet use within 30 days/1 year of PCR testing date" i.e. after the PCR test, while the table itself mentions "medication use in the preceding 30 days/1yr", so before the PCR test. Please clarify and adapt.– One of the three covariates used in the regularized logistic regression model to predict the likelihood of a positive infection . We hypothesize that the reason for this is that patients who are on anticoagulants/antiplatelets generally have more underlying conditions and thus take more precautionary measures to avoid SARS-CoV-2 infection. In addition, Table 1 shows anticoagulant/antiplatelet use in the 30 days/1 year prior to the PCR testing date. We have revised the name of this covariate in Table 1 for clarification.Figures:– Figure 2A: Why are no data points provided for the -30 to 0 days of the COVID negative patients while these data are shown in Figure 2B and E?pos cohort had Fibrinogen lab test results reported during this time range. For all of the temporal trend plots, we only show data points for which there are at least 3 patients in the COVIDpos and COVIDneg cohorts. We have clarified this in the legends for the updated Figures 1, 2, and 3, and 4.In the revised manuscript, Figure 2A has been renamed to Figure 3A. This plot, which shows the temporal trends of the Fibrinogen lab tests, has no data points between days -30 to 0 because none of the patients in the COVID– The same question as above but then for the APTT and D-dimer data in Figure 2 vs. Figure 3 and magnesium in Figure 1 vs. Figure 3. I guess the cohorts are different between the two figures, maybe this should be stated in the figure legends.The cohorts are different because the previous Figure 3  is restricted to patients with 3 or more lab tests during the study period, while the previous Figures 1 and 2  include patients with 1 or more lab tests during the study period. We have added this statement to the figure legends for clarification.– Figure 3: we agree with the authors that the fibrinogen decline and platelet increase in COVID positive patients is re-emphasized in this manner. Also the increase in magnesium and decrease in alkaline phosphatase seem to stand out. Could the authors comment on this?We agree that the increases in magnesium and alkaline phosphatase do seem significant. We are planning to explore this trend in future analyses.– Could the authors comment on the number of thrombotic events that were radiographically-confirmed?We do not have the radiology reports available in this dataset, so unfortunately we cannot comment on the number of radiographically confirmed events.– Figure 5: for the individual patients could the authors comment on the heparin therapy with regard to the source of heparin (LMWH vs. UFH) and dosing (prophylactic vs. therapeutic)?We have added annotations to Figure 5 indicating which medications were prescribed for prophylaxis. In addition, we have updated the Results section to include a note on these results.– Figure 6 is very hard to understand; please find another way to graphically display the findings in a clear manner.In order to keep the paper focused on clinical data, we have now excluded the previous Figure 6, which was based on single cell data.Specific statistical comments:1) Cohort identification and description. (a) For the 1.3 million lab tests on 194 assays over the 60-day window: provide the mean/range of number of tests separately for positive and negative patients, and also provide the mean/range of the number of tests for pre- vs. post-index "0" date; (b) For positive patients, the proportion of those whose PCR positive test is the first test in the 60-day window; (c) If at all possible, COVID-19 related info at the first PCR test for all patients; (4) If possible, reasons for hospitalizations at the day=0;We include Tables 7 and 8 which contain the relevant numbers for (1) (a)-(b); this should hopefully provide some further context. For PCR-positive and PCR-negative patient cohorts, we include the overall number of tests and number of patients with tests; we do this for all the tests in aggregate and 5 coagulation-related tests in particular. Table 7 has these numbers for the PCR-tested patient population; Table 8 has them for the study population of PCR-tested patients with longitudinal test data. We included general demographic characteristics as well in Table 6. We were not able to address (c) further or (d) at this time unfortunately.2) It is not clear why it is important to assess lab tests in relation to diagnosis. For studying association between lab tests with diagnosis, ideally, the positive cases should use the "time of infection" as time zero which of course is intractable. But before variation in time from infection to diagnosis by PCR tests, the relevance of test results for diagnosis is unclear. Further, because of the propensity score matching, the cohort may not be suitable for assess the diagnosis: if some matching variables are associated with any of the test results, matching will artificially deflate the association. We therefore suggest the focus of this paper on prognosis only. The Abstract indeed focused on the prognosis, but diagnosis was mentioned multiple times in the manuscript.We have clarified that the focus of this paper on prognosis rather than diagnosis. Furthermore, only explicit statements “at the time of diagnosis”, “after diagnosis”, or “following diagnosis” are retained in the Discussion and Results section. This should address the key concern from a statistical analysis standpoint above.3) BERT method: The accuracy of BERT was established in an unpublished manuscript developed by the same research group. What is the proportion of patients who were classified as "Maybe", "Yes" AND "No"? It is helpful if chart review is performed on 100, say, patients, to validate this algorithm in the study context;We have added in validation statistics for the BERT model on a randomly selected set of 1,000 sentences with thrombotic event phenotypes (see Table 5). Overall, we observe that the BERT model has an out-of-sample accuracy of 94.7%, recall of 97.8%, and precision of 92.8%. We have added a paragraph to the Materials and methods section about this validation procedure and the results.4) Tables 3 and 4 indicate that the endpoints were significantly enriched for positive cases who had longitudinal data. But it is possible that patients received more tests because of indications of more negative outcomes. As a result, patients with longitudinal lab tests are representative of all positive patients as indicated by the enrichment. This is an important weakness of the study as this may be a source of bias, making the study results not generalizable. It is important to provide additional information on the comparing characteristics of patients with and without longitudinal tests. It may be worthwhile to provide information on reasons of repeating tests for negative patients.pos and COVIDneg (matched) to be similar in their propensity for more severe disease outcomes.We have added in a table showing the demographics of the patients who received PCR testing and our study population of patients who received PCR testing and took at least one lab test on 3+ separate days (Table 6A and B respectively). In addition, we agree with the reviewer that the study cohort of patients with longitudinal lab tests is enriched for patients with more severe disease outcomes. However, in this revision we have added in medical history clinical covariates to control for in the propensity score matching step of this analysis, so we believe that the COVID


Negative attitudes towards vaccines and an uncertainty or unwillingness to receive vaccinations are major barriers to managing the COVID-19 pandemic in the long-term. We estimate predictors of four domains of negative attitudes towards vaccines and identify groups most at risk of uncertainty and unwillingness to receive a COVID-19 vaccine in a large sample of UK adults.Data were cross-sectional and from 32,361 adults in the UCL COVID-19 Social Study. Ordinary least squares regression analyses examined the impact of socio-demographic and COVID-19 related factors on four types of negative vaccine attitudes: mistrust of vaccine benefit, worries about unforeseen effects, concerns about commercial profiteering, and preference for natural immunity. Multinomial regression examined the impact of socio-demographic and COVID-19 related factors, negative vaccine attitudes, and prior vaccine behaviour on uncertainty and unwillingness to be vaccinated for COVID-19.16% of respondents displayed high levels of mistrust about vaccines across one or more domains. Distrustful attitudes towards vaccination were higher amongst individuals from ethnic minority backgrounds, with lower levels of education, lower annual income, poor knowledge of COVID-19, and poor compliance with government COVID-19 guidelines. Overall, 14% of respondents reported unwillingness to receive a vaccine for COVID-19, whilst 23% were unsure. The largest predictors of both COVID-19 vaccine uncertainty and refusal were low-income groups , having not received a flu vaccine last year, poor adherence to COVID-19 government guidelines, female gender, and living with children. Amongst vaccine attitudes, intermediate to high levels of mistrust of vaccine benefit and concerns about future unforeseen side effects were the most important determinants of both uncertainty and unwillingness to vaccinate against COVID-19.Negative attitudes towards vaccines are a major public health concern in the UK. General mistrust in vaccines and concerns about future side effects in particular will be barriers to achieving population immunity to COVID-19 through vaccination. Public health messaging should be tailored to address these concerns and specifically to women, ethnic minorities, and people with lower levels of education and incomes.The Nuffield Foundation [WEL/FR-000022583], the MARCH Mental Health Network funded by the Cross-Disciplinary Mental Health Network Plus initiative supported by UK Research and Innovation [ES/S002588/1], and the Wellcome Trust [221400/Z/20/Z and 205407/Z/16/Z]. We searched PubMed for articles published in English from 1 January 2020 to 20 September 2020 with the following keywords: (“COVID19 vaccine” OR “coronavirus vaccine”) and . Our search found 639 titles. Several previous studies have examined predictors of intent to vaccinate for COVID-19 when it becomes available. Reasons for unwillingness to receive the COVID-19 vaccination when it becomes available centred on concerns about its newness, safety, and potential side effects. Socio-demographic predictors of uncertainty and unwillingness to vaccinate identified to date include female gender and low socio-economic status. However, estimates and predictors of negative vaccine attitudes in general and how these attitudes in turn predict uncertainty and unwillingness to vaccinate in the context of COVID-19 are unavailable.The attitudinal and behavioural barriers to being unsure about receiving a COVID-19 vaccine and not intending to receive one were largely overlapping; 1) did not get a flu vaccine last year, 2) poor adherence to COVID-19 government guidelines, 3) concerns about the unforeseen future effects of vaccines, and 4) and general mistrust in the benefits and safety of vaccines.Mistrust towards vaccines represent a significant challenge in achieving the vaccination coverage required for population immunity. Taken together, there is evidence that groups most vulnerable to falling ill and dying of COVID-19  have more negative attitudes towards vaccines and are less willing to vaccinate against COVID-19. Women, people living with children, and those who show poor compliance with COVID-19 guidelines are also at risk for these negative attitudes and unwillingness to vaccinate. Although we did not find an association between low confidence in government to handle the pandemic with unwillingness to vaccinate, low confidence in the health system to handle the pandemic was associated with more negative attitudes on all but one domain  and increased likelihood of uncertainty to vaccinate against COVID-19. Others have found that inconsistent public health messages and low confidence in government and science are relevant for COVID-19 vaccine refusal. Not everyone who intends to receive a COVID-19 vaccination will be able to do so because of practical barriers such as lack of accessibility and government decisions on the availability of the vaccine, underscoring the importance of improving vaccine attitudes in the general population to improve vaccine uptake amongst those who are offered a vaccine and prevent widening socio-economic health inequalities. Vaccine safety communication to increase public trust by the time a COVID-19 vaccine is available should begin now.Alt-text: Unlabelled box1The long-term success of the public health response to the coronavirus disease 2019 (COVID-19) pandemic will depend on acquired immunity in a sufficient proportion of the population (herd immunity), which is estimated to be 67% for COVID-19 Findings from nationally representative studies suggest unwillingness and uncertainty about receiving a COVID-19 vaccine will be a significant challenge in achieving the vaccination coverage required for population immunity. Early in the pandemic (April 2020), 26% of adults across seven European countries including the UK were unsure or unwilling to get a COVID-19 vaccine when available Concerns identified to date for intending not to receive the COVID-19 vaccine include worries about the newness and safety of the vaccine as well as about potential side effects ,8,10,13.Consequently, there is an urgent need for a more updated and nuanced understanding of attitudes towards vaccines and factors determining vaccine intent in the context of the COVID-19 pandemic in order to tailor public health messaging accordingly 22.1Data were drawn from the COVID-19 Social Study; a large panel study of the psychological and social experiences of over 75,000 adults (aged 18+) in the UK during the COVID-19 pandemic. The study commenced on 21 March 2020 and involves online weekly data collection from participants for the duration of the COVID-19 pandemic in the UK. The study is not random and therefore is not representative of the UK population. But it does contain a well-stratified sample that was recruited using three primary approaches. First, convenience sampling was used, including promoting the study through existing networks and mailing lists , print and digital media coverage, and social media. Second, more targeted recruitment was undertaken focusing on groups who were anticipated to be less likely to take part in the research via our first strategy, including (i) individuals from a low-income background, (ii) individuals with no or few educational qualifications, and (iii) individuals who were unemployed. Third, the study was promoted via partnerships with third sector organisations to vulnerable groups, including adults with pre-existing mental health conditions, older adults, carers, and people experiencing domestic violence or abuse. The study was approved by the UCL Research Ethics Committee [12467/005] and all participants gave informed consent. Participants were not compensated for participation.n = 12). Due to insufficient statistical power, we also excluded individuals who had selected “other” in response to gender (n = 134) and “prefer not to say” on ethnicity (n = 95). Seventeen of these individuals had selected both responses, leaving a total analytical sample size of 32,361. Analyses were cross-sectional as all predictor variables but one  were collected during the same study week as the vaccine module.For the purpose of the current study, we focused on individuals who had started the vaccine module administered from 7 September to 5 October 2020. A total of 32,585 participants met this criterion. We then excluded participants with any missing data on vaccine outcome variables  Scale Uncertainty and unwillingness to vaccinate against COVID-19 when available were based on one item (“How likely to do you think you are to get a COVID-19 vaccine when one is approved?”). Response options ranged from “1- very unlikely” to “6 – very likely”. An ordinal variable was coded: (0) intend to vaccinate (responses of 5-6), (1) unsure about whether to vaccinate (responses of 3-4), and (2) unwilling to vaccinate (responses of 1-2).2.2.2Socio-demographic factors included gender , age group  ethnicity , education  or vocational training, GCSE  or lower [equivalent to education to age 16], and no formal qualifications), income , employment status (not employed vs. employed), area of dwelling , living arrangement , and government's identified key worker status (not a key worker vs. key worker). The latter included people with jobs deemed essential during the pandemic  and who were required to leave home to carry out this work during the lockdown.Participant reports of whether they had received clinical diagnoses of a mental illness  or chronic physical health condition , cancer, or other physical health condition) were used to create two binary variables (yes/no) to indicate the presence or absence of pre-existing physical and mental health conditions.Coronavirus anxiety during the past two weeks was measured using the Coronavirus Anxiety Scale (CAS) Confidence in government and the health service to handle the pandemic were assessed with one question each. Response options ranged from 1  to 7 (lots). Two binary variables were created to compare individuals who had a lot of (5-7) versus low (1-4) confidence in the government and health system.Responses to the question on compliance with government COVID-19 guidelines were on a scale from 1  to 7 (very much so). We analyse this as a binary variable reflecting higher (6-7) vs lower (1-5) compliance. Knowledge of COVID-19 was measured with the question: rated on a 7-point scale from 1 (very poor knowledge) to 7 (very good knowledge). Responses of 1-4 were categorised as very poor/poor compared to very good/good (5-7) COVID-19 knowledge. The presence or absence of having had COVID-19 was categorised as a binary variable . Prior vaccine behaviour was based on two yes/no questions. See Supplemental Table S1 for a listing of all question wording.2.3Pearson correlations were used to determine correlations among the negative vaccine attitudes scales. Ordinary least squares (OLS) regressions were carried out to examine socio-demographic and COVID-19- related predictors of each of the four negative attitudes toward vaccines subscales. We then fitted one multinomial regression model to examine associations of socio-demographic, COVID-19- related factors, negative vaccine attitudes, and prior vaccine behaviours with uncertainty and unwillingness to vaccinate against COVID-19. The outcome variable in this model was coded such that those very likely to vaccinate were compared to those who were i) uncertain about whether to vaccinate and ii) very unlikely to vaccinate. Multinomial regression coefficients were exponentiated and are presented as relative risk ratios (RRR) with corresponding 95% confidence intervals (CI). RRR is the ratio of the probability of an outcome in the exposed group to the probability of an outcome in the unexposed group 2.3.1N = 32,361). Imputation models included all variables used in the analysis, as well as additional auxiliary variables . Substantives results using cases without any missing data  and the imputed sample were similar . To account for the non-random nature of the sample and increase representativeness of the UK general population, all data were weighted to the proportions of gender, age, ethnicity, country, and education obtained from the Office for National Statistics The pattern of missing data in the study sample is presented in Supplemental Table S3. The proportion of missing data ranged from 0.01% for having had COVID-19 to 19.66% for the Coronavirus Anxiety Scale. Multiple imputation by chained equations 2.4The funders had no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication. All researchers listed as authors are independent from the funders and all final decisions about the research were taken by the investigators and were unrestricted. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.3r = 0.38, p < 0.001; mistrust and preference for natural immunity: r = 0.48, p < 0.001) to large . 64% of the sample said they intended to receive the COVID-19 vaccine if and when one becomes available, compared with 23% who were uncertain and 14% who were unwilling  and 3.4 times more likely to have decided against having a COVID-19 vaccine .Demographically, groups at increased risk for uncertainty and unwillingness to vaccinate against COVID-19 were women , those who were keyworkers , and people living with children .Socio-economic factors were also associated with uncertainty and unwillingness to receive the COVID-19 vaccine, with people with lower levels of education more likely to be unwilling and those with lower incomes more likely to be uncertain. Age was unrelated to uncertainty around the COVID-19 vaccine and only slightly related to unwillingness, with adults over 65 more likely to be willing than younger adults (ages 30-49 and 50-64) to get the COVID-19 vaccine.Finally, ethnicity, long-term mental and physical health conditions, and low confidence in government to handle the pandemic were unrelated to intentions relating to the COVID-19 vaccine.4This is the first study to comprehensively describe predictors of negative vaccine attitudes and factors influencing uncertainty and unwillingness to vaccinate against COVID-19 as separate outcomes. Concerningly and congruent with prior work, groups most at risk of mistrust in vaccines are the same groups also at increased risk for illness and death from COVID-19; ethnic minorities Our findings are particularly worrisome given the announcement by the UK government on 5 October 2020 that not only will less than half the population will be able to receive a COVID-19 vaccine, but the vaccine will only be for adults ages 18 and over, certain key workers, the vulnerable, and those over the age of 50 Novel to this study is that we also specifically examined factors that predict uncertainty and unwillingness to be vaccinated against COVID-19 as their own outcomes. Individuals who are uncertain may be a stronger group for potential interventions Substantial work has already been undertaken to develop resources for policy makers and other stakeholders to guide effective confidence-building in vaccines. Technical resources are publicly available from the World Health Organization (WHO),There are several limitations to this study. It is not nationally representative, although it does have good stratification across all major socio-demographic groups and analyses were weighted on the basis of population estimates of core demographics. As the recruitment strategy combined multiple approaches, it is possible that some individuals from the same household participated whose views may have influenced one another, yet data on households was not gathered so it was not possible to adjust for any household clustering that may have arisen. However, as the sample was large and heterogeneous, this is not anticipated to have created any meaningful clusters within the data. Further, we used robust standard errors in our regression models to reduce any bias due to non-independence of observations. Our analyses were also cross-sectional, thus limiting inferences about temporal precedence regarding associations among variables. Despite the diversity of the sample and the rich demographic measures, it is possible that more extreme views on vaccines were not adequately captured or that certain specific sub-groups within the population were not fully represented. Because we lacked statistical power to look in more detail at sub-groups of different ethnicities, our binary representation likely led to an over-simplification of these diverse categories. Although our examination of attitudes towards vaccines in general rather than towards a COVID-19 vaccine specifically is a strength of our study, we cannot be sure to what extent participant responses to vaccine attitudes were influenced by fears of a COVID-19 vaccine specifically. Further, our use of a Likert scale to assess vaccine intentions could have led to central tendency bias. However, our approach of examining predictors both of vaccine refusal and vaccine hesitancy allowed us to explore predictors of central and more specific intentions in detail. Finally, this analysis focused on attitudes towards vaccination at the start of the autumn 2020 as the second wave of the virus was beginning in the UK and before any announcements about success of vaccine trials were made. Future research tracking changing attitudes towards vaccination will be important as this pandemic continues and if and when a vaccination is approved and rolled-out.Our findings suggest widespread mistrust and negative vaccine attitudes amongst the general UK public. Many of the specific groups with the most misinformation about vaccines and who are less likely to vaccinate against COVID-19 are also at highest risk for becoming seriously ill with and dying from COVID-19. Despite calculations that more than two-thirds of the public will need to be vaccinated to bring the pandemic under control,https://www.covidsocialstudy.org/. Statistical code is available upon request from Elise Paul (e.paul@ucl.ac.uk).The COVID-19 Social Study documentation and codebook are available for download at Ethical approval for the COVID-19 Social Study was granted by the UCL Ethics Committee. All participants provided fully informed consent and the study is GDPR compliant.DF conceptualised and designed the study. DF also acquired funding, led the investigation, provided oversight on the methodology, administered the project, provided software and other resources, and supervised the project. Data were curated, validated, and formally analysed by EP. EP created visualisations, wrote the original manuscript draft with input from all authors. All authors reviewed and edited the manuscript. All authors approved the final version of the manuscript and had full access to and verified the data.All authors declare no conflicts of interest.


Sleep and functioning are associated with a risk of early workforce exit. However, patterns of change in sleep and functioning through time have not been investigated using person-oriented approaches to show what features of sleep and functioning are associated with an early exit. We examined the pattern of interactions between sleep and health functioning characterizing homogenous subgroups of employees and their associations with premature work exit. An additional aim was to provide a tutorial providing detailed description on how to apply these models, compared to traditional variable based risk factors. We analyzed data from 5148 midlife employees of the City of Helsinki, Finland, surveyed over three phases . Using repeated measures latent class analyses (RMLCA) we classified people into groups based on their trajectories in sleep and functioning. We identified four longitudinal groups: (1) Stable good sleep and functioning (reference), (2) Persistent sleep problems and good or moderate functioning, (3) Poor functioning with good sleep, and (4) Problematic sleep and health functioning. Compared to group 1, elevated risk was found in all classes with group 4 being the worst. In conclusion, focusing on person-orientated patterns of interactions between sleep and functioning helped produce qualitatively different and quantitatively stronger predictions than using conventional risk factor methodology. Thus, longitudinal person-oriented approaches may be a more powerful method for quantifying the role of sleep and health functioning as risks for premature exit from work. Premature exit from work creates serious health and societal burdens in modern societies, including Finland . TherefoOne issue with all of these studies is that they have employed variable oriented statistical approaches . This crSleep can be characterized in numerous ways and it’s not clear which sleep problems, and in what way, are associated with premature exit from work. Moreover, it is unclear whether the evolution of these problems through time predicts the development of work disability. Findings from studies into the role of sleep duration are somewhat equivocal ,6. The aAs such we decided to apply a person-oriented approach where individual development in sleep and functioning is seen as a process characterized by states that change through time  in a welConsequently, this study had two aims, being partly an investigational study and partly a tutorial on the general approach. The first aim of this study was to examine the 10-year pattern of interactions between sleep and health functioning to characterize subgroups of employees and to determine whether the risk for premature exit from paid employment differs between these subgroups. Our second, additional aim was to provide more details of the method, using our study as an example of person-oriented approaches to RMLCA models to complement and to aid their understanding about when these might be employed in future studies. While the method has been used in other research fields RMLCA methods appear to have been very seldom used in sleep or insomnia research.n = 5148 people; 4204 women and 944 men). Except for missing data there were no exclusion criteria, as all participants were midlife employees at baseline, and we focused on their premature exit from the workforce during the follow-up. We did not have information on other sleep or mental health disorders to use as criteria for exclusion in the surveys.This study is part of the ongoing Helsinki Health Study (HHS) focusing on health and well-being, including health related functioning among employees of the City of Helsinki, Finland ,17. InclThe ethical committee of the Faculty of Medicine, University of Helsinki has approved the study, and the City of Helsinki has provided permission to conduct this study. All participants could choose if they wanted to participate the surveys, and can at any time withdraw their consent.The Jenkins Sleep Questionnaire (JSQ) was used to assess insomnia-related symptoms over the past 4 weeks . The JSQ3 × 23) to 216 (23 × 33).In addition, self-reported sleep duration at baseline was assessed by asking: How many hours on average do you sleep per night during the week? The response options were as follows: five hours or less, six hours, seven hours, eight hours, nine hours, and 10 h or more were classified into three groups: ≤6 (short sleepers), hours, 7–8 h (ref), and ≥9 h (long sleepers). A preliminary latent class analysis at baseline showed that sleep duration did not separate latent classes with an acceptable clarity (results not shown). Therefore, and as sleep duration did not add new information, it was not considered as an indicator variable in RMLCA models  but instead as a potential characteristic of revealed RMLCA groups. Additionally, if it were added in the model, this would have made the models a lot more complex, because measuring sleep duration requires more categories. Using three categories for sleep duration would increase the number of possible response patterns from current 64  because it attempts to measure tiredness and fatigue in the same way as does the JSQ item 4. Second, when the seven other SF-36 subscales were used as indicator variables in a latent class analysis at baseline it was found that the best latent class separation was obtained by using subscales BP, GH, GM, and SF (results not shown). Therefore, we considered only these four subscales in our further analyses as indicator variables to model normal and poor health functioning in RMLCA models. The lowest quartile was used as a cut point to dichotomize the four subscales for the purposes of the latent class analyses. Each original subscale included two to 10 items, and those included in the current models, two to five. Because of the varying number of items, standardized scores were used, as advised . Higher Premature exit from the labor market was the outcome variable. In Finland during 2000–2012 the general retirement age was flexible between 64 and 68 years. Consequently, only retirements before 64 years of age were considered as potential events, i.e., reflecting premature exit from paid employment. In some professions  the minimum retirement age is lower than 64 years. Therefore, only events such as taking an early retirement pension, unemployment pathway to retirement, or disability pension were combined into the outcome variable of premature exit from the labor market.The baseline questionnaire included information on several pertinent potential background confounding factors . SimilarIn addition, intention to retire early was assessed with: Have you considered retiring before normal retirement age? Possible answers were as follows: (1) No, I have not; (2) Yes, sometimes; (3) Yes, often; and (4) I have already submitted a pension application. Self-assessment of the opportunities to continue working until the official retirement age was assessed with: Do you think you will be able to continue working until your normal retirement age for your job?: (1) most likely; (2) not sure; and (3) I do not think so.Premature exit from work is a classic example of time-to-event count data. Time-to-event analyses are not possible within PROC LCA (SAS v9.4). Therefore, the classify-analyze approach with 2 steps was chosen. First, classification of individuals using a person-oriented approach by repeated measures latent class analyses (RMLCA). Second, time-to-event analysis of premature retirements between defined RMLCA groups by Cox’s proportional hazards modelling .3 × 23 = 64 cells as a starting point for each RMLCA model. Thus we had 64 potential response-patterns or ’trajectories’. All these response patterns form a matrix or array of response patterns. If we added, for example, even one additional indicator in our model, the number of possible response patterns would increase to 512 (23 × 23 × 23), and adding a fourth one, the number would be 4096 (23 × 23 × 23 × 23), etc. Including 7 indicators into the model at the same time would mean that we would end up with more than 2 million possible response patterns to analyze . If the indicator variables had more than two categories, the numbers would also be notably higher. As pointed out in the measurements section (see above) there were three dichotomized indicators of insomnia-related symptoms  and four dichotomized indicators of health functioning from SF-36 . Consequently, altogether there were 3 × 4 = 12 possible RMLCA models. These can be seen as 12 different operationalizations of one model where the two underlying constructs are insomnia and health functioning. For each of these models, a series of consecutive solutions with different numbers of latent classes (from one to seven) were estimated with SAS PROC LCA (v9.4). Individuals were assigned to the best-fitting class based on their probability of class membership. Only those solutions where homogeneity and latent class separation were considered to be sufficiently good enough for reliable classification were considered in this analytical step.At the first analytical step, RMLCA, was used to fit models of change in sleep and health functioning over the three phases of data collection. The approach is similar to growth mixture modelling in the sense that each latent class is associated with a characteristic vector of responses over time . HoweverWe aimed to analyze whether identified RMLCA groups  were associated with the risk of premature exit from work. Therefore, the second analytical step was a time-to-event analysis of premature retirements between the identified RMLCA groups. That was performed by Cox proportional hazards modelling. Because a person’s age is an obvious determinant of retirement, age was used as the time-scale in the Cox models instead of time-on-study . BecauseBaseline background characteristics of the study participants are shown in RMLCA Models of Change in Sleep and Health Functioning Over Three Phases of the Study.Altogether 12 possible combinations of two  indicator variables (four indicators of functioning × three indicators of sleep) were analyzed. In other words, we tried to define different patterns of movements in and out of problem categories of sleep and functioning across study phases. For each model, a series of different solutions (1–7 latent classes) was estimated and the best of them was selected considering the balance between model’s absolute fitness  and parsimony, latent classes’ homogeneity (reliability of posterior class membership probability > 0.700), and their interpretation.The best solution was always considered to be a four-class solution. In In our additional sensitivity analyses using PCS and MCS in place of the subscales (6 models/figures), the broad picture of the results were very similar. Thus, we identified patterns of change in each model that can be described as subgroups of having consistently poor functioning and poor sleep, consistently good functioning and sleep, poor functioning but good sleep, and good functioning but poor sleep. For specific insomnia symptoms, there was an indication of different levels of problems, but the patterns of change could be classified similarly. Due to the problems described in the methods section, and due to our aim to increase our understanding about the more specific risk groups and patterns of change in sleep and functioning, we preferred to retain the subscales in our main models.Importantly, across different models the interpretation of defined latent classes in acceptable models were relatively similar. Latent Class 1 (LC1) was labelled as the reference class (ref.) and was characterized by the absence of any kind of sleep problems throughout all study phases. Latent class 2 (LC2) was named as persistent sleep problems. The class was characterized by high probability of sleep problems (sometimes with increasing trend) with absence (or low probability) of poor health functioning. Latent class 3 (LC3) was named as poor health functioning and was characterized by stable (or increasingly) high probability of poor health functioning with absence or low probability of sleep problems. Latent class 4 (LC4) was named as problematic sleep and health functioning and was characterized with stable (or increasingly) high probability of poor functioning both in sleep and health indicators. The prevalence of membership in the latent classes in different acceptable RMLCA models varied as follows: LC1 (48.0–73.0%); LC2 (5.2–17.9%); LC3 (10.8–28.2%); and LC4 (5.0–14.6%). Illustrations of the latent class profiles for interpretation are shown in Age-at-Event Analysis for Premature Retirement Events between Different RMLCA Groups. Across the study phases 2065 retirements took place. Of these 1702 were statutory old age retirements and 363 were caused by other reasons, mainly disability pension events. These 363 retirements were considered as adverse outcome events in age-at-event analyses using Cox’s proportional hazards modelling to compare the risk of premature retirement events in different latent classes in the RMLCA model. Individuals were assigned to groups indicated by the model’s latent classes using the maximum probability assignment rule. Each individual was assigned to the group indicated by maximum posterior probability. The reliability of this classification is shown in The results of these age-at-event models was that all models were statistically significantly associated with unwanted retirements events. The strength of the association depended on RMLCA model indicators, type of the RMLCA class and number of latent classes in the model. The strongest associations were found by models with indicators of SF-36 subscale GH with difficulties in initiating sleep or nonrestorative sleep. Additionally, strong associations were found in a model with SF-36 subscale BP and difficulties in initiating sleep, as well as in the model with SF-36 subscale SF and difficulties in initiating sleep. Interestingly, models with indicators of mental functioning  revealed the weakest  associations with difficulties in initiating sleep. The strongest risk (independent of model indicators) was always found among members of latent class 4 , the group with a relatively stable and high level of problems in both health functioning and sleep. The fully adjusted HRs for premature exit from work in this group varied between 2.19 and 5.17 when compared with the reference group. The next strongest risk was found among members of latent class 3 , the group with prevalent problems in different dimensions of health functioning which either were at a stable level or were increasing during the study phases but almost without sleep problems. The fully adjusted HRs for premature exit from work in this group varied between 1.64 and 3.20 when compared with the reference group. Class 2 (persistent sleep problems), the group with high probability of sleep problems which were either relatively stable or increasing during study phases but who were characterized by only a low probability of health functioning problems were next. HR estimates in this group varied between 1.18 and 2.96. The above-mentioned Cox models with the lowest and highest HR estimates of each latent class are shown in The RMLCA groups were built using insomnia-related symptoms and SF-36 health functioning subscales as latent class indicators. Their predictive power remained even after adjustments for several pertinent covariates. However, because of obvious over adjustment, self-reported sleep duration was not included among covariates. Therefore, we analyzed baseline self-reported sleep duration distributions across RMLCA groups. Prevalence of baseline sleep duration groups are shown in p < 0.0001) and it was also always higher than in the group with LC3 poor health functioning (p < 0.0001). With one exception it was also always higher than in LC2 persistent sleep problems (p < 0.010). The prevalence of short sleepers was lowest in the reference group, which differed significantly from all other groups except in four models from LC3 poor health functioning (p < 0.400). In addition, short sleep duration differed less consistently between LC2 and LC3 . Long sleep duration did not generally differ between RMLCA groups. In only two cases, among all accepted model solutions, there was a statistically significant difference found in the proportion of long sleepers.There was a crude positive association between the predictive power of RMLCA groups and the prevalence of short sleepers at baseline. The highest prevalence of short sleepers was found among the group with Problematic sleep and health functioning (LC4), which was the most powerful estimator of premature exit from work. The prevalence of short sleepers (>40%) in that group was about 2.2 times higher than in the reference group . Conversely, the association with long sleep was always statistically significant .In a conventional variable-oriented approach the aim of the analyses would be to test whether sleep and health functioning variables (which were used as indicator variables in RMLCA) were associated with the outcome in a statistically independent way. The fully adjusted Cox models with baseline SF-36 subscales and insomnia-related Jenkins items as risk factors for premature retirement suggested that in most cases they were. However, the strength of the association was weaker than that found in person-oriented approach between RMLCA groups and the outcome. Variable oriented analyses indicated that HR estimates for SF-36 subscales varied between 1.04 and 2.04. Statistically non-significant estimates were also found in models with the GMH subscale. The HR estimates for Jenkins insomnia-related variables varied between 1.52 and 1.95. The interaction term  was always statistically non-significant. When sleep duration was added into the fully adjusted models it was found that in all models the effect of short sleep was always statistically non-significant . The next highest risk was found in a subgroup characterized by stable or increasingly high probability of poor health functioning with concurrently absent or low probability of sleep problems . On average a somewhat lower, although not statistically significantly, risk was found among a subgroup characterized by constantly high or increasing probability of sleep problems with concurrently good health functioning .To the best of our knowledge, previous studies of premature/disability retirement have all used variable-oriented approaches analyzing relations between variables and assuming that these relations apply across all individuals . InsteadSickness absence as a generic indicator of overall health and work-related disability has been shown to be associated with future health outcomes ,31. AlthInsomnia-related symptoms have been repeatedly shown to be associated with disability retirement  and sickPrevious studies have suggested that insomnia-related symptoms have the strongest effect on sickness absence when insomnia symptoms are coexisting with short sleep duration . ExtremeFuture studies could aim at addressing the changes in sleep and functioning across more phases all preceding the retirement event, or focus also on the patterns of changes after retirement. Another approach could be to examine joint development of sleep and functioning by fitting dual trajectory models with continuous variables, and using the group membership as a determinant of later retirement events.Our study has limitations. First, as this study is observational, no causal inferences can be made. Thus, we show associations between the identified latent groups and age at exit from paid employment during the follow-up. More specifically, our design using RMLCA to identify patterns of change in sleep and functioning over 10 to 12 years, and examining premature exit over the three time points, does not allow for causal predictions to be made. There was a relatively large attrition among participants who answered all three questionnaires  resulting in missing information on some sleep or SF-36 variables, i.e., on the indicator variables of the RMLCA models. If this attrition is not completely random it may have influenced the composition of RMLCA subgroups. However, we had seven indicators among which we analyzed 12 possible RMLCA models and found four-class solutions to be the best models with relatively consistent interpretation across all models. If the attrition were selective picking out one or few indicators but not others it would have produced some deviant  latent classes. Because that was not found, we believe that the attrition did not have meaningful effects on our models. In addition, we analyzed one of the models including all individuals . We found only very slight differences in the class prevalence (0.1% to 0.7%), and classification error very slightly increased  without any practical meaning when compared with the model solution with individuals with complete information. Consequently, we think our results are statistically reliable. We acknowledge that there are different ways to handle missing data, such as maximum likelihood estimation. None of the techniques for missing data is without limitations, as compared to actually having complete data (no missing responses). As these data appeared to be largely missing at random, any technique to handle missing data is, however, unlikely to notably change the results. Still, we acknowledge that due to missing data, we are left with a degree of uncertainty. We should also consider generalizability of these findings. This cohort was female-dominated and represented only the municipal sector and their largely female midlife employees. Any further generalization to other employment sectors, countries or different age groups should be done with extreme caution. Finally, it needs to be noted that there is no ‘absolute’ test to decide the best indicators but the decisions by necessity have to be made by the researchers . FinallyThis study also had some strengths. First, we had a large sample of midlife employees, who were all employed at baseline. This enabled us to follow them up for their risk of early exit from workforce during the 10 to 12 year follow-up. The same individuals responded to the surveys three times, and survey items at each time point were identical. Thus, we could apply the RMLCA and study the developmental patterns of change in sleep and functioning, and how they are associated with early exit. The measures of insomnia symptoms and health functioning were validated and have been used in numerous studies for decades ,21. As sPerson-oriented repeated measures latent class analyses (RMLCA) can be used to compliment traditional variable-oriented methods, and pinpoint risk groups that were previously missed. In the present analyses looking at premature exit from the workforce of municipal employees of the City of Helsinki we found that the associations gleaned from this method may provide both qualitatively different and quantitatively more comprehensive view and show better predictive strength to conventional risk factor methodology. Workers at the City of Helsinki are likely at heightened risk of early workforce exit particularly when they have poor or worsening sleep and/or health functioning. It would be very useful if this analysis could be repeated in a study with more than three time points.