ClinicalTrials.gov and PubMed to identify articles linked to COVID-19 studies, and developed criteria based on the trial phase, intervention, location, and record recency to develop a prioritized list of result publications. We found 760 articles linked to 419 interventional trials , with 418 identified via abstract-link in PubMed and 342 via registry-link in ClinicalTrials.gov. Of the 419 trials publishing at least one article, 123 (29.4%) have multiple linked publications. We used an attention score to develop a prioritized list of all publications linked to COVID-19 trials and identified 58 publications that are result articles from late phase (Phase 3) trials with at least one US site and multiple study record updates. For COVID-19 vaccine trials, we found 69 linked result articles for 40 trials . Our method allows for the efficient identification of important COVID-19 articles that report results of registered clinical trials and are connected via a structured article-trial link.In response to the COVID-19 pandemic many clinical studies have been initiated leading to the need for efficient ways to track and analyze study results. We expanded our previous project that tracked registered COVID-19 clinical studies to also track result articles generated from these studies. We conducted searches of  ClinicalTrials.gov (CTG) registry, known as regCOVID.2 The framework uses data science methods that computationally identifies COVID-19 clinical studies using a keyword search. The framework also uses a computerized code to regularly monitor and analyze key features relating to COVID-19 interventional trials, observational studies, and patient registries registered at CTG.The COVID-19 pandemic led to the initiation of thousands of clinical studies testing various interventions and studying the natural course of the disease. For researchers or the public, it can be difficult to navigate and organize a large number of such studies. We previously created a framework for monitoring registered COVID-19 studies using 3 A completed study with one or more study results journal articles provides the most value to researchers and the public. Poor information about study status or study results may lead to reduced public trust in clinical trials enterprise.4A study may publish three types of information: (1) registration data at study initiation , (2) basic summary results at study completion , or (3) an article with well commented full study results . Prior analyses of phase-2-or-higher interventional trials indicate that only 27.8% publish a study result article.3 Since the total amount of all published COVID-19 articles may be overwhelming, we propose focusing only on articles that are linked to formally registered studies to facilitate an effective review of COVID-19 scientific literature. Unlike many efforts that use predominantly manual review to provide the public with an overview of trials and their results, we use a computational clinical research informatics approach to assess which COVID-19 studies are publishing, what they are publishing and when.5 We understand a reader may have limited time to read and review articles or abstracts and therefore the purpose of this research is to create a system to prioritize which articles to read to best understand the current state of clinical trial research for COVID-19. Our computerized processing script can also be generalized and applied to other conditions.In this study, we extended our regCOVID monitoring project to now identify study result articles that are linked to registered COVID-19 trials.https://github.com/lhncbc/r-snippets-bmi/tree/master/regCOVID/regCOVIDpublications) includes our computer code, supplemental files, analysis results and a detailed web-based results report.6 We also refer to the project using a short name of regCOVIDpub. Throughout methods and results, we reference supplemental files on the project repository by the file name. The script is written in R language. For result reports, we use R Markdown framework. For most analyses, the repository will offer monthly refreshed results.Our project repository  and registered at CTG.For the first step we retrieved all COVID-19 studies  database developed by researchers at Duke University.7 The AACT database is created by parsing the XML study data from CTG.7 We used the ‘result_reference’ XML field within the study record. Using prior knowledge that some result_reference articles are incorrectly labelled as such, we used article publication date to remove misclassified articles . See this prior publication for details. 3 We then linked the results publications found in the CTG study records to the PubMed abstract to identify key details about the article, such as article title and type. For context, a prior study on a set of 8 907 trials completed between 2006 and 2009 found that 7.3% of trials tend to have at least one registry-linked result article.3Registry linked result articles are those included in the study record on the CTG registry. We used the Aggregate Analysis of 3Abstract linked articles are those where authors of trial result articles follow guidance of the International Committee of Medical Journal Editors and reference properly the relevant trial identifier in the article abstract. This reference is processed by PubMed and turned into searchable article metadata . We retrieved abstract linked articles by a metadata search in PubMed as articles where the article secondary identifier contained a CTG identifier (NCT ID) of a COVID-19 trial. For context, the same previously mentioned prior study found that 23.3% of trials tend to have abstract-linked result articles.We combined the lists of publications from these two search methods to generate a master list of linked COVID-19 articles  in a trial and discussed in a publication contributes to how significant the publication is in the research landscape. Interventions must progress through the phases of interventional trials (phase 1/2/3) to receive regulatory approval for a given indication. Different interventions were studied for COVID-19 and advanced to different phases. Therefore, we created an intervention significance score for each intervention studied. The score was calculated by assigning phase-based numeric value based on whether an intervention has a trial in a given phase and adding 0.01 for each trial in that phase to add significance for the existence of multiple trials in that phase. For example, tocilizumab had 12 phase 3 trials so that would add 3.12 to the intervention score . The higher the score the more significant the level of study of the intervention in the COVID-19 research landscape. For trials that combined two phases, we counted the trial as being of the higher phase .Our goal was to generate a ranked list of publications with the most significant publications appearing on top. We used a construct of an attention score that gives the most significant publications higher values. The score is based on the recency of the publication, the phase of the trial, the intervention significance score, the number of times the trial record has been updated , and whether the trial includes a US site. In other words, publications ranked higher if they were recent, from a later phase trial, involved a significant intervention, involved a CTG study record that had been updated multiple times and had at least one US site. For scoring purposes, if a trial was a combination of two phases, such as a phase 2/3 trial, we considered it under the higher phase (phase 3 in this example case).We also retrieved article type from PubMed and gave publications that were not study result articles, such as protocols or editorials, less significance, and therefore lower attention scores, than study result articles.In the final ranked publication list, we also present to the user further important publication and study metadata that are not input parameters for the calculation of the attention score. This information includes, the study sponsor, the journal where the publication was published, and whether study results were deposited on CTG as part of the trial record. This information can be seen in the 1 Similar to, the overall set of COVID-19 studies, we analyzed the vaccine trials based on key trial and publication features and generated attention scores for each publication associated with a trial of a COVID-19 vaccine.Due to the great importance and interest in vaccine trials for COVID-19, we looked specifically at a subset of COVID-19 vaccine interventional trials. The subset was developed by searching for the term vaccine in the trial’s title .We also analyzed both observational studies and registries. Similarly, to interventional trials, we identified both abstract and registry linked publications and assigned attention scores based on the recency of the publication, the number of study record updates and whether or not the study included a US site. Phase is not relevant for observational studies and registries.6 Repository history mechanism and formal data releases allow retrieval of any data release over time. The repository contains a report generated using an R notebook framework (computer code combined with user friendly result outputs). In addition to the report, important results are available as separate files in spreadsheet format. Such separate files are referred to in the results prefixed with ‘regCOVIDpublications_’.All analytical results presented below were based on a query date of 15 August 2021. We plan to publish refreshed results at the study repository.As of the query date (15 August 2021), we identified and analyzed a total of 2 669 recruiting, active or ended (completed or terminated) COVID-19 interventional trials . On the trial level, a total of 419 trials  have at least one linked result article. 123 trials have multiple publications, with 63 trials having published three or more articles.The total number of trial-article-link-type combinations was 760, with 418 (55.0%) articles identified via abstract link and 342 (45.0%) identified via registry link. 11 (1.5 %) articles overlapped and were identified via both link types. Since the same article can be linked to multiple trials , we found that there were 679 distinct publications linked to all included COVID-19 interventional trials.It is important to consider the level of effort  to link a publication to a trial. Abstract linking is easier and faster because the article author can simply state the NCT ID in the abstract and the article-study linkage is auto-generated thanks to the automated processing of PubMed abstracts. The majority of result articles (55.0%) were abstract-linked. On the other hand, registry linking requires update of the record in CTG by either XML file submission though their application protocol interface or by using CTG’s web-based data entry system . Per our methodology, 964 registry-linked articles were removed as incorrect, misclassified result articles .Using our computerized approach, we identified 3 295 interventions used in COVID-19 interventional trials. Of these 3 295 interventions, 549 had at least one publication connected to a trial. While Hydroxychloroquine was the intervention with the most publications (81) and highest intervention score (8.301) based on the number of trials and the breadth of the phases the trials covered, Convalescent Plasma was the intervention with the most distinct sponsors studying it (103). 708 interventions had at least one phase 3 (or phase2/3) trial. While multiple vaccine candidates have progressed through each phase, the intervention significance score is lower than most other interventions that progressed to a similar phase since the volume of trials studying the vaccine candidate is usually limited by the fact that only the developer (and select co-sponsors) are studying the vaccine candidate. For example, the vaccine candidate mrna-1273 from Moderna has 9 total trials  with an intervention significance score of 6.09, which is lower than most other interventions that also proceed to phase 3  with at least one US site and had multiple study record updates). Of the 760 trial-publication combinations, 234 (30.8%) were phase 3, 186 (24.5%) had at least one US site, and 528 (69.5%) had multiple study record updates.The use of the attention score and prioritizing certain facts about a trial and publication greatly reduces the list of all publications to a manageable list of publications for readers to review (58 publications compared to 760 publications). Assuming a researcher may spend two minutes on each abstract, reviewing the full list versus the prioritized short-list results in a difference of 23.1 hours in terms of total review time.For the subset of 290 total COVID-19 vaccine trials (as of the query date), we found at least one publication for 40 (13.9%). For those 40 trials, we identified 69 trial and publication combinations, with 55 (79.7%) being abstract linked. Due to the urgency and significant public interest in COVID-19, we observed significant result articles published for trials that are formally ongoing, such as the Pfizer phase 2/3 trial for its vaccine candidates BNT162b1 and BNT162b2, which has already published four result articles, but does not have a listed completion date until 2 May 2023.As of the query date, no vaccine trial has formally deposited basic summary results to the CTG registry. Legal mandate allows for one year to do so for applicable US trials after the formal completion of the trial. This shows that vaccine trial sponsors may prefer publishing a result article in an academic journal as opposed to registry result deposition to communicate the results to the public. Although, this imbalance is also impacted by legal rules  governing registry result deposition, namely, the official primary completion date and one year allowed legal time window after this date greatly influence when registry result deposition is performed.Because of the mostly computerized nature of our analysis, we executed the same analyses on COVID-19 observational studies and registries. We found 485 result articles for observational studies. In contrast to interventional trials, more publications were registry linked  than abstract linked (205 publications). On the study level, 246 COVID-19 observational studies  had at least one result publication.We found 111 result articles for registries and similarly to observational studies, the majority were registry linked . On a study level, 53 COVID-19 registries  had at least one linked study result article.Unlike applicable interventional trials, US law does not mandate registration of observational studies or registries. A lack of a registration mandate does not allow for the determination of the proper denominator . The whole method of using abstract link or registry link (relying on NCT ID) naturally fails for unregistered studies. Researchers must rely on traditional PubMed searches to discover result articles of unregistered studies.3 A systematic review by Bashir et al. from 2017 found that a median of 23% (ranging from 13% to 42%) were linked to a result article.8 With much increased public attention during the COVID-19 pandemic, we were motivated to find out what would be the percentage for COVID-19 studies. Our results, as of the query date, show that only 15.7% of COVID-19 interventional trials have a linked result article. However, it is too early to arrive at a formal number due to the relatively recent completion date  of many trials.There are several prior analyses that report on how many studies provide results to the public. Huser et al. analysis from 2013 reported that 27.8% of analyzed interventional trials had published a linked result article.Our methodology quickly identified result publications for prominent trials, such as trials involving vaccines approved in the US. Targeted review of those studies shows that such studies updated their CTG record frequently, which gives more confidence in the study metadata and study status . In terms of paring trials with their result-reporting journal articles, the majority of linked result articles for interventional COVID-19 trials were found via abstract-link (55.0%), perhaps due to the easier practice of including the NCT ID in the article abstract.The main advantage of our approach is offering researchers and the public a structured overview of literature with valuable metadata that combines information from scientific literature (PubMed) and clinical trial registry (CTG). It allows researchers to sort or aggregate articles based on various useful parameters . Such capability is not possible with existing tools. Neither PubMed search nor clinical trial registry allow for review that would combine data from both sources. It allows for an overview of the clinical research in a given disease generated though automated computer script. For example, a review of all articles for a given intervention (such as hydroxychloroquine) could reveal if there is a consensus opinion on its efficacy or if there is a divide and more research is needed. In the case of hydroxychloroquine, a review of five results articles from four clinical trials in the US (on the prioritized short list) all expressed that the intervention was ineffective. A review of a full article master list  would show a total of 81 articles from 38 trials studying hydroxychloroquine , making it difficult for interested parties to know the outcome of the trial. An even more extreme case of minimal trial information are trials with no linked result articles and zero updates  to the CTG study record . Our project, regCOVID, is the first to utilize number of registry record updates (and the type of this update) as a novel, computed study metadata construct to further categorize studies by level of activity. This can be helpful in comparing studies with identical official study status and improve the prioritization of result publications stemming from these studies.As an alternative to publishing study results through an article, many studies chose to distribute study results by depositing them on CTG. A total of 61 trials deposited basic summary results. Within those, 35 trials only did registry result deposition and have no study result article and the remaining 26 trials did both result deposition and published a result article.prior to trial initiation, (2) after trial initiation and prior to completion (during), and (3) after trial completion. For the set of all COVID-19 trials the breakdown was 2 237 (44.9%) prior to trial initiation, 2 157 (43.3%) during the trial, and 584 (11.7%) after the trial completion. In comparison, when considering all studies initiated in 2020 (not restricted to COVID-19), 59.3% registered prior to starting, 27.1% registered during the study and 13.6% registered after the study was completed. The comparison shows that COVID-19 studies are more likely to register late .As part of our analysis, we found that trials register at three different points in time: (1) 9 In Publication of study results, including peer review, can be a complex and lengthy process. Prior studies indicate that it can take 21 months.NCT04470427) which has an official primary completion date of 27 October 2022, the study result article was published in December 2020 (PMID: 33378609). This situation is, in fact, quite common. We found 364 trial result articles linked to 164 COVID-19 trials that are not formally completed as of the query date.While primarily clinical trials publish study results articles after the formal trial completion date, for high profile trials it is not uncommon to see the opposite situation. During an ongoing pandemic, timely publication of results is important. For example, for the widely known trial regarding the Moderna COVID-19 vaccine  specified a termination reason that helped explain why the trial was terminated. Most often, COVID-19 trials were terminated due to the inability to recruit and enroll participants. Other termination reasons were: intervention safety concerns, futility of the intervention, or availability of results from other trials making trial continuation unnecessary.While manual review of abstracts of result publications was out of scope, we understand the potential presence of publication bias that may lead some trials to not formally publish results in a medical journal. For example, with reports of clearly terminated plans for further vaccine developments by some sponsors, a lack of result articles for certain trials and vaccine candidates hints at possible publication bias in vaccine trials.5 Another study tracker is published by the NIH.10 While it was out of scope to manually curate a sophisticated list of COVID-19 studies, or do a comprehensive review and comparison of our results to manual COVID-19 study trackers, we did compare the vaccine subset of COVID-19 studies identified through our methodology with those identified by the New York Times and NIH COVID-19 vaccine study trackers. Our motivation was to see how inclusive our methodology was. Our computerized approach study identification methods identified 25 of 32 phase 3 vaccine trials included in the New York Times vaccine tracker and included 5 of 6 trials present in the NIH study tracker.Besides computational methods to obtain the most relevant COVID-19 journal articles, alternatively, it is possible to rely on websites (and research teams) that provide manually reviewed lists of completed studies with reported results. For example, The New York Times maintains a vaccine and therapy tracker.11 finds a list of studies for a given condition .1 A second project called regCTGpublications (or regCTGpub for short) generates a ranked list of result articles for trials in a given condition . The regCTGpub project repository12 contains web-based result reports  is a significant factor in world-wide regulatory context (with some exceptions). Fifth, interventions are entered into CTG as free text and proper linkage of identical interventions  depends on a computational algorithm that can miss some linkage of identical interventions.Our study has several limitations. First, we rely on structured links between a registered study and the result article. A prior study for trials completed from 2004 to 2008 indicates that the negative predictive value of such a link may be as low as 56%.We developed a data science driven approach to quickly identify and track linked articles for COVID-19 clinical studies. We characterize which studies are publishing, what type of trial-article link is used, and design a ranking score to prioritize the most significant publications for understanding clinical research for COVID-19. For a set of 2 669 active or ended interventional trials, we found 760 published study result articles, including a short list of 58 key articles from late phase, US based trials with multiple study updates. We separately analyzed trials for COVID-19 vaccines and found 69 linked result articles . The computerized nature of our many analyses allows for the publication of monthly refreshed data at our GitHub repository and the development of a generalized format that can be used to perform similar analysis for other conditions.Supplement 1


The computer simulation presented in this study aimed to investigate the effect of contact tracing on coronavirus disease 2019 (COVID-19) transmission and infection in the context of rising vaccination rates.This study proposed a deterministic, compartmental model with contact tracing and vaccination components. We defined contact tracing effectiveness as the proportion of contacts of a positive case that was successfully traced and the vaccination rate as the proportion of daily doses administered per population in Malaysia. Sensitivity analyses on the untraced and infectious populations were conducted.At a vaccination rate of 1.4%, contact tracing with an effectiveness of 70% could delay the peak of untraced asymptomatic cases by 17 d and reduce it by 70% compared with 30% contact tracing effectiveness. A similar trend was observed for symptomatic cases when a similar experiment setting was used. We also performed sensitivity analyses by using different combinations of contact tracing effectiveness and vaccination rates. In all scenarios, the effect of contact tracing on COVID-19 incidence persisted for both asymptomatic and symptomatic cases.While vaccines are progressively rolled out, efficient contact tracing must be rapidly implemented concurrently to reach, find, test, isolate and support the affected populations to bring COVID-19 under control. Successful digital contact tracing can keep the reproduction number , exposed (E), traced exposed (T), quarantined symptomatic infected (R), death (D) and vaccinated (V). The susceptible compartment (S) was composed of all healthy individuals who could get infected with SARS-CoV-2. Individuals in the exposed compartment (E) were those who had been infected with the virus but remained in their latent period and untraced. The traced exposed (T) compartment referred to infectives in their latent period who were successfully traced and isolated. The quarantined symptomatic infected (R) compartment, and those who received COVID-19 immunity through vaccination formed the vaccinated (V) compartment. Lastly, victims who died from COVID-19 were represented by the compartment D. Individuals could transition from one compartment over time but were only allowed to be in one compartment at a time. These compartments are summarized in Table We developed a deterministic, compartmental susceptible-exposed-infected-recovered-vaccinated (SEIRV) model to study the transmission dynamics of COVID-19 when contact tracing and vaccination were incorporated. The human population was subdivided into 10 classes according to their disease status, namely, the susceptibles (S) could become infectives when they met with either symptomatic (T) and untraced exposed (E) compartments, that is, compartments T) through self-isolation. Therefore, we assumed that all the traced individuals in quarantine (T) will have full compliance.The susceptibles  compartment at a rate of E) compartment at a rate of As they were unaware of their disease status, asymptomatic infectious individuals (E) entering the asymptomatic infectious compartment (T) who would move into the asymptomatic infectious quarantine (E) moving into the symptomatic infectious compartment (T) entering the asymptomatic infectious quarantine  and S). We assumed that the vaccinated individuals gain perfect protection and will not be infected after interacting with an infectious individual, except for when they have lost their acquired immunity due to the probability of vaccination failure  and the probability of transmission given contact with symptomatic individuals (=0.05), whereby the transmission probability was based on the infectiousness of the original COVID-19 strain during early 2020. We calculated the transmission rate 20 to get the The values of parameters in this paper were either estimated using Malaysian COVID-19 data or adapted from the literature. From a COVID-19 modeling study in Malaysia conducted by Gill et al.,22 Hence, we have taken the midpoint of the interval (35%) as the estimated contact tracing effectiveness in Malaysia in one of our simulation scenarios . The vaccination coverage per day (v) was estimated from Malaysian vaccination statistics, where around 1.0–1.5% of the total population were vaccinated daily. Also, we included the values for vaccination coverage (v=0.8%) and contact tracing effectiveness  along with the parameter values in Table v and contact tracing effectiveness In order to solve our SEIRV model, we implemented a numerical integration method, Runge–Kutta of order 5, by using the Scenario 1 simulated five different contact tracing effectiveness values ranging from 30 to 90% against the backdrop of a fixed vaccination rate of 1.4% per day Figures  and 3. TIn scenario 2, we conducted simulations by pairing higher contact tracing effectiveness with a lower daily vaccination rate and vice versa. Despite a higher vaccination rate, the simulated trend of new daily untraced, asymptomatic cases contingent on 30% contact tracing effectiveness was estimated to peak at day 42 with 1.46 million cases. The variant with 90% contact tracing effectiveness but a lower vaccination rate delayed peaking at day 74 with 182 000 cases Figure . A low cWe also simulated two other scenarios where the contact tracing effectiveness was fixed at 90% Figures  and 7 an23 However, some of the interventions, particularly lockdowns and a cessation of economic activities, have negatively impacted the economies and psychosocial well-being of the affected populations.25 One major component of PHSM is the enhancement of surveillance and response actions through contact tracing, testing and isolating close contacts, as well as providing the necessary support mechanisms.23The pandemic saw major movement and travel restrictions, lockdowns and personal protective measures, implemented globally in various forms and stringency. The PHSM are effective in limiting COVID-19 transmission and death.6 and the integration of digital technology with the conventional contact tracing approach could result in a swifter response to stem COVID-19 transmission.In this paper, we simulated the impact of contact tracing on COVID-19 transmission and infection in the context of rising vaccination rates. We observed that a combination strategy of a high daily vaccination rate and low contact tracing effectiveness would significantly increase the untraced infectious population. While the vaccine has been presented as one of the most vital tools to take us towards the restoration of postpandemic normality, we found that contact tracing is key to COVID-19 control. However, the ability of countries to perform contact tracing is challenged by a lack of human resources (contact tracers) and compliance with self-isolation orders, as well as a paucity of timely and accurate contacts data. Successes observed in countries like Singapore and South Korea leveraged technology to aid contact tracing,27 However, the contact tracing of private social gatherings in homes remains an issue and has to be carried out manually, therefore hampering the effectiveness of contact tracing. With the rapid increase in COVID-19 infections and an overwhelmed public health system, the effectiveness of contact tracing in Malaysia has been hindered and currently stands at approximately 30 to 40%.22 Given the significance of contact tracing in controlling the spread of infectious diseases, it is pivotal to capitalize contacts data and automate the data processing and application process,4 as well as to develop more efficient strategies to improve the effectiveness of contact tracing.In Malaysia, personal details are collected for entry to all premises outside of one's residence using a mobile application (MySejahtera) or are documented in writing for contact tracing purposes.In this study, we modeled the number of untraced cases, both symptomatic and asymptomatic. Because this population contributes to forward transmission, an uncontrolled number of these untraced and unisolated individuals could potentially cause a surge in COVID-19 cases. In addition to transmission and infection, future work could be considered using an extended model that incorporates disease severities and health system capacity to estimate the effect of contact tracing on COVID-19 mortality. Furthermore, we set the two main intervention parameters—contact tracing and vaccine—to approximate the ground realities in Malaysia. Notwithstanding testing only on these two PHSMs, we observed that the scenario with low contact tracing and increasing vaccination rates successfully mimicked the current transmission trend in Malaysia. However, it should be noted that this is a theoretical strategy exploration study, as the model was not calibrated to reflect the observed outbreaks in Malaysia and forecast future trends. Hence, further parameterization using local data is warranted to generate outcome estimates more salient to Malaysia.28 While vaccination rates have progressively increased in Malaysia and some parts of the world, efficient contact tracing must be rapidly implemented to reach, find, test, isolate and support the affected populations to bring COVID-19 under control.Contact tracing has been at the forefront in controlling the spread of infectious diseases, and an effective and efficient system could prevent the spread of disease, save lives and allow the economy to resume.This study showed that effective contact tracing is a critical component in COVID-19 control. Our findings indicate the need to inform immediate intervention, such as by using technology and automation, in improving the effectiveness and efficiency of the contact tracing strategy to reduce COVID-19 transmission, infection and death in Malaysia and other countries affected by COVID-19.ihac005_Supplemental_FilesClick here for additional data file.


Unprecedented public health interventions including travel restrictions and national lockdowns have been implemented to stem the COVID-19 epidemic, but the effectiveness of non-pharmaceutical interventions is still debated. We carried out a phylogenetic analysis of more than 29,000 publicly available whole genome SARS-CoV-2 sequences from 57 locations to estimate the time that the epidemic originated in different places. These estimates were examined in relation to the dates of the most stringent interventions in each location as well as to the number of cumulative COVID-19 deaths and phylodynamic estimates of epidemic size. Here we report that the time elapsed between epidemic origin and maximum intervention is associated with different measures of epidemic severity and explains 11% of the variance in reported deaths one month after the most stringent intervention. Locations where strong non-pharmaceutical interventions were implemented earlier experienced much less severe COVID-19 morbidity and mortality during the period of study. Estimating the effects of non-pharmaceutical interventions for COVID-19 is challenging, partly due to variations in testing. Here, the authors use viral sequence data as an alternative means of inferring intervention effects, and show that delays in implementation resulted in more severe epidemics. Because of the immense economic and social consequences of such interventions, the utility of travel restrictions and lockdowns have been questioned by the public and media. Mathematical models can help determine whether these non-pharmaceutical interventions (NPI) have been effective in reducing transmission. Previous investigations based on mathematical models have shown that travel restrictions in Wuhan delayed arrival of the virus into other Chinese cities by around 3 days, and early implementation of control measures was associated with lower incidence2. Similarly in Germany, changes in epidemic growth rates are correlated with the timing of interventions3. In order to assess the impacts of NPIs across European countries, Flaxman et al. estimated the rate of epidemic growth over time from reported COVID-19 deaths and pooled information across European countries4. The impact of NPIs was significant across all countries, reducing the reproduction number below 1, and reducing the number of deaths compared to a scenario with no intervention. Differences in case reporting complicate international comparisons, but as the scale of epidemics during the first wave varied so drastically across countries, such comparisons are essential. Dye et al.5 found that the 100-fold difference in cumulative COVID-19 deaths across European countries was not attributable to lower rates of transmission and epidemic growth, but rather due to differences in dates of national lockdown.To prevent the spread of the SARS-CoV-2 virus, countries have implemented unprecedented measures ranging from school closures and travel bans to large-scale lockdowns6, global comparisons are complicated by the fact that the virus was introduced in different places at very different times. Epidemiological models are typically fitted using data on reported infections, deaths, and seroprevalence surveys. These provide estimates of the basic reproduction number, R0, the average number of onward transmissions per case in a susceptible population; and the time-varying effective reproduction number, Rt. However, because a high proportion of SARS-CoV-2 infections are not detected and seroprevalence surveys have only recently been carried out, parameterising models for epidemics in February through April when large-scale NPIs were enforced is not straightforward. Critically, it is usually not understood how long SARS-CoV-2 circulated in populations prior to detection, which is essential information for comparing timing of NPIs between locations.International comparisons are more complex due to greater variability in epidemiological surveillance and epidemic growth rates. Furthermore, while sustained transmission took place across most of Europe from around mid-FebruaryRt directly from viral sequences7. Analyses of viral sequences have demonstrated that transmission of SARS-CoV-2 went undetected within the USA from mid-January11 and many epidemics were seeded between states rather than as a result of international travel. In contrast in Guangdong, China, the majority of new diagnoses appeared to be imports, demonstrating the effectiveness of surveillance and interventions in interrupting community transmission12.Viral genetic data provide an alternative source of information for understanding when epidemics originated and how quickly they grew during the first wave. Analysis of SARS-CoV-2 genetic sequences can help distinguish between imported and local transmission; and inclusion of sample dates allows for further time-resolution of epidemic dynamics, yielding dates of viral introduction into specific regions. Phylodynamic analysis uses genetic data to parametrise epidemiological models, for example estimating In the present analysis, we reconstruct the epidemic trajectories of SARS-CoV-2 outbreaks for locations across the world for which genetic data were available, to evaluate the scale of epidemics. We estimate the time of viral introduction into each region and calculate time to lockdown for each region. We then determine whether reported and estimated epidemic sizes relate to the duration of times to lockdown.1; as well as case and death counts and census population sizes for each location  meeting inclusion criteria where dates of public health interventions could be obtained and where publicly available SARS-CoV-2 sequences enabled phylodynamic analysis using non-parametric methods . An additional 14 days of transmission before maximum NPI was associated with a 2.91-fold (95% CI: 1.35–6.27) increase in deaths 1 month after maximum NPI in the Deming model and a 2.00-fold (95%CI: 1.19-3.32) increase in the univariate regression. Meanwhile, time from the tenth reported case was not significant in either model . Census population size of each location was not significant in a univariate model predicting deaths (p = 0.08).We constructed a series of regression models with time from CEST to maximum NPI (in days) as predictor variable and a measure of the scale of the epidemic (deaths one month after the NPI) as the outcome. As estimated and extracted variables contained some level of uncertainty, we used Deming regression models, which account for measurement error in both predictor and outcome variables and allow for weighting of observations based on the precision of their estimatesn = 57), we applied a skygrowth model17 (version 0.3.1) to estimate viral effective population size through time and growth rates of effective population size. Under appropriate conditions, effective population size can be used as a proxy statistic for epidemic prevalence7. Viral effective population size at maximum NPI was similarly associated with time to maximum NPI in the Deming . In a multivariable model (R2 = 0.38), time from CEST to maximum NPI remained a significant predictor (p = 0.0005) but census population size did not (p = 0.09). The number of deaths and viral effective population size were highly correlated  and 4.18 . A subset of sites were analysed in BEAST2 v6.1 using model-based phylodynamic methods.45–5.76.Rt, corresponded with changes in Google human mobility metrics for 29 of 30 locations where these data were available  was significantly associated with the severity of the epidemic in each location during the first wave. This result is consistent with analyses5. An association between population size and the number of deaths could result from depletion of susceptibles, limiting onward spread in places with smaller populations. However, serological surveys and patterns of per-capita death do not support the hypothesis that herd immunity has been reached anywhere in Europe22. In our global analysis, census population size was predictive of the number of estimated infections, viral effective population size and the number of deaths in univariate models. But when controlling for the time from CEST to lockdown, population size no longer had an effect, which is consistent with the hypothesis that herd immunity has not substantially limited transmission. The timing of viral introductions and all measures of epidemic scale were inferred independently.A previous comparison across European countries found that earlier lockdown dates were associated with fewer deaths, and that countries with fewer COVID-19 deaths had fewer inhabitants23. Viral effective population size is not always a quantity directly relatable to the number of infections7; however, our analyses support the use of viral effective population size at maximum NPI as a proxy for SARS-CoV-2 epidemic size at that time. Where possible, we compared our estimates to those modelled or measured elsewhere. Estimates of R0 and Rt were in line with those previously reported for the same locations28 and Rt decreased synchronously with reductions in human mobility, as previously demonstrated for Rt estimates derived from traditional epidemiological models31. Dates of introduction into Europe aligned with previous reports demonstrating sustained transmission from mid-February6.International comparisons of NPI effectiveness have been complicated by widely varying testing strategies in different locales and most epidemiological models are highly reliant on reported COVID-19 diagnoses and deaths. Our ability to run analyses on such a wide range of locations derives from the fact that our model is parameterised entirely by genetic data. We have shown that this approach produces accurate estimates of the numbers of infections in simulations 34 on June 7, 2020. Genetic data were cleaned and prepared for time-resolved phylogenetic analysis in R v3.6.1 . Compared to more sophisticated phylogeographic models implemented in BEAST38, our method has the advantage of greater computational scalability, while still integrating over phylogenetic uncertainty.The timing of viral introductions was estimated through time-resolved phylogenetic analysis and parsimony reconstruction. We included all sequences available for that region as well as all close exogenous sequences (≤2 substitutions away) in a maximum likelihood (ML) tree, built using IQtree17 (version 0.3.1) to estimate viral effective population size through time and growth rates of effective population size which under appropriate conditions can be used as a proxy statistic for epidemic prevalence7. Time-resolved phylogenies were constructed, as detailed above, with all exogenous sequences removed prior to analysis. Growth rates and viral effective population size were estimated using skygrowth 0.3.116 using Markov chain Monte Carlo (MCMC) and 500 thousand iterations for each time tree and using an Exponential (10−4) prior for the smoothing parameter. The final results were produced by averaging across 100 time trees estimated for each region. Code to reproduce this analysis is contained in the sarscov2 R package .For each region, we applied a skygrowth model19 to estimate the effective reproduction number and the number of infections through time from SARS-CoV-2 genetic sequences. This model allowed us to assess the reporting rate for each site, the proportion of estimated infections that were diagnosed on each day. The phylodynamic model is designed to estimate epidemiological parameters from sequence data. The model of epidemic dynamics is based on a susceptible-exposed-infectious-recovered model and is described in detail in Supplementary 40. Importantly, the model accounts for bidirectional migration between a region of interest and an international reservoir, and splits the infected compartment into categories representing individuals with high or low rates of onward transmission. Earlier modelling efforts in SARS-CoV-1 and SARS-CoV-2 demonstrated that the inclusion of a high transmission rate compartment is crucial to realistically capturing case numbers41. The ability to accurately reconstruct epidemic dynamics using the Bayesian MCMC inferential framework and phylodynamic model was assessed in a simulation experiment (Supplementary https://github.com/veg/tn93). It is possible that this number of exogenous sequences is insufficient to correctly estimate import rates into our regions of interest; however, this objective was not a focus of the present analysis. For each regional dataset, we then constructed a phylogeny in IQtree. Polytomies were resolved at random ten times, each time generating a new starting tree for the analysis in BEAST2, totalling ten independent chains.Finally, we utilised a compartmental structured coalescent model in the BEAST2 v6.1 PhyDyn packagehttp://github.com/emvolz-phylodynamics/sarscov2Rutils).Each of the 10 runs was set up for 20 million steps. Subsequently, log files were examined for convergence in Tracer v1.7.1, problematic runs excluded, and log files and trajectory files were combined and cleaned using the sarscov2 R package (available at 1 (downloaded 20/06/2020). We used the date of measure C6 (shelter-in-place), as classified by the OxCGRT dataset. If the region never underwent a full lockdown, we used the date of school closures or recommendation to work from home, whichever came first  and the period of time from symptom onset to death (95% occur within 19 days43).We obtained case and death counts, and census populations sizes, for inclusion in our models and to assess case reporting  to each NPI and the time from the tenth reported case to each NPI for each site and constructed a series of regression models, looking for relationships between this delay and the severity of the epidemic at each site. Deming regression models were utilised to take into account for uncertainty in both predictor and outcome variables and allow for weighting of observations based on the precision of their estimatesR0 as predictors. Population size, viral effective population size, estimated infections, reported infections and death counts were log-transformed prior to analysis, to reflect their exponential rate of increase. As a sensitivity analysis, as well as the CEST, we also calculated the 5th and 25th percentiles of the distribution of viral introduction times and recalculated the delay to maximum NPI for each definition, using these variables as predictors in regression models. All analyses were conducted in R v3.6.1.For all sites, we built Deming regression models with time from CEST to maximum NPI (in days) as predictor variable and number of deaths 1 month after the maximum NPI as the outcome. To evaluate if the time that cases were first reported could serve as a proxy for time of epidemic origin, we reran these models changing our predictor variable to time between the tenth reported case and maximum NPI. For the 30 locations where the number of infections was estimated, we built a model with time from CEST to maximum NPI as a predictor and the estimated number of infections as an outcome. We repeated all three CEST-based models using univariate linear regression, then expanded them into multivariable models, including population size and Rt31. Google mobility data measures daily mobility at the sub-region level, in relative deviations from maximum mobility prior to the WHO pandemic declaration. Mobility data were available from the 13th of January 2020 and up until the date of cut-off for genetic data (June 7th). For each site analysed, we plotted both variables for the time period between 13th of January and the date of the last sample available for that site. We limited our analysis to mobility associated with transit stations only (one of six streams). Daily Rt estimates through time were extracted for the same time period.For sites analysed using the BEAST phylodynamic model, we examined the relationship between mobility data provided by Google (google.com/covid19/mobility) and Google mobility data (google.com/covid19/mobility) are released in strict adherence to Google privacy protocols, from users who have chosen to turn on “Location History”. Data are fully anonymized, aggregated at the sub-region level and provided as relative deviations from mobility prior to the WHO pandemic declaration.Further information on research design is available in the Supplementary InformationPeer Review FileDescriptions of Additional Supplementary FilesSupplementary Dataset 1Supplementary Dataset 2Reporting Summary


Sudden unexpected death in infancy (SUDI) is the sudden and unexpected death of an apparently healthy infant occurring within the first year of life where the cause is not immediately obvious. It is believed that a proportion of unexplained infant deaths are due to an infection that remains undiagnosed. The interpretation of post-mortem microbiology results is difficult due to the potential false-positives, a source of which is post-mortem bacterial translocation. Post-mortem bacterial translocation is the spread of viable bacteria from highly colonised sites to extra-intestinal tissues. We hypothesise that although post-mortem bacterial translocation occurs, when carcasses are kept under controlled routine clinical conditions it is not extensive and can be defined using 16S rRNA gene sequencing. With this knowledge, implementation of the 16S rRNA gene sequencing technique into routine clinical diagnostics would allow a more reliable retrospective diagnosis of ante-mortem infection. Therefore, the aim of this study was to establish the extent of post-mortem bacterial translocation in two animal models to establish a baseline sequencing signal for the post-mortem process. To do this we used 16S rRNA gene sequencing in two animal models over a 2 week period to investigate (1) the bacterial community succession in regions of high bacterial colonisation, and (2) the bacterial presence in visceral tissues routinely sampled during autopsy for microbiological investigation. We found no evidence for significant and consistent post-mortem bacterial translocation in the mouse model. Although bacteria were detected in tissues in the piglet model, we did not find significant and consistent evidence for post-mortem bacterial translocation from the gastrointestinal tract or nasal cavity. These data do not support the concept of significant post-mortem translocation as part of the normal post-mortem process. Sudden unexpected death in infancy (SUDI) is the sudden death of an apparently healthy infant occurring within the first year of life where the cause is not immediately obvious. SUDI is the leading cause of infant mortality in the developed world . DespiteIn diagnostic laboratories worldwide, classical bacterial culture is the gold-standard method used to identify potential causative organisms as part of routine clinical autopsy investigation . HoweverIn recent years, these caveats have encouraged the use of alternative molecular techniques, such as quantitative PCR (qPCR) targeting known pathogenic bacteria, or the sequencing of the bacterial 16S rRNA gene. The 16S rRNA gene is present in all bacterial species and each has a unique sequence. These sequences act as a bacterial fingerprint to enable bacterial identification by comparison to curated 16S reference databases, such as Greengenes, SILVA, and RDP . This meApparent false-positive PM microbiology results can arise due to the natural putrefaction process of the body or contamination during sample collection at autopsy . When poStaphylococcus aureus following intranasal inoculation of a mouse model and found that these bacteria could be detected in all organs 1 hour PM  the bacterial community succession in regions of high bacterial colonisation, and (2) the translocation of these microbes to extra-intestinal tissue over a 2-week period PM.n = 15) that had been housed for 6 days to allow for acclimatisation and standardisation of the microbiota for another study were culled on day 0 according to Schedule 1 of the Animals (Scientific Procedures) Act 1986 and carcasses were donated to this study. Culled mice were placed into airtight freezer bags and kept on wet ice until the first sample was collected, no longer than 30 minutes PM. Tissue and blood samples were collected from one mouse per cage (C) on the following days (D) PM: D0, D3, D7, D10, and D14 (three mice sampled per day) in a sterile manner to limit contamination. Tissue samples were collected in the following order: cardiac blood, heart, lung, liver, spleen, and the lower GI tract. Mice were stored in airtight freezer bags at 4°C until sampling.Nine-week old C57BL/6 female mice (n = 1) and a local farm (n = 2). The piglets used in this study were of a pre-weaning age. Sampling was initiated no more than 3 hours PM. Samples were obtained using sterile 16-gauge Quick-Core Biopsy Needles  on D0, D3, D7, D10, and D14 PM. On D14 PM, carcasses from the local farm were dissected using sterile scalpels. Nasal (left and right nares) and rectal swabs were obtained using polyurethane cellular foam dry swabs . Samples were immediately snap frozen and stored at −80°C until further analysis. Tissue type confirmation was performed using tissue-specific reverse transcriptase PCR as described in the Three 1-month-old Welsh piglet carcasses were obtained from the Royal Veterinary College, London (DNA was extracted using the QIAamp DNA Mini Kit (Qiagen) as per the manufacturers protocol with an initial lysing step using Ribolysing Matrix B  and bead-beating for 1 minute at 50 oscillations/second. DNA templates were amplified for sequencing using the Taq PCR Core Kit (Qiagen) and region-specific primers targeting the V3-V4 region of the 16S rRNA gene with adapters and indexes attached as previously described . This meSee Sequencing data was demultiplexed on the MiSeq platform. Paired-end reads were merged using FLASH (version 1.2.11)  and qualvegan package  was assessed with a non-metric multi-dimensional scaling (NMDS) plot based on Bray-Curtis dissimilarities between the samples. The effects of sampling day on bacterial community changes were measured by performing a permutational multivariate analysis of variance (PERMANOVA) test on Bray-Curtis dissimilarity measures using the phyloseq package   on R Stu package . Mouse G package . A Krusk package . For sta package  and Grap package .P = 0.07). The gut microbiome in all mice sampled at each time point were similar as expected due to their inbred nature and controlled housing environment and diet prior to this study. Based on the Bray-Curtis distances calculated for each community, there was no significant difference between communities from each mouse (P = 0.096). Significant changes were, however, observed in the communities on D10 (P = 0.007) and D14 (P = 0.037) when compared to D0. To investigate bacterial families responsible for these differences a Kruskal-Wallis by rank test was performed. Dominant families did not experience any changes in rank over the study period. Significant differences were observed in Bifidobacteriaceae (P = 0.013) and Rikenellaceae (P = 0.041). At each time point from D0 to D10 Bifidobacteriaceae represented less than 1% of the total bacterial community. This was followed by a sharp increase on D14 where their relative abundance increased to 7.2%. Rikenellaceae was also present at less than 1% in the starting community. There was a steady increase to D10 where the relative abundance peaked at an average of 3.9%, which decreased again at D14 where it represented just 0.6% of the community.The calculated beta-diversity at each time point was compared to that of D0 to track microbiota changes over time. The first 7 days PM showed no significant changes in the gut microbiome (P = 0.004). The number of successfully sequenced samples did not increase with time PM.Of the 75 mouse tissue samples collected, only 4 were considered as positive for the 16S rRNA gene using the quality filtering threshold described in the methods section  detected in each tissue sample were cross-checked with OTUs identified in the GI tract to assess the potential translocation of bacteria from this site. Two of the four samples (C1_D3_Liver and C2_D14_Blood) did not share any OTUs with the GI tract samples collected from the same mouse at the same time point. Sample C2_D3_Liver shared all OTUs with the corresponding GI tract sample and contained bacterial families including Enterobacteriaceae, Lactobacilliaceae, and Porphyromonadaceae. These families were of high abundance in the GI tract. Sample C2_D10_Heart shared 4 of 11 OTUs identified with its corresponding GI tract sample.Three of the four tissue samples were dominated by either Enterobacteriaceae or Enterococcus, which represented >45% of each sample. To identify whether these were truly positive reads, a specific qPCR was performed. These bacteria were identified in three gut samples from cage 2, at low relative abundances between 5 and 11%. All tissues were negative for Enterococcus. Two of the three tissues were positive for Enterobacteriaceae, but generated a very late signal with a cyclic threshold (CT) value >34 compared to the positive controls which gave an average CT value of 27.P = 0.018). The relative abundance of bacterial families in each piglet are shown in The microbes within the rectal cavities were significantly different in each piglet as expected due to differences in genetics, diet and environmental conditions (P = 0.33). The convergence was largely due to the increase of Pseudomonadaceae, which began to emerge on D7 and increased in abundance until the end of the study period. On D14 PM, Pseudomonadaceae represented >90% of bacteria within the nasal cavity of each piglet.Despite differences in early communities, nasal swab communities from all three piglets progressed in a similar fashion . On D0, P = 0.0001). Variation amongst the reads obtained from each piglet was observed, but there were no associations with sampling day or tissue type.As the piglet tissue samples were obtained using needle biopsies in a blind fashion the tissue type was confirmed using a reverse-transcriptase qPCR  shared an OTU that was present at relative abundances >1% in the rectal swabs . Of the The aim of this study was to establish the extent of PM bacterial translocation in two animal models to establish a baseline sequencing signal for the PM process. This study provides insight into the community succession within the mouse GI tract, and the piglet nasal and rectal cavities. Our results also demonstrate that significant and consistent PM bacterial translocation was not observed in the mouse model, shown by broad-range 16S rRNA gene sequencing and confirmed with more specific, bacteria-targeted qPCR. Although bacteria were detected amongst tissues in the piglet model, we did not find evidence for true bacterial translocation from the GI tract or nasal cavity using these techniques. These data do therefore not support the concept of significant PM translocation as part of the normal PM process and demonstrate that the 16S rRNA gene sequencing technique could prove useful in diagnosis of infection PM.Though the mouse gut microbiome was relatively stable over the first 7 days, significant changes were observed in Bifidobacteriaceae and Ruminococcaceae families at latter time points, increasing in abundance over time. These bacteria are known anaerobes, which explains their proliferation at a time when oxygen sources are low, and nutrients limited. The very slight changes in the microbiome under these conditions demonstrate the ability of the refrigeration to slow the decay process. This is in line with a study performed by We observed an exponential increase of Pseudomonadaceae members within the nasal cavity of the piglets as time PM increased. Studies have identified Pseudomonadaceae on porcine carcasses and are associated with food spoilage, given their ability to proliferate at low temperatures . It is aThe mouse model used in this study failed to demonstrate ubiquitous PM translocation using 16S rRNA gene sequencing, with most samples possessing bacterial DNA loads too low to be sequenced. These results are reflected in a similar study performed on rabbit carcasses which investigated the effects of temperature on the thanatomicrobiome where they failed to amplify bacterial DNA in both lung and kidney samples due to low bacterial load . The stuThe low-level detection of Enterobacteriaceae and Enterococcus in the mouse tissues and the late signal produced using a highly sensitive assay show that the bacteria are potentially present within the tissue, but at very low numbers. If this were due to translocation, it is unlikely that it would only be seen in these few tissues at seemingly random time points. This observation could therefore represent PM bacterial translocation but given the low read numbers this is unlikely. It has been found elsewhere that in human autopsies Enterobacteriaceae is found at higher numbers in autopsies performed more than 24 h after death using both molecular and conventional culture analysis . This fiWe found a larger number of positive tissues and sequencing read numbers in the piglet model compared to the mouse model. The similarity of bacterial families shared between tissues collected from the same piglets suggest that these bacterial families originate from the piglet itself, rather than the external environment. Given the lack of concordance between bacterial families found in the tissues and swabs it is unlikely that these bacteria originate from such sites. Bacterial families, such as Moraxellaceae, Clostridiaceae, Flavobacteriaceae and Peptostreptococcaceae have been found in high abundance on the skin, tonsil, and nasal cavities of piglets in various studies . It is tThe vast number of positive cultures that are typically detected in human SUDI PM examinations are reflected in positive piglet tissues in this study; > 80% of human PM tissues from SUDI autopsies generate positive culture results . In thesWe acknowledge limitations of this study including the lack of skin sampling which could enhance the interpretation of bacteria within tissues. It is also important that this study was carried out on healthy animals free of known infection. If an infection were to be present in these animals the conclusions may have been different, perhaps due to increased invasive abilities of pathogenic bacteria, their disruption to commensal microbes or damage to tissue epithelium compromising barrier function. We also acknowledge that the time between death and sampling in these model organisms was also controlled as would not be the case in SUDI. Although this study could not detect PM bacterial translocation in these model organisms, it is important to note that the PM process in human subjects may differ. Given the small sample size and the use of animal models, no clinical conclusions can be made but this study provides an insight into the investigation of PM overgrowth of bacteria and acts as a proof-of-principle study into the use of 16S rRNA gene sequencing under these circumstances.A deeper understanding of the bacterial changes that occur PM in clinical settings will allow for improved diagnostics, particularly in cases of SUDI due to sub-clinical infection. By determining the PM process using 16S rRNA gene sequencing we have demonstrated how this technique could be incorporated into routine PM diagnosis of infection. The results of this study did not, however, provide convincing evidence to support ubiquitous PM bacterial translocation. Future works should focus on implementing this technique on PM tissues from cases of SUDI to enhance our understanding and retrospective diagnosis of fatal infection.PRJNA698635.The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: NCBI BioProject Ethical review and approval was not required for the animal study as animals whose carcasses were used in this study were not killed for this work. No aspect of the animal’s life or death was altered for the scientific purpose of the present study.NS and NK came up with the conceptual idea for this study and the clinical need for such work. LG and DA devised the study plan. LG performed all the experimental work, developed bioinformatics pipelines for the data analysis and visualisation, and wrote the manuscript with consultation from NS, NK, and DA. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


In cases with collateral score = 3, V2 was significantly smaller or absent compared to V1; in those with collateral score 2, V2 was slightly larger than V1, and in those with scores 1 and 0 V2 was significantly larger than V1. These relationships were observed in both early and late time windows. Conclusions: The collateral status determined the evolution of the baseline hypodensity on NCCT in patients with anterior circulation stroke who had MT reperfusion. Damage can be stable or reversible in patients with good collaterals while in those with poor collaterals tissues that initially appear normal will frequently appear as necrotic after 24 h. With good collaterals, it is stable or can be reversible while with poor collaterals, normal looking tissue frequently appears as necrotic in follow-up exam. Hence, acute hypodensity represents different states of the ischemic brain parenchyma.Introduction: The aim of the study was to assess the impact of collaterals on the evolution of hypodensity on non-contrast CT (NCCT) in anterior circulation stroke with reperfusion by mechanical thrombectomy (MT). Methods: We retrospectively included stroke patients with middle cerebral artery occlusion who were reperfused by MT in early and late time window. Artificial intelligence (AI)-based software was used to calculate of hypodensity volumes at baseline NCCT (V1) and at follow-up NCCT 24 h after MT (V2), along with the difference between the two volumes (V2-V1) and the follow-up (V2)/baseline (V1) volume ratio (V2/V1). The same software was used to classify collateral status by using a 4-point scale where the score of zero indicated no collaterals and the score of three represented contrast filling of all collaterals. The volumetric values were correlated with the collateral scores. Results: Collateral scores had significant negative correlation with V1 ( The introduction of mechanical thrombectomy (MT) revolutionized the treatment of acute ischemic stroke in patients with large vessel occlusion (LVO).Studies analysing the efficacy of MT have identified several pretreatment and treatment-related determinants of outcome. These include age, baseline National Institute of Health stroke scale (NIHSS), Alberta Stroke Program early CT score (ASPECTS), non-viable and salvageable tissue volume, time from stroke to reperfusion, degree of reperfusion, collaterals and hyperglycaemia ,3,4,5,6.Current guidelines state that the decision for MT should be guided by estimation of non-viable (core) and possibly-salvageable (penumbra) tissue volumes, especially in extended time windows of >6 h from the onset of stroke ,8. CT peHowever, a recent study by Hendrix et al.  as well Although promising, such a simplified approach remains debatable because the importance of the acute hypodensity volume represented by the ASPECTS remains unclear and there is as yet no unequivocal evidence for how it should be interpreted. Additionally, it is not yet clear whether acute hypodensity at NCCT represents an irreversibly lost necrotic zone or salvageable tissue or some kind of mixture. These factors motivated us to design a study comparing baseline and follow-up hypodensities on NCCT and analysing their association with clinical and imaging parameters. We considered the factors which are thought to modify the evolution of hypodensity volume and focused on the status of the collateral circulation.We used artificial intelligence (AI)—based software to assess the collaterals, and we used the same software package for calculation of the hypodensity volumes with revision by an experienced neuroradiologist. The rationale for this approach is the increasing ubiquity of AI-based software packages supporting physicians in the diagnostic work-up of stroke patients and the paucity of studies validating their accuracy in the assessment of collateral status.The main aim of the study was to assess the evolution of hypodensity volumes on NCCT as correlated with collateral status in anterior circulation stroke patients successfully reperfused with MT.The local Ethics Committee of the Pomeranian Medical University in Szczecin approved the study. Informed consent was waived considering the retrospective character of the study and the use of routinely performed procedures. Data of consecutive patients who presented to our comprehensive stroke centre from February 2019 to October 2021 with suspected acute stroke and underwent head NCCT were retrospectively retrieved using our Radiological Information System and Picture Archiving and Communication System. The initial inclusion criteria were NIHSS score ≧ 6 and modified Rankin Scale (mRS) score ≦ 2. Patients with intracranial haemorrhage or ASPECTS score < 6 at baseline NCCT were excluded. Those within the therapeutic window of ≦4.5 h without contraindications received intravenous thrombolysis (IVT).The subsequent parts of diagnostic work-up included CTP and CT angiography (CTA), and MT was indicated in patients with LVO detected at CTA within a therapeutic window of ≦6 h. Patients in extended therapeutic windows of >6 h were evaluated for MT according to DEFUS-3 or DAWN criteria ,13. PatiWe included patients with the middle cerebral artery (MCA) LVO because the AI-based software performs calculations only in the territory supplied by this vessel.Another inclusion criterion was MT with cerebral infarction scale (TICI) 2b or 3 reperfusion. Patients with TICI 0 to 2a reperfusion were not analysed to exclude any confounding effects because gradual tissue loss probably continues beyond 24 h in cases of poor reperfusion and the hypodensity volume at the follow-up NCCT 24 h post MT may not represent the final infarct volume. Another reason was inhomogeneity of this population as TICI 0/1 means no or minimal reperfusion while in TICI 2a partial reperfusion reaching almost 50% of ischaemic territory can be achieved. This difference could interfere with assessment of the impact of collaterals. Additionally, the excluded group including only 58 patients was too small for performing reliable statistical analysis and drawing general conclusions. Besides, we were focused on investigating whether baseline hypodensity would progress, resolve or stabilize after nearly complete or complete reperfusion with TICI 2B or 3 depending on collateral status.The remaining exclusion criteria were lack of a 24-h follow-up NCCT after MT, any hyperdensity representing haemorrhage or contrast staining on follow-up NCCT and technical inadequacy of any CT examination. We also excluded patients without acute hypodensity on baseline NCCT because in these cases it was impossible to calculate V2/V1. The study selection flow chart is presented in All patients were scanned on a 128-slice multi-detector CT SOMATOM Definition AS+ .NCCT, CTP and CTA were performed with the technique and parameters described in previous study .A board certified interventional neuroradiologist with 15 years of experience (M.S.) evaluated all CTAs and recorded the presence, side, and location of LVO. He was blinded to all clinical and imaging data.All CTP exams were assessed on a regular basis by the radiologist on duty for clinical use, but those results were not included in this analysis.The baseline and follow-up NCCT hypodensity volumes were calculated with the automated e-ASPECTS tool, a part of the e-Stroke Cloud imaging suite .The collateral status was scored by using the e-Stroke Cloud e-CTA tool.The e-Stroke Cloud image processing algorithms follow an AI approach, with a combination of traditional 3D graphics and statistical methods and machine learning classification techniques. The input DICOM data are first resampled to correct any gantry tilt and standardize the input resolution. Then, a fast proprietary registration approach is applied to re-align the data, removing any tilt and rotation. This ensures that the image is presented in a standard reference frame, which simplifies human interpretation of the scan.To identify acute hypodensity volumes e-ASPECTS tool compares corresponding regions of normal and ischaemic hemisphere voxel by voxel. Acute hypodensity is defined by difference between hemispheres in Hounsfield Units with threshold values. The process is based on AI combined with 3D graphics and statistical methods. To eliminate the possibility that some subacute changes on the follow-up NCCT were missed by the software all follow-up exams were revised by interventional neuroradiologist (M.S.). Manual correction by radiologist was needed in 6 cases.The e-CTA process uses a combination of machine learning and deep learning algorithms to identify LVO in the anterior circulation.The e-CTA also uses multiphase images derived from CTP to define and quantify asymmetrical contrast densities from which the degree of collateral blood vessel density can be estimated, quantified both as a relative percentage and using the Tan collateral score .These algorithms have been trained on a large dataset  containing a wide range of real-world CT scans from stroke patients and negative controls as well as ground-truth data from additional imaging data such as magnetic resonance (MR) images acquired within 1 to 2 h of the CT scan along with other modalities and clinical information. The training dataset contains examples of CT scans captured with scanners from all major manufacturers and from a wide range of countries worldwide. The e-Stroke Cloud suite received its European Union CE mark certification in 2018.For collateral scoring, CTA images  with 3-mm thickness) were automatically reconstructed from the CTP source data as timing-invariant (TI)-CTA using e-CTA. TI-CTA visualizes vessels by overlapping all timeframes and displaying maximal enhancement over time. This technique is timing invariant because of the choice of the temporal maximum, meaning that the peak enhancement of a vessel is displayed regardless of the delay of contrast arrival. Accurate assessment of the collaterals is possible because TI-CTA is not sensitive to delayed arrival of contrast material in cerebral vessels. This technique was introduced by Smit et al. . It has As a reference, collateral scoring was also assessed by a board certified interventional neuroradiologist (M.S.) who analysed the same TI-CTA images derived from the CTP as were used by the e-CTA software. The level of interobserver agreement in collateral scoring between the neuroradiologist and e-CTA was calculated, and the collateral scores from the neuroradiologist were used as the reference for further analyses.Beside age and sex, the following data were recorded: occlusion site, classified as the M1 or M2 segment of the MCA; technique and result of MT as TICI 2b or 3 based on the report in the radiological information system; time gap from the onset of stroke symptoms or last seen well to the arterial puncture preceding MT, grouped as ≤3 h, 3 to 6 h and >6 h, and the administration of IVT.V1, defined as the volume of acute hypodensity at the baseline NCCT;V2, defined as volume of hypodensity at the 24-h follow-up NCCT;V2 − V1, defined as the difference between the follow-up and baseline hypodensity volumes;V2/V1, reflecting the follow-up/baseline volume ratio.The following data were registered based on the e-ASPECT calculations for the NCCT:The results provided by the AI-based software were revised by the neuroradiologist (M.S.) to eliminate significant miscalculations.0, no contrast filling of the collateral supply to the occluded MCA territory;1, collateral supply to >0% but ≤50% of the occluded MCA territory;2, collateral supply to >50% but <100% of the occluded MCA territory;3, contrast filling of the collateral supply to 100% of the occluded MCA territory.The collateral scores from e-CTA and the neuroradiologist (M.S.) were recorded by using the scoring system proposed by Tan et al. . This syThe level of interobserver agreement in collateral scoring between the neuroradiologist and e-CTA was calculated, and the collateral scores from the neuroradiologist were used as the reference for further analyses.The time gap from MT to follow-up NCCT was noted.The processing time for e-Stroke, defined as the time from data transmission to the reception of results, was also recorded.t test, one-way analysis of variance, Mann–Whitney, or Kruskal–Wallis tests as appropriate. Categorical variables were compared with the χ2 or Fisher exact tests as appropriate. Spearman’s rank correlation coefficient (r) was used for calculation of correlation between continuous variables. The level of interobserver agreement in collateral scoring between the neuroradiologist and the e-CTA was assessed with Krippendorff’s alpha coefficient.Continuous variables were reported as mean  or median  and were compared with the Student’s Statistical calculations were performed by a biostatistician (K.S.) using Statistica 13 data analysis software  with Plus Bundle 5.0.96.The data of 581 patients were collected, and after implementation of the exclusion criteria, 105 were finally included in the study.There were 53 women and 52 men with a mean age of 69.7 years . Detailed patient characteristics are presented in p = 0.04).The mean time gap from the onset of stroke symptoms or last seen well to the arterial puncture preceding MT was 4 h 12 min (range 1 h 36 min to 9 h 40 min). Sixty-six (62.9%) patients were within the therapeutic window of 4.5 h for IVT, and IVT was administered in all of them except three who had contraindications. V1 was significantly smaller in the patients who had IVT compared to those who did not  patients, aspiration with stent retriever in 31 (29.5%) and stent retriever alone in 5 (4.8%).The mean time to the follow-up NCCT after MT was 24 h 33 min .p = 0.035). The strongest correlation was found with V2 . Collateral score significantly negatively correlated with V2–V1  and V2/V1 .The collateral score was significantly negatively correlated with all hypodensity volume parameters, although the correlation with V1 was borderline  revealed the same associations between collateral status and hypodensity volume parameters as for the entire study population. These findings are presented in No correlation was found between collateral score and other parameters.In the group with collateral Grade 3 the follow-up hypodensity volume appeared to be smaller than the baseline volume, and was absent in some cases. In the group with collateral Grade 2 there was a slight increase of the hypodensity volume and in the groups with collateral Grades 0 and 1 the increase was significant, as presented in There were 17 patients  who were treated with MT in later time windows (time from onset of stroke to groin puncture ranging from 6 h 10 min to 9 h 40 min). Among seven who had collateral scores of three, three showed hypodensity volume decrease; two showed no change and two showed increases. The collateral scores were 1 or 2 in the remaining ten cases, and only one of these, a patient with score two, showed decreased hypodensity. Volume increases were observed in the other nine and the greatest volume increases were observed in two cases with collateral score one. These results are presented in The level of interobserver agreement in collateral scoring between the neuroradiologist and the e-CTA algorithm was very good .The mean processing time was 103 s (±28 s) for e-ASPECTS and 81 s (±23 s) for e-CTA.Our study showed that the collateral circulation status in stroke impacts the baseline hypodensity volume at NCCT and modifies its evolution after recanalization. These effects are independent of time from onset of stroke to groin puncture or IVT administration.Patients who were eligible for and received IVT had moderately smaller baseline hypodensity volumes compared to those without thrombolysis probably because IVT administration is avoided in patients with large hypodensities at baseline NCCT due to the high risk of haemorrhagic transformation.p = 0.035), we did not find marked differences in baseline volumes between collateral scores treated as unordered levels of nominal variables. This relationship was completely different in the follow-up exams. In patients with the best collateral status (score three), the follow-up NCCT usually showed partial or even complete regression of hypodensity. In the group with decent collateral score of two stabilization or slight increase of hypodensity was typically observed. In patients with poor or absent collaterals (scores one and zero) significant progression of hypodensity was detected despite efficacious recanalization. These dependencies were observed in both early and in late time windows.Although we observed significant correlation between collateral scores treated as rank variables and baseline hypodensity volumes (Although our study was limited to assessment of a single imaging factor as an outcome measure, the findings could be regarded as consistent with results of studies showing that collaterals determine functional outcome by the extensively used mRS at 90 days ,30,31,32A novelty of our study is the application of AI-based software instead of conventionally used human reading in the assessment of collaterals.Our study also adds evidence that baseline NCCT does not reflect the true status of ischemic brain parenchyma.We showed that baseline hypodensity on NCCT can be reversible in patients with good collaterals. On the contrary, in the presence of poor collaterals, tissue that looked normal at the baseline NCCT frequently progressed to necrotic at the follow-up exam despite successful recanalization. Moreover, it appears that such evolution is independent of the time from stroke to reperfusion, as it was seen in cases with of both early and extended therapeutic windows. This may lead to the interpretation that the initial pattern on NCCT represents different states of ischaemic brain tissue. With good collaterals, acute hypodensity is not consistent with necrotic zone as it is potentially salvageable after reperfusion. Contrarily, with poor collaterals, initially normal looking tissue could be already irreversibly damaged before reperfusion. In such cases this zone, which looks normal at the baseline NCCT appears as necrotic at the follow-up exam. This phenomenon could be explained by a well-known fact, that NCCT detects the signs of acute ischaemia with some delay.This puts into question the reliability of ASPECTS as an independent predictor of imaging and clinical outcomes because it directly represents the acute hypodensity volume, and there may be some implications regarding the weight of the ASPECTS score in the selection of stroke patients for MT.According to current guidelines ,8, in thHowever, advanced imaging can be prohibitively expensive and is not widely available across the world, and requiring this technology for selection excludes many patients from being considered for late MT.This issue has been addressed in two recent studies by Hendrix et al.  and NguySuch an approach seems to be promising. Although our results show that hypodensity at baseline NCCT may be an unreliable indicator of final infarct in many cases because it can significantly increase or decrease depending on collateral status, the proposed simplified selection criteria could be optimized by including the scoring of collaterals which is a strong modifier of acute hypodensity in NCCT and can be assessed with routinely performed CTA without requirement for advanced imaging techniques.Without doubt, this hypothesis needs to be verified by further comprehensive research.The study population was limited to stroke patients with M1 and M2-MCA occlusions treated with MT resulting in TICI 2b and 3. We did not include untreated patients, those with tandem or ICA occlusions or those with TICI 0 to 2a. We also did not analyse all known modifiers of outcome and there was no analysis of functional outcomes. Another limitation is the use of only one type of CT scanner. Comparing results from different scanners would be needed to verify that the trend found in this study holds true for other acquisition parameters. Lastly, the population with collateral score zero consisted of only three patients and only 17 patients were treated in the late window, which could bias the results.In anterior circulation stroke patients who were reperfused with MT, evolution of hypodensity volumes on NCCT was correlated with collateral status in both early and late time windows. Baseline hypodensity is commonly stable or can be reversible with good collaterals, while with poor collaterals, normal looking tissue frequently appears as necrotic in follow-up exam despite apparently successful recanalization.Considering that acute hypodensity represents different states of ischemic brain parenchyma, ASPECTS as predictor of outcome should be analysed in association with collateral status.