Peripheral blood mononuclear cells (PBMCs) are blood cells that are a critical part of the immune system used to fight off infection, defending our bodies from harmful pathogens. In biomedical research, PBMCs are commonly used to study global immune response to disease outbreak and progression, pathogen infections, for vaccine development and a multitude of other clinical applications. Over the past few years, the revolution in single-cell RNA sequencing (scRNA-seq) has enabled an unbiased quantification of gene expression in thousands of individual cells, which provides a more efficient tool to decipher the immune system in human diseases. In this work, we generate scRNA-seq data from human PBMCs at high sequencing depth  for more than 30,000 cells, in resting, stimulated, fresh and frozen conditions. The data generated can be used for benchmarking batch correction and data integration methods, and to study the effect of freezing-thawing cycles on the quality of immune cell populations and their transcriptomic profiles. PBMCs are used in a multitude of different areas, such as vaccine development, infectious disease study, immunology, disease modeling and biomarker identification, just to name a few. However, due to the complexity of PBMCs, which contain multiple different cell types, studying the function of the individual cell types can be difficult, and studies often rely on bulk measurements. Single-cell RNA-sequencing (scRNA-seq) approaches can be used to overcome these problems, allowing for the identification and quantification of the subpopulation of cells that make up the PBMC sample quite easily. ScRNA-seq has emerged as a central tool for identifying and characterizing cell types, states, lineages and circuitry2. The rapid growth in the scale and robustness of laboratory protocols and associated computational tools has opened the way to substantial scientific discoveries. ScRNA-seq was also applied to the sequencing and analysis of PBMCs transcriptomics in various health and disease contexts. For instance this technology was used quite recently to analyze the immune response landscape of COVID-19 and Influenza patients3. In another study, Wang and colleagues used scRNA-seq on PBMCs to study the global immune response in Kawasaki disease (KD) patients. KD is the most common cause of acquired heart disease in children in developed countries. The study showed that the most differentially expressed genes were found in monocytes, with high expression of pro-inflammatory mediators, immunoglobulin receptors and low expression of MHC class II genes in acute KD, and that the percentage of CD8+ T cells is decreased in acute KD, and notably effector memory CD8+ T cells compared to healthy controls4. Here we use scRNA-seq to sequence more than 30,000 cells from human PBMC samples to high depth of coverage . We provide data from resting and stimulated PBMCs. We used lipopolysaccharides (LPS) bacteria extracts to induce an immune response and transcriptional changes. We also provide a dataset where cells have been frozen according to a demonstrated protocol for the preparation of fresh frozen human PBMCs for scRNA-seq analysis. We illustrate how those datasets can be used for benchmarking computational methods for data integration and to analyse the consequences of freezing/thawing cells before scRNA-seq library preparation.Peripheral blood mononuclear cells (PBMCs) are blood cells that are a critical part of the immune system used to fight off infection. They work together to protect our bodies from harmful pathogens. Due to their primary location in peripheral blood, they act as a line of defense against infection and disease. PBMCs are used by medical researchers to study immune cell behavior when exposed to various pathogens, disease progression in the human body and factors affecting long-term immunityFresh blood was provided by the EFS (Etablissement Francais du Sang) from two healthy anonymous donors  who gave informed consent for experimental research work. The EFS is a public administrative body responsible for collecting, preparing, qualifying and distributing labile blood products  for blood transfusion in France. EFS is authorised, for its non-transfusion activities, to collect, prepare, store and sell blood or its components for teaching or research purposes, excluding any therapeutic use. Only adults (older than 18 years old) can give their blood and harvesting is only carried out with the written consent of the donor. The doctor informs the donor of the importance of biological samples for the progress of medical research.Twelve ml (2 tubes by donor) of fresh blood was collected in EDTA anticoagulant. PBMCs were isolated using HISTOPAQUE-1077 . After centrifugation (285 rcf × g for 5 min at room temperature without the brake), PBMCs remained at the plasma-HISTOPAQUE-1077 interface and were carefully transferred to a new tube. PBMCs were washed two or three times with 3x volume 1X PBS (cat# 10010023 Gibco) followed by a centrifugation (285 rcf × g for 5 min at room temperature without the brake). A visual check was done after each centrifugation to ensure that no platelets remain. After that, we followed two different paths  containing 0.04% filtered Bovine Serum Albumin . Resuspended cells were centrifuged at 285 rcf × g for 5 min at room temperature. The supernatant was removed delicately and the cell pellet was resuspended in 1 mL 1X PBS containing 0.04% filtered BSA. For the frozen cells, PBMCs were thawed and resuspended according to the manufacturer’s protocol . In the end, the cells are also resuspended in 1 ml 1X PBS containing 0.04% filtered BSA. Cells were again counted with a Malassez counting chamber. According to the manufacturer’s protocol, a concentration between 700,000 and 1,200,000 cells per ml is required for the next step. Consequently the volume was adapted if needed to reach this concentration range.Single Cell RNA-Seq was performed using the 10X Genomics Chromium Next Gem Single cell 3′ Reagents Kits v3.1  and Dual Index kit TT set A  according to the manufacturer’s instructions. For the resting and stimulated samples, the target was estimated at 8000 cells, while for the fresh and frozen samples, the target was set at 6,000 cells. Briefly, the cell suspension barcoded gel beads and partitioning oil were loaded onto the 10X Genomics Chromium Chip (Next GEM chip G) to generate single cell Gel Beads In emulsion (GEMs). Captured cells were lysed and the transcripts were barcoded through reverse transcription inside individual GEMs. The constructed libraries were sequenced on Illumina NextSeq500 (FC High output 300 cycles) or Novaseq (FC S1 standard 200 cycles) and processed using Cell Ranger version 6.0.1 (10X Genomics).5 version 4.1.0 was used for quality control and most of the analysis. In addition, we used Scrublet6 for doublets estimation and the software tool MultiMAP7 for certain steps of dimensionality reduction. In order to annotate the clusters described on Fig. https://github.com/erbon7/sc_pbmc).The fastq and index files were processed with Cell Ranger and the human genome GRCh38 as the reference (10X Genomics annotation file refdata-gex-GRCh38-2020-A). The R package Seurat8.The raw data consisting of nucleotide sequences (FASTQ files) along with Cell Ranger filtered feature-barcode matrices  for each sample are publicly available in the NCBI GEO database (GEO:GSE226488)6, using the default parameters. The Cell Ranger summary reports, which include many different quality indicators, did not report any value outside the expected ranges. Table 9. This PBMC dataset comes from a healthy donor and was sequenced at a depth of approximately 110,000 reads/cell, for a target of 10,000 cells. The FASTQ files were downloaded from the site and processed with the same version of Cell Ranger and with the same parameters. We can see in Table After sequencing, all the samples were processed with Cell Ranger with default parameters for quality control and gene counting. For the resting and stimulated samples, we set the “–expect-cells” parameter to 8,000 and for the fresh and frozen samples we set this parameter to 6,000. We used the human genome GRCh38 as the reference (10X Genomics annotation file refdata-gex-GRCh38-2020-A). Doublets rates were estimated with the software ScrubletFor this analysis example, we filtered the two PBMC datasets to remove low quality cells. First we removed the doublets predicted by Scrublet (308 for resting PBMCs and 344 cells for stimulated). Then, we analysed the distribution of the median number of genes per cell and the percentage of mitochondrial genes per cell to define the filtration thresholds. We removed cells having a low or very high median number of genes , and cells having a percentage of mitochondrial genes greater than 14%. After filtering, we were left with 6,206 cells and 7,057 cells for resting and stimulated PBMCs respectively.10. The main applications of this technology are related to the discovery of cell types and states, the reconstruction of cell trajectories and fate decisions, and the modelling of spatially complex tissues. However for most studies data is generated separately, i.e. at different times and with different operators. For very large scale studies (e.g. a cell atlas), data may also be generated in multiple laboratories, with different cell dissociation protocols. These factors result in batch effects, where the expression of a gene in a batch differs systematically from that in another batch. These differences may introduce spurious structures in the data or mask the underlying biology, leading to wrong conclusions. Thus, it is necessary to correct such effects before further analysis11. Several methods have been developed to correct batch effects in single-cell transcriptomics data, using different statistical and mathematical frameworks (see for example13). It is essential to benchmark all those methods properly in order to evaluate precisely their respective strengths and weaknesses. For instance, Haghverdi and colleagues13 use different datasets to benchmark their mutual nearest-neighbor (MNN) algorithm: simulated data for a simple scenario with two batches of cells having varying proportions of three cell types, two hematopoietic datasets generated in different laboratories using different scRNA-seq protocols (SMART-seq2 and MARS-seq) and a complex dataset on human pancreatic tissue composed of four different sources of data, generated using two different scRNA-seq protocols (SMART-seq2 and CEL-seq2). Obviously, datasets generated from biological samples are highly valuable for these type of benchmarking studies, preferably generated using a variety of scRNA-seq protocols. Hence the interest in our resting versus stimulated PBMC dataset. The 13,263 resting and stimulated cells of our dataset are clearly separated on the two dimensional UMAP projection  and the MultiMAP method implemented in the Python package of the same name7. The Seurat alignment workflow (CCA) is based on the canonical correlation analysis, which aims at finding linear combinations of features across datasets that are maximally correlated, thus identifying shared correlation structures. The first step of this method is to find cell pairs that are in a matched biological state  that will be used to correct the differences between datasets. The reciprocal PCA (RPCA) is a slightly modified version of the CCA workflow, using reciprocal PCA to identify the ‘anchors’ between the datasets. This procedure is faster and constitute a more conservative approach where cells in different biological states are less likely to align after integration. The Seurat developers recommend using RPCA when a substantial fraction of the cells in one dataset have no matching type in the other, when datasets originate from the same platform, or when there are a large number of datasets or cells to integrate. MultiMAP7 is an algorithm for the dimensionality reduction and integration of multiple datasets. It constructs a nonlinear manifold on which diverse high dimensional data reside and then projects the manifold and data into a shared low-dimensional embedding space. MultiMAP is a generalization of the UMAP algorithm for multiple datasets with different dimensions. It can integrate any number of datasets, can leverage features that are not present in all datasets and is scalable to very large datasets. Figures 12. An interesting way to benchmark these data integration methods is to remove abundant or rare cells from one of the conditions  and run the alignment procedure to see how it handles the missing data. More than 50 integration methods are currently available for single-cell data integration. In order to efficiently benchmark them, there is a need for an objective methodology and metrics, i.e. the ability of the methods to remove batch effects while preserving biological variation, and of course a diverse set of robust datasets that reflect the variety and complexity of nonlinear, nested batch effects that can occur in real experimental conditions14.High-throughput single-cell transcriptomics has become a very popular and powerful tool for unbiased profiling of complex and heterogeneous cellular systems, thanks to the availability of commercialized workflows and improvements in cost and throughput12. As an example, we calculated the genes that are conserved markers in both stimulated and resting NK  cells, which are an important component of the innate immune response. We found that 1,088 genes have adjusted p-values for both resting and stimulated groups <0.05, of which 91 also have a fold-change greater than 1. Among these potential conserved marker genes, SH2D1B  is a cytoplasmic adaptor that regulates receptors of the signaling lymphocyte activation molecule (SLAM) family15. This gene is known to play a role in regulating the effector functions of natural killer (NK) cells by controlling signal transduction16. The Fig. IFI6 , also shown in the Fig. CXCL10 is significantly differentially expressed with an adjusted p-value of 1.45 × 10−81 and an average log2 fold change of 4.9, and indeed appears strongly upregulated in stimulated B cells on the Fig. CXCL10 is a pro-inflammatory cytokine involved in a variety of processes including chemotaxis, differentiation and activation of peripheral immune cells. It plays an important role during viral infections by stimulating the activation and migration of immune cells to the infected sites18. Note that more samples would be needed to strengthen the observations that we make here on this dataset.Once the cells have been integrated and the different cell types identified, it is possible to look for cell type markers that are conserved in both resting and stimulated conditions, or to compare the datasets to find cell type specific responses to stimulation. Here, we used the “FindConservedMarkers” function of the Seurat R package to identify canonical cell type marker genes that are conserved across the two conditions. This function performs a differential gene expression test for the for each group, in our case resting and stimulated, and combines the p-values using a meta-analysis method19. It is well known that once tissue samples have been removed from a patient, they promptly begin to undergo gene expression changes and RNA degradation21. Therefore it was not possible to process fresh and frozen samples the same day and on the same machine for our experimental design. This might theoretically induce a batch effect if the 10X Chromium machine has some technical problem but we think that it is very unlikely. A technical test note from 10X Genomics22 shows that for four PBMC libraries generated on three different chips (thus three different dates) the number of genes recovered and the cell clustering are very similar. There is also a critical step during the 10X library preparation, after the run of the chromium controller for the GEMs (Gel Beads-in-emulsion) generation where it is easy to visually assess whether the emulsion is correct, and we did not detect any problem at this step for the two runs. Finally, if a technical problem had occurred at the library step preparation, we would likely see major differences in the quality control values displayed in Table 24. Some of those protocols use dimethyl sulfoxide (DMSO) or methanol to cryopreserve cells. DMSO is frequently used to preserve animal cells and to prevent the formation of ice crystals. Methanol, on the other hand, dehydrate cells and cause nucleic acids to appear in a collapsed form, thus preserving them with only minor modifications. It is worth noting that some commercial formulations are readily available for scRNA-seq-compatible cell preservation.As described in the methods, we used technical replicates issued from the same individual to avoid biological variations which are quite common for PBMCs. In order to have realistic conditions for the frozen samples, they were kept in the freezer for seven days. For the fresh PBMCs, cells were processed immediately for single-cell library preparation as recommended by the manufacturer and as recommended for best results in single-cell library preparation


Previous research suggests that dog mass vaccination campaigns can eliminate rabies locally, resulting in large human and animal life gains. Despite these demonstrated benefits, dog vaccination programs remain scarce on the African continent. We conducted a benefit-cost analysis to demonstrate that engaging into vaccination campaigns is the dominant strategy for most countries even in the absence of coordinated action between them. And quantify how coordinated policy measures across countries in Africa could impact rabies incidence and associated costs. We show that coordinated dog mass vaccination between countries and PEP would lead to the elimination of dog rabies in Africa with total welfare gains of USD 9.5 billion (95% CI: 8.1 – 11.4 billion) between 2024 and 2054 (30 years). Coordinated disease control between African countries can lead to more socially and ecologically equitable outcomes by reducing the number of lost human lives to almost zero and possibly eliminating rabies. Control of rabies in Africa through mass vaccination of dogs may be compromised by cross-border transmission. Here, the authors use mathematical modelling and a benefit-cost analysis to demonstrate that coordinating rabies control measures across borders could lead to the elimination of dog rabies in Africa. Rabies is a fatal neurological pathogen primarily spread by dogs, and is estimated to cause ~59,000 human deaths, 3.7 million disability-adjusted life years lost and USD 8.6 billion economic losses annually2 and at least 250,000–500,000 dog deaths4. Due to successful anti-rabies campaigns in the past decades, Dog-transmitted rabies is unequally distributed, with the low and middle income countries carrying 99% of the burden5. The World Health Organization (WHO) aims at zero human deaths from dog-mediated rabies by 20306. Human deaths from rabies can be prevented through timely administration of post-exposure prophylaxis (PEP) for people who have been bitten. However, PEP availability remains scarce in most local healthcare systems and PEP compliance is poor and dog vaccination is limited.Rabies remains a neglected disease and persistent human and animal health problem throughout the low and middle income countries7, in principle dog-mediated rabies can be controlled and eliminated by the mass vaccination of dogs if a sufficiently high coverage can be achieved8. A One Health (OH) approach aims to demonstrate an incremental benefit from a closer cooperation between human and animal health that cannot be demonstrated if the sectors work in isolation10. Compared to PEP, dog mass vaccination has the main advantage that it has the potential to permanently interrupt transmission of rabies and to also prevent millions of unnecessary human and animal deaths11. The main challenges with dog rabies vaccination campaigns are access to vaccines, the high mobility among dogs, mostly through human-mediated dog transport13, resulting in an almost certain pathogen reintroduction of rabies at the local and national levels15. Large-scale synchrony of canine rabies incidence in Africa strongly indicates cross-border transmission of dog rabies1. Active protection of the country’s borders could avoid the reintroduction from the outside through direct dog-to-dog contact and human-mediated dog transport in principle16, but is likely not feasible across countries.While the use of PEP alone cannot address dog-mediated rabies17. We use this approach to investigate and illustrate the strategic dilemma underlying current policies and the potential benefits of reaching a socially optimal policy equilibrium through cooperation18.The constant threat of pathogen importation means that the impact of policy actions taken by each country becomes dependent on the efforts made by others. Such strategic policy choices can be analyzed through a mathematical framework (game theory). In such a framework, we monetize the outcomes of different policies for all interacting actors; that way, we can compare them to find the most profitable self-interested or cooperative choices. In this paper, we present the economic and health impacts at the country level of various combinations of human post-exposure prophylaxis and dog rabies vaccination programs throughout the African continent. The goal is to compare how and how much inter-countries cooperation and intersectoral cooperation affect subsequent health, financial and environmental outcomesOur model suggests a total canine population of 141,398,505  across the 48 countries included in our study for the base year 2024. For different strategy profiles, country-level payoffs are presented in the Supplementary Data 2.Our Susceptible-Exposed-Infectious-Removed (SEIR) model suggests a total of 15,275,368  humans exposed to rabid dogs for the baseline between 2024 and 2054. Thus leading to an estimate of 2,902,293 clinical cases without PEP  and a number of human lives lost around 459,068  assuming estimated incomplete PEP reach numbers as previously published by HampsonFigure The average estimated cost per vaccinated dog was 4.47 USD, ranging from 1.8 USD (95% CI: 1.46 USD–2.11 USD) for Burundi and 13.1 USD (95% CI: 10.12 USD–16.54 USD) for Equatorial Guinea per canine. The country-specific average number of rabies-induced years of life lost (YLL) in the working age (16–64 years of age) ranges between 23.5 years (Tunisia) and 26.4 years (Guinea and Uganda) with a median value of 25.8 years (mean = 25.6 Years). For the base year 2024, the country-specific capital loss of one life lost due to rabies is valued between USD 6764.04 (Burundi) and USD 238,904.47 , with a median value of USD 32,993.13 .Excluding the human capital effect (HCE) cost from the payoff is equivalent to comparing the costs of vaccination and PEP use. Considering the same strategy profiles as before and the aggregated payoff over all 48 countries, the breakeven point between the baseline strategy (1) and the coordinated mass dog vaccination 2) is in 2033. In comparison, the other strategies are less profitable than the baseline strategy . This means that without cross-country collaboration, the countries of Group 1 will reap the supplementary gains of the dog vaccination campaign reported in the column “One VAC” of Table By the year 2054 (before the African Union Agenda 2063), the gain from a nationwide coordinated vaccination campaign resulting in an elimination of rabies within the animal reservoir will lead to a total gain of 9547 m USD  when compared to the baseline.In our model, the variable having the most significant impact on the payoff variance is the uncertainty on the exposure with total-order Sobol indices around 0.65 [Q25–Q75: 0.58–0.76]. After that, the probability of receiving PEP with a mean value of total-order Sobol indices about 0.16 [Q25–Q75: 0.01–0.23], and for the initial dog population, the indices are around 0.14 [Q25–Q75: 0.10–0.17]. We have almost the same values for the first-order and total-order Sobol indices, meaning that almost no variance comes from the interactions of the studied variables. We mention that for countries with a high probability of receiving PEP (>99%), the variance of the distribution was reduced. For those countries, we can see that the human capital effect is reduced, so the impact of the vaccination price and the initial dog population are on par with the exposure uncertainty in the case of the pathogen reintroduction  in the elimination of dog rabies in Latin America21. The successful Rinderpest elimination in Africa by 2011 considered “the legitimate need of each stakeholder group and the power relationships”22 to translate the willingness to cooperate at the national, district/province, and the household level through public-private-community partnership. Such social innovations are part of an transdisciplinary OH approach23 and can enable the realization of the benefits through cooperation at the human-animal interface on a country level that otherwise remains unrealized. Further research should focus on this ecology of games24. We provide a brief discussion of the different layers of the underlying games in Supplementary Information 26. The situational analysis of the collaboration and coordination between sectors and countries revealed the need for multiscale political commitment based on robust economic incentives evidence of One Health. The effective collaboration relies on the institutions and countries’ governance, the capacity and knowledge in coordination, the transformed institutional and legal frameworks for complex health problem such as Rabies which is embedded in different ministries.The required regional collaboration to achieve sustained rabies elimination can inspire from similar experiences in the elimination of fox rabies in Europe through centralized financial incentives by the European Union27. Following the suggestion from Anyiam et al., we assume that a country’s whole dog population can be vaccinated within 1 year, which is necessary for rabies elimination due to the short average canine live-span28. As suggested by Mindekem et al., we emphasize the need for two consecutive vaccination campaigns, each with a reach of 70%, to eliminate rabies11. We based our model on the currently available data and acknowledge that rabies elimination might well take up to 10 years within a country. For our analysis, this would mean that human exposure would increase, thus reducing the overall benefit but not the utility ranking. A careful planning of a coordinated Africa-wide campaigns could be achieved through the well-established rabies vaccine bank at the World Organization of Animal Health (WOAH) and a good accessibility framework by countries as implemented by the COVAX initiative. The coordination mechanism in the strategic option implies a strong link between the PVS and the IHR, the legitimacy of the regional economics regions, mutual learning in the health systems10.Acknowledging that the scaling up the initial results of the SEIR model parameterized to the N’Djamena setting has its limitations, we have chosen to follow similar research that has built their continental models upon limited available dataPEP costs were assumed to follow the same distribution across all countries in scope. As this research is not targeted at deriving exact costs for intervention but on estimating utilities for a game-theoretical discussion, we were willing to take this approach as a starting point and expect heterogeneity of the underlying input parameter to be included in further research. Nevertheless, from our sensitivity analysis, we can argue that if the actual distributions and the ones used are not significantly different , the effect on the model would be marginal. Not all potential associated costs were included regarding monetization, but neither were all benefits. On the cost side, we have solely included the public costs for the PEP, assuming free public availability of PEP. This is the case for some countries like Mali but not for all. The costs for PEP administration would be higher if the opportunity costs of the personnel costs and storage would have been included. The benefit side captures only the direct effects of avoided YLL under GDP contribution and does not consider any additional effects like avoided preventive vaccination. Not including the full social and economic benefits results in underestimating the benefits. We are confident that our estimates are conservative, and the total benefit will likely be higher.29.Canine background vaccination at the household level does not reach sufficiently high levels to interrupt transmission due to financial constraints or lack of knowledge. Additionally, canine vaccination is not endorsed by the public health authorities as they see their responsibility in administering PEP. As the veterinary authorities do not see significant gains from vaccinating the canine population, an intervention from that side is not implemented either. The costs of rabies prevention through canine vaccination seem to exceed the status quo costs for any involved party. Through our work we add an additional overarching dimension to the integrated OH approach as suggested by Scoones et al. combining process, pattern and participation1. If countries take the human capital loss into consideration African inter-governmental platforms like the African Union Inter-African Bureau for Animal Resources (AU-IBAR) together with the African centre of Disease Control (African CDC) and the Economic Community of West African States (ECOWAS) One Health Coordination mechanisms composed of both West African Health organization (WAHO), the Regional Animal Health Centre (RAHC) and the Department of Environment (DoE) are best placed to achieve such coordination30. In this way, responding to the Lancet’s call for more ecological equity through a One Health approach31, the African continent could materialize these incredible financial gains and avoid the continuous loss of numerous human and animal lives, and eventually eliminate rabies.Large-scale synchrony of dog rabies incidence in Africa and the importance of intervention responses suggest that control of canine rabies in Africa will require sustained efforts coordinated across political boundaries15. This model allows for pathogen reintroduction and estimated country-specific dog-rabies cases incidence. The parameters of the Susceptible-Exposed-Infectious-Recovered (SEIR) model are derived from recent publications from the same context32 and can be found in Supplementary Table 27. The human population distinguished by rural and urban was derived from the World Bank and the total population as well as the population growth rate by the United Nations World Population Prospects 201933.We adapted and extrapolated an mathematical model (deterministic meta-population model) from Laager et al. for N’Djamena, Chad34. Secondly, an uncoordinated national level dog vaccination campaign, with medium term pathogen reintroduction from outside the country border. Thirdly, a coordinated regional vaccination campaign, with coordinated efforts both within countries , and cooperation between countries.Using all currently available data  per capita. The combination of the years of life lost with GDP contribution derived from the age distribution with the country-specific GDP contribution results in the weighted average years of live lost (YLL) with GDP contribution for one average bite victim in a specific country. The details of this methodology are presented in Supplementary Information 28 we calculate the intervention costs of each strategy covering the PEP costs for treating bite victims of suspected rabid dogs according to the estimated PEP reach numbers as previously published by Hampson2  or combining it with a mass dog vaccination campaign. The total cost of the strategy depends on 1) the vaccination cost, (2) the post-exposure prophylaxis cost and (3) the human capital effect cost. Pairing the resulting number of rabid dogs from the SEIR model with a multiplication factor of 2.3 (the exposure factor in our model) in the same geographical location from the 2013 animal bite survey on animal bite injuries the vaccAs each country can make a different choice, we have a large number of possible combinations. Although, for our analysis, only some are sufficient. Therefore, we will consider the following strategy profiles : all countries stay at baseline to continue to react to human cases using PEP , all countries collaborate to realize a mass dog vaccination program , and finally, the case where only the studied country mass vaccinates dogs meaning that we have a pathogen reintroduction in the midterm. For the last combination, we have 48 “one VAC” profiles (one per country), which we summarize as one column by keeping only the relative gains or losses, as only the studied country will have non-null results in this profile , or the one corresponding to the profile with the highest possible gains in total for all countries , which is in our case the cooperative solution. The mathematical (game) definition and proofs are in Supplementary Information All authors have made essential contributions to the conceptualization, data analysis and writing of this study. As a desk study using publicly available data without including data collection on humans or animals, we consider that there are no ethical issues with this work.Further information on research design is available in the Supplementary InformationPeer Review FileDescription of Additional Supplementary FilesSupplementary Data 1Supplementary Data 2Supplementary Data 3Supplementary Data 4Supplementary Data 5Supplementary Data 6Supplementary Data 7Supplementary Data 8Supplementary Data 9Reporting Summary


Repetitive negative thinking (RNT) is a cognitive risk factor for various disorders. Although brief mindfulness-based interventions  are effective tools to reduce RNT, the effect of a minimal (5-minute) MBI remains largely unknown.We investigated the acute changes in RNT induced by a 10-day minimal MBI (body scan before sleeping) using an ecological momentary assessment (EMA) administered during the MBI training phase. In addition, we examined longer-term effects on the postintervention and 2-month follow-up assessments for questionnaire-based RNT and psychological distress.A total of 68 participants  were randomly allocated to either the intervention group  or the no-training control group . Both groups completed a 10-day EMA phase of RNT, during which only the intervention group performed a daily 5-minute body scan before sleeping.dGMA=−0.91; P<.001), but this effect was not observed during the EMA phase or at the postintervention assessment. Furthermore, the intervention group showed significantly larger decreases in stress both at the postintervention  and follow-up  assessments than the control group. We found no intervention effects on depressive and anxiety symptoms.The intervention group showed a significantly larger reduction in questionnaire-based RNT than the control group at the follow-up assessment (for growth-curve modeling analysis [GMA], A 5-minute body scan before sleeping reduces RNT and stress when continued for at least 10 days; however, the results suggest that this effect only appears with some time lag because no acute changes during and immediately after the intervention emerged for RNT. It is estimated that approximately one-third of the global population is affected by mental disorders , which aAs a promising treatment approach for a variety of mental health outcomes, mindfulness-based interventions (MBIs) have increasingly captured the interest of the scientific community over the past 3 decades . StandarHowever, programs such as MBSR and MBCT require substantial time investment from practitioners . Hence, Although brief MBIs are suitable for addressing the accessibility issue, a consensus has not yet been reached concerning how much exercise should be included in a single session and for how many days the intervention should be continued, which poses an obstacle for the widespread implementation of MBIs as a mental health tool ,18. A syHowever, the longevity of the effects of brief MBIs is unclear ,20, and To expand these earlier EMA-mindfulness works, this study used EMA to examine the effect of a minimal  MBI on a marked cognitive vulnerability factor for psychopathology, that is, RNT. We targeted RNT as the main outcome here because the theory underlying MBCT  highlighAlthough RNT is typically operationalized as a ruminative tendency (or trait factor), it has been shown that RNT has some temporal variability that may hint at how RNT-focused interventions can be implemented. An EMA study  found thIn this study, participants performed a 5-minute body scan before sleeping, which we expected to prevent the occurrence of, and help disengage from, RNT. As supporting evidence for an MBI at night, it has been suggested that a 6-minute mindfulness induction before sleeping improves athletes’ sleep quality  . TherefoParticipants were recruited at the university campus of Ludwig-Maximilians-Universität München (LMU Munich) and from the Munich metropolitan area on the web , as well as via flyers and by using the university’s email distribution list (covering graduate and undergraduate students). Participants were required to be aged at least 18 years and fluent in German and were excluded if they had practiced mindfulness daily in the past 3 months before the study  or had ag=−1.13 in the postintervention levels of rumination in nonclinical samples, which gives the required sample size as 28, with the assumptions of α=.05 and power=0.80.An a priori power analysis was conducted using G*Power  to deterParticipants attended the preintervention assessment (T0) at the laboratory of LMU Munich, where they provided written informed consent and completed questionnaires for demographics and baseline symptomatology. Note that data were collected within a larger study  using a test battery, including other measures, that are not reported here. Participants were randomly allocated to either the intervention or the no-training control group following the randomization sequence that had been created before data collection. No blinding procedures were performed. Participants were instructed to use the EMA app , which they had installed on their Android smartphone before the appointment . Upon complying with this instruction, participants read through all EMA questions, and any unclear points regarding the study procedure were resolved by an experimenter. The intervention group received further instructions for the 5-minute MBI. The day after the T0 assessment, all participants started the 10-day EMA and, if they were allocated to the intervention group, the MBI training sessions. Participants completed the same questionnaires as the ones administered at T0 on a web-based platform the day after the completion of the 10-day EMA  and 2 months later . Furthermore, the T2 assessment included 1 question asking whether participants had performed mindfulness practice during the period between the postintervention and follow-up assessments  per hour for assessments and had the chance to win 1 of 10 €20 (US $22) vouchers, depending on their compliance during the EMA. For student participants, course credit was given based on compliance during the EMA. In addition, all participants who completed both T1 and T2 assessments had the opportunity to win a €50 (US $55) voucher. The data collection started in November 2019 and ended in May 2020, and we did not change any aspect of the study protocol even after the COVID-19 pandemic.The ethics committee of the Department of Psychology at LMU Munich approved the study protocol .Participants in the intervention group performed a 5-minute guided version of the body scan derived from MBSR  as an MBnever to 4=almost always. The total sum score for all 15 items was computed. The PTQ is a valid measurement instrument for RNT [RNT was assessed using the 15-item Perseverative Thinking Questionnaire (PTQ) . Three s for RNT . In thisdid not apply to me at all to 3=applied to me very much or most of the time. The total score for each subscale was computed . The DASS showed good internal consistencies: Cronbach α=.83 (depression), Cronbach α=.77 (anxiety), and Cronbach α=.83 (stress).Depressive, anxiety, and stress symptoms were measured using the German short form  of the Dnot at all to 7=very much concerning their current RNT: that is, “The same thoughts keep going through my mind again and again” (repetitiveness), “Thoughts come to my mind without me wanting them to” (intrusiveness), “I get stuck on certain issues and can’t move on” (uncontrollability), and “I feel disturbed by negative thoughts” (distress). The distress item was adapted from the original wording [unpleasant to 100=pleasant). We found no significant effect of the MBI on the affect  ratings, and thus we focused exclusively on RNT in the Results section. The first EMA prompt of each day was sent at 9 AM, and the remaining 4 prompts were scheduled at random times between 10 AM and 10 PM, each prompt separated by an average of 2 hours. The estimated time to answer all questions was 2 minutes.The EMA was conducted using a smartphone app, MobileQ . Partici wording  to fit i wording ). A dura wording . The num wording . Further wording  because lme4 [lmerTest [Multilevel modeling was performed on the intention-to-treat sample with (restricted) maximum likelihood (ML) estimation implemented by the R packages lme4  and lmerlmerTest . This anPrompt level:Person level:ijRNT is the momentary level of RNT for participant i for the prompt j of the EMA. Group is a dummy code indicating group allocation, with 0 for the control group and 1 for the intervention group. The residual is denoted by r. The intercept and slopes (β0j and β1j) were allowed to vary across participants with person-level random effects (u0j and u1j), and the cross-level interaction effect (γ11) represents the group differences in the (linear) change over the MBI training phase. In other words, we tested whether the rate of change in RNT (β1j) would differ between the 2 groups, and this difference was modeled by the interaction effect γ11. We performed model selection using information criteria (with the ML estimator) to see whether a polynomial function would better approximate the changing pattern of RNT than a linear function of time. However, we found that the linear function was better than the quadratic and cubic functions, and thus we decided to focus exclusively on the linear change in the Results section.To test the effects of our MBI at the T1 and T2 assessments , we estimated a similar 2-level model for each of the 4 outcome measures (but with the restricted ML estimator): the PTQ and DASS depression, anxiety, and stress subscales. Each model is formulated as follows:Assessment level:Person level:ijOutcome is the outcome value of participant i at assessment j. The time variables DT1 and DT2 are dummy codes representing the differences between the T0 and T1 assessments and between the T0 and T2 assessments, respectively. Both the intercept and slopes  were allowed to vary across participants with person-level random effects . The variance of random effects was fixed at 0 if the estimates exceeded 0. The 2 interaction effects γ11 and γ21 were of particular interest because they represented group differences in the changes between the 2 given assessments.dGMA was calculated by first multiplying the fixed effect by time  and then dividing the product by the raw SD of the outcome. This metric is comparable with Cohen d. For each analysis, we tested whether the results were unchanged even after controlling for covariates .Post hoc (simple slope) tests were conducted for any significant interaction effects between time and group. Effect sizes were defined in the framework of the growth-curve modeling analysis (GMA) , that isThe final sample consisted of 68 participants aged 18 to 55  years . The descriptive statistics for the T0, T1, and T2 assessments are presented in group, time (EMA prompt), and their interaction as predictors of EMA-assessed RNT . Our design might be suboptimal to capture the immediate effect of a body scan on RNT at night.To test the temporal changes in RNT throughout the MBI, a multilevel model was estimated with ssed RNT . The resssed RNT . As an egroup and DT2 (but not DT1). Both groups exhibited an equally significant reduction from the T0 to T1 assessments . However, the intervention group showed a larger reduction from the T0 to T2 assessments than the control group . These results suggest that the PTQ score continued to decrease even after the completion of the MBI . However, for the control group, participants who spontaneously performed mindfulness practice after the T1 assessment up to the follow-up assessment showed lower PTQ scores than those who did not .To test hypothesis 2, separate multilevel models were estimated for each outcome measure of the PTQ and the DASS. For the PTQ, the results  showed s the MBI A. The reB=−2.65, SE 0.55; t89.6=−4.80; P<.001; dGMA=−0.78) and from the T0 to T2 assessments  than the control group , but this covariate reduced the effect slightly at DT2 . This may point to the possibility that the sustained stress reduction at the follow-up assessment is attributed to the spontaneous engagement in mindfulness practice after the postintervention assessment. However, an exploratory analysis identified no significant difference in the levels of stress at the follow-up assessment between participants who continued mindfulness practice and those who did not  for the MBI group. For the control group, participants who spontaneously performed mindfulness practice after the T1 assessment up to the follow-up assessment showed slightly lower stress levels than those who did not .For the DASS, we found no significant main or interaction effects on the depression and anxiety subscales . HoweverBrief MBIs are a promising tool for improving mental health in an accessible way. However, little is known about the real-time effect of minimal  MBIs on RNT, which is assumed to be the underlying reason for the effectiveness of an MBI. This study investigated whether a minimal MBI (a 5-minute body scan performed daily for 10 days) before sleeping would have positive effects on RNT and psychological distress symptoms  by comparing an intervention group with a no-training control group. First, we examined RNT in real time using EMA during the MBI training phase (hypothesis 1). Contrary to our expectations, the results showed no significant group differences in RNT during the training phase. Second, we examined the effects of the 5-minute body scan immediately after the intervention (T1) and at the 2-month follow-up assessment . We found that both groups had significant reductions in questionnaire-based RNT at the postintervention assessment ; however, the intervention group showed a significantly larger reduction than the control group at the follow-up assessment. As hypothesized, only the intervention group showed a decrease in stress at both postintervention and 2-month follow-up time points. We did not find any intervention effects on depressive or anxiety symptoms.Taking a closer look at hypothesis 1, the results regarding the momentary levels of RNT, as assessed by EMA, showed that a minimal MBI may not induce acute changes in RNT during the training phase. This finding is inconsistent with the theory  and contExamining hypothesis 2, we found that a minimal MBI administered before sleeping is effective in decreasing questionnaire-based RNT, although significant group differences emerged only at the T2 assessment and not at the T1 assessment. Although the effect of the intervention started to appear early, it was only after continued improvement during the follow-up phase that the questionnaire-based RNT scores were significantly different from the control condition, that is, the effect seems to have accumulated gradually, and the difference from the control condition became visible not immediately during and after the intervention but at the 2-month follow-up assessment with increased mindfulness practice experience. This explanation is supported by the fact that 61% (19/31) of the participants in the intervention group continued mindfulness practice even after the MBI training phase. However, the exploratory analysis revealed no statistically significant difference in RNT levels at the T2 assessment between participants who continued the MBI and those who did not, which contradicts the idea that the MBI effect accumulates through spontaneous practice. Another aspect that should be mentioned is that the control group showed a significant decrease in RNT at the postintervention assessment. This decrease may suggest that the EMA worked as a self-monitoring technique, which helped participants to realize how frequently they had engaged in RNT. In addition, control participants who spontaneously started mindfulness practice after the T1 assessment showed lower levels of RNT and stress than those who did not. Taken together, it seems that the spontaneous use of mindfulness practice plays an important role to maintain and enhance the effect of mindfulness practice, but we cannot draw a solid conclusion only from these data. Furthermore, because a previous EMA study  documentExamining psychological distress symptoms, we found that participants in the intervention group showed significantly larger decreases in their stress levels than those in the control group at both the T1 and T2 assessments. This finding is in line with the body of evidence reporting the beneficial effects of an MBI on stress ,13,57. HMoreover, stress may be more acutely influenced by mindfulness practice than depressive and anxiety symptoms via autonomic arousal: in a study measuring heart rate variability over a 10-day MBI, perceived stress and acute physiological stress decreased for the mindfulness practice . By contWe observed excellent compliance rates for MBI completion in this study, underscoring the feasibility of our 5-minute body scan. In addition, we noticed continued interest in training in our sample because the majority of the participants  spontaneously maintained mindfulness practice after the active training phase. Thus, our data suggest that a 5-minute MBI at night is feasible and that the minimal time investment of 5 minutes can be seen as getting a foot in the door of mindfulness practice, leaving room for the possibility that those who experience benefits may eventually increase their practice times.Participants subsection of the Methods section), and this may have prevented us from detecting a significant effect of the MBI on EMA-assessed RNT. Third, adherence to the MBI was only measured via self-report, leaving the possibility that the actual amount of mindfulness practice differed from reported adherence. Fourth, we chose to implement an MBI at night; however, we were unable to incorporate conditions examining the effect of the MBI at other times of the day. Therefore, the opposite of our proposed rationale may be true. Although we have argued that targeting RNT at night when it is at its highest [Similar to the findings of other studies in the field, our findings need to be interpreted considering a number of limitations. First, we did not use a blinded experimenter, and participants in both groups were informed about the aim of the study. Therefore, we cannot exclude the possibility that the results are biased by social desirability and a tendency to please the experimenter. This may explain the frequent (and somewhat unexpected) engagements in mindfulness practice among control participants, who might have been primed by the study information and assessments. Second, our data had an imbalance in terms of gender because the majority of the participants  in our study were women. Another potential issue concerning the sample characteristics is that we did not selectively recruit individuals with high levels of RNT or psychological distress. This is because we wanted to show the effectiveness of a minimal MBI used daily in a nonclinical population as a method of RNT reduction and stress management. However, the downside is that the effect might be underestimated because many of the participants showed low-to-moderate levels of RNT and distress at baseline. In addition, it should be noted that the sample size was relatively small  at different times of the day  to establish concrete recommendations for individual practitioners. This would enable the optimization of an MBI that could effectively promote mental health both acutely and in the long term.


Deep learning models are seeing increased use as methods to predict mutational effects or allowed mutations in proteins. The models commonly used for these purposes include large language models (LLMs) and 3D Convolutional Neural Networks (CNNs). These two model types have very different architectures and are commonly trained on different representations of proteins. LLMs make use of the transformer architecture and are trained purely on protein sequences whereas 3D CNNs are trained on voxelized representations of local protein structure. While comparable overall prediction accuracies have been reported for both types of models, it is not known to what extent these models make comparable specific predictions and/or generalize protein biochemistry in similar ways. Here, we perform a systematic comparison of two LLMs and two structure-based models (CNNs) and show that the different model types have distinct strengths and weaknesses. The overall prediction accuracies are largely uncorrelated between the sequence- and structure-based models. Overall, the two structure-based models are better at predicting buried aliphatic and hydrophobic residues whereas the two LLMs are better at predicting solvent-exposed polar and charged amino acids. Finally, we find that a combined model that takes the individual model predictions as input can leverage these individual model strengths and results in significantly improved overall prediction accuracy. In fact, several deep neural network models, including large language models (LLMs) and convolutional neural networks (CNNs), have helped with the engineering of proteins with enhanced function/stability10. Interestingly, the deep learning models employed to date differ substantially in their architectures and, more importantly, the type of input data they are trained on. While LLMs are typically trained on large amounts of sequence data, CNNs and other model architectures have also been trained on input consisting of protein structures13, and models trained on both types of input data have produced meaningful insight on protein variants.Machine learning models such as neural networks are increasingly being used to computationally explore protein variants suitable for protein engineering and to predict their effects on protein structure and function15, predicting post-translational modifications and biophysical attributes4, predicting protein structure17, and even predicting entire protein-protein complexes18. At the same time, CNNs trained on structure data have been successful in enhancing enzyme function activity by suggesting activity enhancing protein variants10. Because sequence- and structure-based models learn from different protein representations, it is not clear whether they have inherent differences or instead make substantially equivalent predictions. Additionally, because the different model architectures have typically not been applied head-to-head to the same problem, excellent performance of one architecture in one problem area does not imply that a different architecture might not perform similarly well. Further progress on this question requires a direct comparison of different model architectures on the same problem.Sequence data is significantly more abundant than structure data, so it would be convenient if models could be trained purely on sequence data. And in fact, sequence-based models, and specifically protein large language models (protein LLMs) using transformer architectures, have been successfully employed in a number of contexts, including predicting variant effects and protein fitness4 and ESM1b5, and we compare them to two structure-based 3D CNN models, which we refer to here as CNN19 and RESNET12. We find that all these models generally predict the masked residue with similar accuracy. However, because the models differ in how they “see” protein biochemistry and what they take as input data, they tend to make distinct predictions that reflect different aspects of the underlying protein biochemistry. Further, we show that we can improve model performance by combining the models into a single, joint model. The combined model can compensate for weaknesses of the individual models by learning how to integrate the various model predictions and preferentially relying on the specific predictions that it estimates to be the most accurate for a given site in a given protein structure.Here, we compare two existing sequence-based and two existing structure-based models head-to-head on the task they were originally trained for, the prediction of a masked residue. We use two different sequence-based LLMs, protBERT5 and protBERT4, and two structure-based models, a 3D CNN19 and a Residual Network (RESNET)12. Our first question was whether any one of the models was consistently outperforming the others. We generated predictions for every residue in a test set of 147 protein structures. We found that the average accuracy across the 147 structures was 60.74%, 64.4%, 64.8% and 68.3%, respectively, for the ESM1b, CNN, RESNET and protBERT models , passes them into two intermediate dense layers, and ultimately outputs a set of 20 probabilities, where each node represents the probability of one of 20 amino acids Fig.  using a We next assessed how well the different models predict specific amino acid classes. We pooled all sites from all 147 test proteins and divided the sites by amino acid class . Therefore, we used it here as an approximation of model accuracy, which cannot be defined for individual sites (a site either is or is not predicted correctly). We found that all models had a tendency to have higher confidence in their predictions for buried residues (RSA near zero) than for exposed residues (RSA of 0.2 or larger)  contributed to differences in predictions between the models. We did this by correlating the Relative Solvent Accessibility (RSA) of each site in our dataset with the model confidence for that site. Model confidence is defined as the probability with which the top scoring amino acid is predicted at a site, and it correlates well with model accuracy  Fig. . HoweverWe have compared the performance of two existing structure- and two existing sequence-based neural network models on the task of predicting masked residues in proteins, the task that all these models were originally trained on. We have found that the two different model types vary widely in their performance on specific proteins, even if their average accuracies are similar. Predictions by structure-based models are not correlated or at most weakly correlated to predictions by sequence-based models. The structure-based models perform best at predicting aliphatic core residues, whereas the sequence-based models provide more accurate predictions for polar, solvent-exposed sites. We have further constructed a combined model that uses the predictions of the individual models as input and turns them into an ensemble prediction. This combined model has achieved significantly higher prediction accuracy than the individual models, and importantly, can leverage the individual models for their specific strengths. The combined model seems to be able to select the individual model predictions that are most suitable for specific sites.22. More recently, the self-supervised technique masked-language modeling (MLM) used to train LLMs has been applied to predict masked residues in protein sequences6. Such LLMs can capture interactions between amino acids that are far apart in the sequence but spatially close in the structure, relying solely on input sequence for inference, without the need for additional features or annotations17. By contrast, 3D CNNs are fundamentally different from LLMs as they take the local protein structure rather than the entire amino-acid sequences as input23. The input consists of a voxelized box built around a focal residue which is removed before training or inference. The advantage of this structural data as model input is that it directly provides information on physical contacts within a protein, which are most critical for accurate residue prediction19.Sequence-based language transformer models and structure-based convolutional models differ widely in their training data, their internal structures, and their strengths and weaknesses. The transformer architecture has grown popular in the fields of natural language processing and image recognitionBecause LLMs do not use any structural information as input, we might expect them to perform consistently worse than 3D CNNs at predicting masked residues. However, we have seen that this is not the case for the four models considered here; both model types predicted residues with comparable overall accuracies. At the same time, there were clear differences between model types. The structure-based models tended to perform better at predicting aliphatic or hydrophobic residues, whereas the LLMs had an edge for polar or charged amino acids. These differences may be due in part to the fact that the latter amino acids are more common on the surface of the protein, where the voxelized input boxes to the structure-based models are partially empty and thus provide less data to base inference on.26. Considering that the distributions of probabilities can differ widely even if the same amino acid is predicted to be the most likely, we expect that these distributions carry meaningful information about the microenvironment surrounding the focal residue in the folded protein. Moreover, we would expect similar microenvironments to have similar distributions27. Consequently, the combined model can use the probability distributions of the four input models to infer the likely microenvironment of the focal residue and then infer which amino acid to place at this location. This is not a simple majority rule among the models but a non-trivial inference task, as can be seen from the fact that the model on occasion makes unique predictions distinct from any of the input models.The consequence of the differing strengths and weaknesses of the two model architectures is that the different models can be combined for improved overall performance. The network output for all four models consists of a distribution of 20 probabilities, one for each of the amino acids. This distribution of probabilities can also be viewed as an embedding for a biochemical environment28, respectively, whereas the structure-based models we used here (both with 12 M parameters) were trained with 30. We saw evidence of potential overtraining when we used more recent models of protBERT and ESM1-b, as well as ESM1-v6. For these models, we obtained accuracies averaging at One challenge we encountered with LLMs is achieving a clean separation between training and test datasets. Because the LLMs require very large training datasets, they are frequently trained on virtually any unique protein sequence available. To put into perspective, ESM-1b (650M parameters) and ProTrans (450 M parameters) were trained with In summary, our results show that two different neural network types, due to differences in how they learn protein features, can make unique contributions to the same residue prediction task. At the same time, predictions that match between a sequence-based and a structure-based model are most likely to be correct, implying that although model predictions are not strongly correlated they do show some overlap. It remains an open question to what extent our results here generalize to other models with similar or differing architectures, and whether LLMs can in principle be trained to the point where they perform as well as or better than a structure-based model. One conceptual difficulty with this question is that LLMs use such large training sets and so many parameters that they in essence memorize the entire known protein universe. To what extent such models actually generalize protein biochemistry is difficult to assess.5 and protBERT4, and two structure-based models, CNN19 and RESNET12. The two structure-based models have been trained on the same dataset, while the LLMs have been trained on separate and distinct training datasets. For all models, we defined model performance as the accuracy with which each model could predict masked amino acids in an array of different proteins.We used four pre-existing self-supervised deep learning models trained to predict masked residues: two sequence-based transformer models, ESM1-b19. Our test set is derived from the PSICOV dataset31, which consists of 150 well studied protein structures commonly used for covariation analyses. From this dataset, we removed three proteins for technical reasons, as described19. We verified that none of the remaining 147 proteins were in UniRef50, the training set of the ESM1-b model. Similarly, the CNN and RESNET training set had been chosen to not contain proteins with a sequence similarity of greater than 50% to any of the proteins in the test dataset19. Because protBERT has been trained on nearly every sequence in the UniRef RCSB databank28, we could not guarantee that our test sequences were not in the training data for that model.We first assessed each model separately, using a test dataset we had previously used in studying the 3D CNN modelhttps://github.com/tijeco/berteome. The berteome library requires prior installation of the transformers package available at: https://pypi.org/project/transformers/. We used version 4.10.0 of the transformers package. The berteome library generates ESM1-b predictions from the pre-trained esm1b_t33_650M_UR50S model5 and protBERT predictions from the pre-trained Rostlab/prot_bert model4. CNN model output was generated using the network with input box size of 20 Å3 as described19. The CNN model is available at https://github.com/akulikova64/CNN_protein_landscape and the RESNET model is available at https://github.com/danny305/MutComputeX.LLM output was generated using the berteome library (version 0.1.6) available at: 32.After assessing each individual model, we used the outputs of all four neural networks as training data for a combined model. The combined neural network model consisted of four layers: an input of 80 nodes (20 per model), two intermediate layers of 120 nodes and 60 nodes (“relu” activation function), respectively, and a final output layer of 20 nodes (“softmax” activation function). Each node in the output layer represents the probability of one of the 20 amino acids. All layers are fully connected. The model was implemented in python (v3.8.9) with the Tensorflow library (v2.9.2) using the keras API (v2.9.0)https://cdn.rcsb.org/resources/sequence/clusters/clusters-by-entity-80.txt. To avoid reusing any proteins previously used to train the individual models, we removed any proteins that were part of the training sets of the 3D CNN (training set as described19), RESNET and ESM1-b (UniRef50), as well as the test set (PSICOV) and their homologs of 80% or greater similarity. We did not filter out proteins used to train the protBERT model as it was pre-trained on the UniRef100 dataset, containing most of the proteins in the RCSB database. Proteins longer than 1024 residues were also removed due to length restrictions of the ESM1-b model. Finally, we removed protein structures where we could not add hydrogen atoms or partial charges with PDB2PQR (v3.1.0)33. The resulting training set contained a total of 3209 protein structures.To compile a training set for the combined model we first downloaded all PDB ID’s of proteins clustered into 80% homology groups from the RCSB website using the following link: To produce training data for the combined model, we generated CNN and RESNET predictions for each residue in our 3209 protein structure files. Similarly, the sequences corresponding to the 3209 protein structures were used to generate predictions from the protBERT and ESM-1b models. The combined network was trained on each protein/sequence position in this dataset for 150 epochs, on the goal of being able to predict the residue at each position in a protein. We used a fixed learning rate of 0.0001 using the Adam optimizer and the “categorical crossentropy” loss function. Finally, the fully trained combined network was tested against the same set of 147 proteins we had used initially to test individual models.34 and applying the normalization constants as described35.Relative solvent accessibility values were calculated using freeSASA softwareSupplementary Information 1.


To describe the impact of the COVID-19 pandemic on tuberculosis services and the different approaches healthcare workers adopted to ensure continued tuberculosis service delivery in Eswatini.This is a qualitative study with a cross-sectional design.Thirteen nurses and 9 doctors who provide tuberculosis care from 10 health facilities participated in an in-depth interview to describe how the COVID-19 pandemic affected tuberculosis services and the approaches adopted to ensure continued patient care. Twenty in-person and 2 telephone interviews were conducted. The participating facilities were selected based on a ranking criterion of the number of patients seen. Data were analyzed using thematic content analysis. NVivo 12 software was used for qualitative analysis, and the Consolidated Criteria guided the study for Reporting Qualitative research (COREQ).Two major themes emerged: COVID-19 impacted services delivery and access; and best practices that ensured healthcare services delivery. Six sub-themes describe how COVID-19 impacted services: all attention focused on COVID-19; COVID worsened the health system challenges; COVID hindered patients from accessing care; patients defaulted due to the lockdown; COVID impacted the quality of care and increased the risk of infection among healthcare workers. Five sub-themes describe best practices that ensure continued service delivery: Home-based care, Patient support, Patient Education, Integrated Services, and Staff rotation.While various strategies were adopted globally to mitigate the impact of the COVID-19 pandemic, these strategies need contextualization to be effective and sustainably incorporated into routine care to ensure continuity of and access to TB and other healthcare services. ImpactsData from different World Health Organisation (WHO) regions indicate access to other health services including HIV, malaria, vaccination, non-communicable diseases, and mental health were affected . The resLike other countries, the declaration of COVID-19 as a global pandemic in March 2020 by WHO activated a national response in Eswatini which mandated the use of face masks, hand washing and sanitization, closure or limited operations at some institutions and the general restriction in movement . These m22.1TB services in Eswatini are coordinated by the Ministry of Health (MOH) through the National Tuberculosis Control Programme (NTCP) . The NTC2.2This qualitative study, evaluated cross-sectionally, is based on the socio-ecological framework and is part of a prospective cohort study which has been described earlier . Healthc2.3The eleven health facilities selected for this study consist of five hospitals, one health centre and five primary care clinics across the four regions in Eswatini. The facilities were purposively selected based on a ranking criterion of the highest number of patients two quarters before the study. Doctors and nurses were invited to participate if they had worked at the health facility for a minimum of one year and provided care for TB patients. Those who met the inclusion criteria and provided informed consent were included.Interviews were conducted between May and June 2022 by a trained research assistant assisted by the principal investigator at the clinic. Participants informed consent included consent to record the interviews. One participant declined to consent to the recording and interview notes were taken. Participants received an orientation on the study and a study information sheet. In-person interviews were conducted with 20 participants and two participants were interviewed telephonically. A semi-structured interview guide (Supplementary File 1) was used for the interview. Each interview lasted 30–45 minutes and healthcare workers used the language of their preference (English or Siswati). The study was approved by the Eswatini Health and Human Research Review Board (EHHRRB-036/2021).2.4Each interview was transcribed immediately afterwards without linking it to the healthcare worker. The research assistant and the study principal investigator reviewed each transcript against the interview recording to ensure the accuracy of the transcript. Inductive and deductive codes were identified from the transcripts and documented in a codebook (VW). Similar codes in the codebook were grouped into sub-themes and similar sub-themes were grouped into major themes for the study . Three research team members reviewed the code book and the themes for accuracy and consistency. Inaccuracies in the codes, sub-themes, and themes were resolved in a study meeting. Reviewing the codes was iterative to ensure the finalized themes were accurate. The results were presented using text analysis. NVivo 12 software was used for qualitative analysis and the Consolidated Criteria for Reporting Qualitative Research (COREQ) guided the reporting of this study ,14  were afraid to come to the clinic because they thought they would contract COVID-19 from the facility” R5“… Patients were not coming to the clinic for monitoring and check-ups due to fear of contracting COVID, this led to other patients being severely attacked by TB and others had their DM escalated” R7“With the uncertainty around COVID-19, most chronic care patients stopped visiting health facilities to avoid infection, hindering routine care, and increasing their risks of poor treatment outcomes.3.2.4The respondents observed that some patients, besides not visiting the health facilities, stopped taking their medications while others could not be reached or contacted . These l3.2.5Some of the COVID-19 infection prevention measures instituted at different health facilities may have impacted the quality of care provided by healthcare workers . Limitin3.2.6Sputum induction was stopped due to the pandemic since we were in fear of the exposure to COVID-19” R2“We were afraid of the patients because we were afraid of contracting COVID-19 from them” R10“One can also talk about the issue of human resources during COVID-19 since many of our staff were affected (infected) this affected the standard of care for patients” R19“Respondents had reservations about interacting with patients as they were at an increased risk of infection with COVID-19. Some of their colleagues nonetheless got infected, reducing the number of staff available for patient care, and further impacting the quality of care negatively.3.33.3.1We usually do community and home visits wherein we do TB contact tracing and TB history; this helps us to uncover TB presumptive cases” R6“we are doing home visits to deliver their medication to relieve them with the transport fare …” R21“… An adaptation to ensure continuity of services for patients was community and home-based care. Patients received essential medical care and medication refill for chronic conditions within their community, at home, or at a pre-arranged service delivery point.3.3.2The respondents also provided different types of support for patients during the pandemic to encourage them and to ensure they continue with their treatment . Some of3.3.3We have dedicated our Wednesdays… for teaching and review of the patients. TB treatment also involves teaching the patient about health education and how to prevent the spread of TB even at home” R10“The respondents felt health education was important to empower their patients to become responsible for their health and minimize TB infections within the community.3.3.4We have introduced the one-stop-shop, wherein we have integrated all the services at one point” R15“Offering services at different sections of a health facility can prolong the time a patient spends at a health facility and requires more personnel. This may discourage some patients from accessing care. To avoid this, the respondents indicated they introduced a system where possible services a TB patient will require during a visit are provided at one service point. This will reduce waiting time, encourage patients to keep their appointments and minimize required staff.3.3.5To ensure continual service provision we do staff rotation, if a TB nurse is not available, one is always assigned to the TB department” R4“Yes, we have enough clinical staff. We normally rotate so that whenever there is a gap someone can cover it up” R9“Limited staffing and skillset are challenges at most health facilities. This was obvious at the peak of the pandemic when several healthcare workers were infected and had to isolate. The respondents stated they had a process where staff were intermittently moved to provide services at other units to avoid delay or suspension of services.4Based on the views of healthcare workers, we describe some of the impacts of the COVID-19 pandemic on TB service delivery and access in Eswatini; and some approaches by healthcare workers to ensure the continuity of TB services. At the height of the pandemic, COVID-19 took priority over other conditions, worsening health system challenges which were already in a precarious state. Different measures adopted to limit the spread of COVID-19 infection hindered patients from accessing care as some services were suspended or limited. This affected the number of patients accessing care and the quality of services provided by healthcare workers. Healthcare workers were equally affected by the pandemic, resulting in reduced number of staff for patient care. Despite these challenges, healthcare workers adopted different methods to ensure patients continued receiving care. These included providing care for patients within their communities or at home; providing additional support to patients such as counselling, transport, food packages, and health education; making different services available at one service point and adopting staff rotation to ensure a healthcare worker is always available to attend to patients.4.1Available evidence at the onset of the pandemic indicated that COVID-19 would adversely affect TB services delivery resulting in reduced access by different population groups with poor outcomes ,15,16. TFindings from this study indicate a complex interaction of patient-level, socioeconomic, and health system factors with limited emergency response capability. These, coupled with the urgent need to control a pandemic caused by a pathogen whose epidemiology was not fully understood vastly accounted for the reduced access to TB services; further contributing to the global reduction in TB achievement. A Nigerian survey describing TB and COVID-19 screening by healthcare workers during the lockdown indicates that 54% of healthcare workers were not screening patients for TB during this period . This isAnother critical factor was the repurposing of TB resources for COVID-19 including the equipment GeneXpert used for TB diagnosis ,25. ThisDue to the pandemic, more people globally have become poorer . TB alre4.2Our qualitative study presents healthcare workers' perspectives on how COVID-19 impacted TB service delivery at the peak of the pandemic and the different steps they took to ensure continued service delivery in Eswatini. This will provide insight into the reduced performance in key TB outcomes in Eswatini and other LMICs and guide TB program implementers on measures that can increase access to TB services. Our study participants were drawn from health facilities across Eswatini so our findings are representative of how COVID-19 impacted TB services. As a limitation, our study does not present data to quantify the impact of the pandemic on key TB indicators. This would have provided a more objective view of the problem. Secondly, we interviewed only doctors and nurses. We did not include other healthcare workers such as community TB officers responsible for active TB case finding, TB treatment supporters, and laboratory and pharmacy technicians. These healthcare workers could have presented a more complete perspective of how the pandemic affected TB service delivery and patients. Finally, we did not interview TB program staff. They would have offered an additional explanation for observations by our participants.5SMS and home-based services can be used for medication dispensing, testing, follow up and psychosocial support. Finally, bidirectional screening and integration of care for different comorbidities can increase access to care.The COVID-19 pandemic affected health systems globally, even in advanced economies. It is vital different approaches that sustained services are standardized so patients can continue receiving care. In the future, tailored and hybrid approaches to care should be developed where patients can access care either at the health facility or remotely with the freedom to visit a health facility if necessary. Due to limited infrastructure, approaches such as telemedicine for consultation may not be feasible immediately in most LMICs but telephone calls, Eswatini Health and Human Research Review Board (EHHRRB) (Ref: EHHRRB 036/2021).10.13039/100006090Global Health PhD Support Programme at the University Medical Center, Utrecht, The Netherlands.VW is funded by the VW conceptualized the study and wrote the protocol under the supervision of AV, DEG, KO, and KKG. DEG and KKG sourced funding for the research. VW, SH, MC, and LD obtained approvals and conducted the study. VW, AV, and KO conducted the analysis. VW and MC wrote the first draft. All authors reviewed the results, contributed to the interpretation, and reviewed the different drafts of the manuscript. All authors reviewed and approved the final draft of the manuscript.The data used for this study is available and will be shared by the corresponding author upon request.None declared by the authors.