A workshop on implementation strategies for the introduction of the RTS,S/AS01  malaria vaccine in countries with areas of highly seasonal transmission, was held as a hybrid meeting in Dakar, Senegal, and online, 23–25 January 2023. Delegates from Expanded Programmes on Immunization (EPI) and National Malaria Control Programmes (NMCPs) from 13 African countries, and representatives from key stakeholders participated. RTS,S is the first malaria vaccine to be recommended by the World Health Organization (WHO). The recommendation followed pilot implementation of the vaccine in Ghana, Kenya and Malawi, which showed that introduction of the vaccine was highly effective at scale, and was associated with a 30% reduction in hospital admissions with severe malaria in age groups eligible to have received the vaccine and no evidence of the safety signals that had been observed in the phase 3 trial. Clinical trials in Mali and Burkina Faso, showed that in children receiving Seasonal Malaria Chemoprevention (SMC), providing the vaccine just prior to high transmission seasons, matching the period of highest efficacy to the peak transmission season, resulted in substantial reduction in the incidence of clinical malaria and of severe malaria. While SMC has been successfully scaled-up despite the challenges of delivery, there is no established platform for seasonal vaccine delivery and no real-world experience. The objectives of this workshop were, therefore, to share experiences from countries that have introduced the RTS,S vaccine in routine child vaccination programmes, with SMC-implementing countries as they consider malaria vaccine introduction, and to explore implementation strategies in countries with seasonal transmission and where EPI coverage may be low especially in the second year of life. Practical implementation challenges, lessons learned for vaccine introduction, and research questions, towards facilitating the introduction of the RTS,S  in countries with seasonal malaria transmission were discussed.The online version contains supplementary material available at 10.1186/s12936-023-04657-5. Plasmodium falciparum malaria in children living in regions with moderate to high malaria transmission. The recommendation of this first malaria vaccine was based on findings from the pilot introduction and evaluation of the vaccine in Ghana, Kenya and Malawi (launched in 2019 and ongoing through 2023), and other RTS,S research evidence showing that the vaccine can be delivered effectively, has a strong safety profile and can have a significant impact in real-life childhood vaccination settings. High uptake of the vaccine was achieved in the pilot countries, showing strong community demand and acceptance of the vaccine by health workers and communities.In October 2021, the World Health Organization (WHO) recommended the use of RTS,S/AS01  malaria vaccine for the prevention of The WHO recommends the RTS,S malaria vaccine be provided in a schedule of 4 doses to children from 5 months of age. For countries with areas of highly seasonal transmission of malaria, the WHO recommendation recognized the impact of aligning the administration of the vaccine just prior to the malaria season, and includes an optional alternative 5-dose seasonal delivery strategy to optimize vaccine efficacy, as well as the additional impact of coordinating the provision of the vaccine with malaria chemoprevention (SMC). Because there is no real-world experience in delivering the malaria vaccine using the seasonal strategy, countries that choose to adopt such a strategy are “strongly encouraged to document their experience, including the vaccine effectiveness, feasibility and occurrence of any adverse events, to feed into future guidance updates” and in addition WHO encouraged “international and national funders to support relevant learning agendas” .This workshop was convened by the OPT-SMC project in collaboration with The Access & Delivery Partnership (ADP) partners, the WHO Malaria Vaccine Implementation Programme (MVIP) of the Department of Immunization and, the Vaccines and Biologicals and the WHO regional office for Africa. The OPT-SMC project supports 14 countries in West and Central Africa to conduct implementation research for optimizing the effectiveness of SMC, working in partnership with the University of Thiès in Senegal, the Special Programme for Research and Training in Tropical Diseases (TDR), Medicines for Malaria Venture (MMV), and the London School of Hygiene and Tropical Medicine (LSHTM), with funding from the European and Developing Countries Clinical Trials Partnership (EDCTP). The ADP works with low- and middle-income countries to ensure life-saving medicines and health technologies reach the people who need them. This partnership is led by the United Nations Development programme (UNDP), in collaboration with PATH, the WHO Regulatory Department and TDR, supported by the Government of Japan.The aim of the workshop was to bring together representatives from Expanded Programme on Immunization (EPI) and National Malaria Control/Elimination Programmes (NMCP/NMEP) in countries with highly seasonal malaria, and relevant stakeholders, to share experiences and lessons learned during pilot implementations, to consider vaccine scheduling and delivery in the context of seasonal malaria transmission and varying levels of EPI coverage. The workshop aimed to support planning of introduction of the vaccine to optimize its impact in reducing child morbidity and mortality caused by malaria.To review the current evidence on the RTS,S malaria vaccine in terms of evidence on vaccine efficacy and impact, and safetyTo share experiences across countries that have introduced the RTS,S vaccine in routine child vaccination services in Ghana, Kenya, Malawi, and SMC-implementing countries that are considering malaria vaccine introduction, to better understand the practical implementation challenges and lessons learned for vaccine introductionTo discuss regulatory and supply management issues when introducing the RTS,S vaccine in the health systemsTo discuss implementation strategies and mode of delivery in countries with seasonal transmission and low or moderate EPI coverage during the 1st and 2nd year of lifeTo discuss the operational/implementation research needs to document the implementation of RTS,S in terms of uptake and effectiveness, safety and acceptability.The participants were representatives from the National Malaria Control/Elimination Programmes (NMCP/NMEP) and Expanded Programmes of Immunization (EPI) from 13 countries in West and Central Africa that are currently implementing SMC .The workshop was opened by the chancellor of University of Thiès , Prof Ramatoulaye Diagne Mbengue, followed by welcome remarks from the meeting co-organisers Prof Jean-Louis NDiaye  and Dr Corinne Merle (OPT-SMC project (presented by Ndiaye) and the ADP initiative  also highlighted the benefits of promoting inter-country collaboration, and sharing of information and expertise, particularly with respect to implementation research (IR) activities, within their respective networks. These benefits also aligned with the premise of this workshop to facilitate the successful introduction of the RTS,S vaccine in countries with different malaria prevalence and EPI coverage and given the limited availability of vaccine.Brief overviews of the RTS, S malaria vaccine current evidence, including efficacy, safety, feasibility and impact and update on the WHO review of the R21/MatrixM vaccine .P. falciparum malaria in children living in regions with moderate to high transmission as defined by the WHO [The global trend in malaria morbidity and mortality remains high, with 241 million cases and 627,000 deaths reported in 2020, with over 80% of the burden of malaria occurring in children in Africa. In March 2022, the WHO released a position paper on the use of the RTS,S/A01 malaria vaccine for the prevention of Pilot implementations to understand the vaccine in routine use will continue through 2023 in Malawi, Ghana and Kenya. These have shown that the vaccine introduction is feasible, safe, impactful and equitable. Importantly, adding a malaria vaccine to current interventions, e.g. insecticide-treated bed nets (ITN), increases access to malaria preventive tools. As of December 2022, 3.8 million vaccine doses have been administered with 1.2 million children receiving at least one dose. In all three countries vaccine coverage is good for the first three doses (> 73%), but drops for the fourth dose (54–72% of children who received the 3rd vaccine dose received the 4th dose).Modelling predictions indicate a significant public health impact across a wide range of malaria transmission settings, and a high level of cost effectiveness at a cost 10 USD per dose with an estimated 400 deaths averted per 100,000 fully vaccinated children.Demand for the RTS,S vaccine is very high, outstripping currently available supply. A number of other malaria vaccine candidates are in clinical development, including R21/Matrix-M. This vaccine is currently in phase III trials, and initial (unpublished) and phase II trial data  show higVaccine efficacy and seasonality of malaria .The WHO estimates that 12 countries in West and Central Africa with areas of highly seasonal transmission account for 50% of the worldwide total of malaria deaths each year , but SMCUpdate on supply of RTS,S with allocation framework .This presentation covered the current malaria vaccine supply situation and the Framework for allocation of limited vaccine doses. Initial vaccine supply is expected to be insufficient to meet the high demand from malaria-endemic countries. Given that over 28 countries have already expressed interest in introducing the vaccine, the current discrepancy between demand and supply is vast (80 to 100 million doses needed per year vs 18 million doses available between 2023 and 2025).The WHO was tasked to develop a framework to guide in a transparent, principles-and evidence-based manner how initial limited malaria vaccine doses should be allocated . The FraWhilst the framework clarifies the allocation principles and criteria upfront, it does not remove all uncertainties, e.g. the timing of a country’s application to Gavi, the number of approved countries and their needs, and the dynamic nature of the supply situation all affect when countries may be able to access vaccine doses. The key implications of the current situation are that not all interested countries will have access to the vaccine initially, and that those who receive a supply allocation, will have to roll-out the vaccine in a phased approach, starting in areas with greatest need.To help manage expectations and support decision-making and planning, transparency and communication will be maintained regarding supply availability over time and allocation decisions.Gavi malaria vaccine programme update .Gavi is in its 5th phase of accelerating access to 17 vaccines globally. In late 2021 Gavi approved funding for the RTS,S malaria vaccine through a recommendation for a new Malaria Vaccine programme. A coordinating body for this programme is co-chaired by the WHO and Gavi, and includes other organizations joining a stakeholder group , to ensure coordination across immunization and malaria partners.Gavi support for the RTS,S malaria vaccine comprises i) facilitation of the phased, sub-national introduction of the vaccine into national vaccination schedules in areas with moderate to high P. falciparum malaria transmission as defined by the WHO; ii) procurement of vaccine doses and associated supplies; and iii) the provision of technical support to countries towards the development of applications and the implementation of the malaria vaccine programme. Notably, Gavi currently does not provide support for malaria vaccine campaigns or catch-up vaccination.There are four Gavi application windows per year with guidelines for support, application process and vaccine funding available in French and English, via the Gavi website . The firThe initial cost of the vaccine is approximately $38/child/4 doses. To facilitate affordability and country uptake of the vaccine, Gavi approved an exceptional, time-limited co-financing approach with initial self-financing ($0.20/dose), followed by a preparatory transition phase and an accelerated transition ramping up co-financing from 20 to 100% of the vaccine price over eight years.A Gavi-led market shaping roadmap has also been developed and aims to increase supply, reduce the price, materialize sustainable demand and incentivize product innovation for next generation vaccines. Meanwhile all available vaccine supply for 2023–25 and its procurement was contracted via UNICEF.Regulatory consideration for introduction of RTS, S in countries with seasonal malaria .The WHO regional office for Africa has led efforts towards regulatory considerations through a joint review of RTS,S. A team of experts from marketing authorization, pharmacovigilance, clinical trial authorization with representation from 17 countries was convened under the African Vaccine Regulatory Forum (AVAREF) with support from the European Medicines Agency (EMA) at WHO. This resulted in non-binding recommendations endorsement in mid-2022. These put in place mechanisms for regulatory pathways for registration of vaccines, and timelines after vaccine prequalification though an agreed action plan. Importantly, national regulatory agencies (NRAs) can use the non-binding recommendations to inform their decision without undertaking a full review themselves. This is facilitated by applicants submitting dossiers whilst NRAs are given access to the WHO prequalification assessment report, EMA opinion report and other relevant data.To date the three countries that piloted introduction of RTS,S – Ghana, Kenya and Malawi – all have their dossier under finalization locally with targeted submission to the respective NRA in early 2023.Overview of SMC  .Seasonal malaria chemoprevention (SMC) is recommended in areas with high malaria transmission, i.e. where at least 60% of yearly malaria cases are concentrated within a 4 month period each year, usually coinciding with the rainy season. SMC is the intermittent administration of a curative dose of anti-malarial medicine during the malaria season, regardless of the infection status of a child. Its aim is to establish anti-malarial drug concentrations in the blood to clear existing and prevent new infections at the height of the malaria season.Recommendations regarding the geography and transmission intensity, number of doses or cycles, specific drugs recommended and taking into account age-based risk among children were recently updated and published . These aSMC administration is not trivial, given the need for 3–5 months of delivery through door-to-door campaigns using community health care workers and/or volunteers. Directly observed therapy (DOT) and/or coupling of SMC with other public health interventions can significantly increase effectiveness, however sustainability of such approaches may be limited due to associated costs. To date around 45 million children have been covered by SMC across Africa, with steady increase in delivery despite Covid-19, and an exponentially higher coverage in Nigeria.Lessons learned and practical experience from pilot introduction of the RTS, S malaria vaccine: implementation in routine child immunization programmes and possible implications for implementation through SMC or other mass drug administration programmes .Malawi, Kenya and Ghana were the first countries to introduce the RTS,S malaria vaccine through routine immunization programmes in 2019 as part of the WHO-coordinated Malaria Vaccine Implementation Programme. Representatives from the EPI and/or NMCPs gave an overview in each of their settings, focusing on lessons learned to date. In each country doses 1–3 were administered at slightly different time points in the first year of life, with the fourth dose at around two years of age  to identify challenges and provide timely solutions was essential, particularly with respect to health care workers addressing caregivers’ questions and relaying relevant health messages consistently.Stakeholder engagement at district level facilitated reciprocal learningPost-introduction evaluation is essential to assess the benefits and caveats of introduction of a new vaccine into routine immunizationChallenges observed included a low fourth dose coverage, health care worker challenges with understanding eligibility criteria for vaccine administration, and insufficient investment for effective social mobilization.Active community engagement was seen as central to addressing these challenges, as was the coordination between EPI and NMCP teams, national and subnational stakeholder engagement, integration of the malaria vaccine into the broader health system and sustained advocacy for second year of life platform vaccines.Opportunities for EPI/NMCP programmatic coordination was seen to be particularly beneficial with respect to maximising data driven decision making; advocacy for continued investment on malaria; communication and community engagement; and integrated service delivery.Good collaboration between the National Malaria Elimination Programme (NMEP), EPI and national regulatory agency (NRA) is essential for the operationalization of plansThe vaccine introduction plan should address health systems strengthening and integration with other child health interventions across the life courseHealthcare workers must be adequately educated to ensure effective communication with caregivers, timing training adequatelyExisting defaulter tracing systems must be reviewed and strengthened to reduce drop-out ratesCatch-up campaigns may be necessary to optimize the uptake of the 4th dose, including leveraging the second year of life strategiesAppropriate strategies should be developed to reach eligible children in difficult to reach populations Where available, the use of electronic vaccination registries can support improving data quality and defaulter trackingStrengthen vaccine quantification and forecasting at all levels to prevent avoidable vaccine stock-outsThe systems for surveillance of adverse event of special interest (AESI) were poorly understood by health care providers and led to low reporting, management will require careful consideration going forward.All three countries reported the beneficial value of the integration of EPI and NMCP activities. However, there were challenges of achieving high coverage for the fourth dose of the RTS,S vaccine within the routine vaccination programmes. This finding is consistent with lower coverage of other EPI vaccines given in the second year of life. The key going forward will be identifying how to tackle the reduction in coverage for older children.Qualitative findings and lessons learned from the pilot countries: perceptions of malaria, the vaccine and other interventions .Facilitators and barriers to the uptake of the RTS,S vaccine were explored through qualitative studies in the three pilot countries. The findings across all three countries and settings indicated a trajectory of growing trust in the vaccine over time, driven by foundational trust in health systems and vaccines, and increasing with RTS,S-specific trust with growing understanding of the benefits and safety aspects of the vaccine. Initial hesitations about RTS,S were overcome with clear communication though trusted channels within the health system, coupled with additional facilitators such as encouragement from within the wider community (e.g. family members). Importantly, by the time the third dose of the vaccine was administered, caregivers understood that RTS,S was partially protective and that therefore the continued practice of other prevention measures and professional care seeking in the event of fever, is essential. Data further showed consistent bed net use over time as RTS,S was introduced. Similarly, prompt treatment seeking for fever or suspected malaria continued for children receiving the vaccine.Barriers to uptake and adherence were identified and differences in these barriers were reviewed by dose uptake category. Uptake of dose 1 seems primarily driven by trust in the health system, with the very limited incidents of non-uptake due to refusals, stemming from low vaccine confidence, compounded by country-specific issues. Routine RTS,S promotion or information, education and communication strategies appeared to be insufficient to reach or persuade those limited populations most-at-risk of missing dose 1. The reasons for non-uptake of dose 2 or 3 (more common among respondents) are similar to those for non-uptake of dose 1. However, partial protection through the vaccine was not an impediment for uptake or use of other interventions.Completion of the full four dose vaccination course was seen to be positively influenced by reminders of vaccine visits, encouragement by health care workers and increasing trust in RTS,S specifically. Routine use of the child health book to remember vaccination dates and vaccination visit reminders also contributed to uptake. Conversely, service interruptions and an abundance (rather than lack) of confidence in the vaccine associated with complacency were shown to contribute to dose 4 defaulting. Personal life barriers also appeared to play a role in the lack of uptake of dose 4. Overall, the findings are helpful to inform future strategies for the introduction of the vaccine in different settings.Practical considerations for RTS, S malaria vaccine supply chain and limited supply management at country level .To stimulate the discussion of implementation of the malaria vaccine particularly in countries with seasonal transmission planned for day two of the workshop (and beyond), four key areas of considerations were flagged – each of these is associated with a significant amount of complexity. 1) Targeted distribution in the context of the need for some countries to prioritize areas of highest need given the limited vaccine supply. 2) Cold chain capacity and a need for analysis thereof across all levels of service delivery in individual countries to evaluate equipment and maintenance needs. 3) Wastage minimization based on accurate data on vaccine wastage to facilitate correct planning for vaccine procurement. And lastly, 4) flexible data driven supply plans to inform reverse logistics and re-distribution, staggered delivery schedules and cold chain equipment purchase and planning.Short presentations were given by country representatives from Benin, Burkina Faso, Cameroon, Tchad and Niger, outlining at what stage each of these countries are at in terms of RTS,S vaccine introduction planning given WHO recommendations and Gavi requirements for accessing support.A rich discussion ensued, with participation from both the audience in the room and online. Questions and answers fed into the group discussions on days 2 and the general discussion on day 3, and are therefore captured under the relevant sections in this report.Considerations and possible modalities for introduction of RTS,S malaria vaccine in countries with seasonal malaria .The WHO recommendations for use of the RTS,S malaria vaccine allows for flexibility, for countries to adapt the schedule to their needs. Notable key findings from previous and ongoing studies were laid out: i) Protective efficacy is seen after completion of the primary series (3 doses), the 4th dose significantly prolongs the duration of protection; ii) schedule considerations represent a trade-off between optimizing vaccination coverage and providing the best protection during the period of greatest risk; and iii) reaching children aged two and beyond is difficult with drop-out rates increasing over time. Furthermore, epidemiological studies show that in areas of high perennial malaria transmission, a considerable risk (20–40%) of severe malaria and death remains beyond age 3. In highly seasonal transmission settings, this risk may still be substantial at least through age 5. Hence, countries may align the choice of schedules with the local disease epidemiology and risk-profile ensuring that majority of children are protected during the period when they are at the highest risk of severe disease and mortality.These findings are particularly pertinent to settings with highly seasonal malaria transmission in relation to scheduling and delivery of the RTS,S malaria vaccine. Providing the primary series and additional annual doses prior to the high transmission season offers high protective efficacy.What is the optimal interval between dose 3 and 4?What should the target age group for the campaign style delivery be?What is the safety and efficacy of a > 5-dose strategy?What is the best delivery strategy depending on the malaria transmission intensity and length of the transmission season?Can improved delivery be achieved through integration with other interventions?Three main delivery models could therefore be considered, 1) age-based through routine EPI delivery platforms, 2) seasonally timed doses through vaccination campaign style delivery, or 3) a hybrid approach. Each of these allows for some flexibility and each is associated with pros and cons that need to be considered context specific , whilst delivering or combining dose 4 through a campaign approach, e.g. with SMC directly or using SMC as means to facilitate vaccine referrals.Consideration of different implementation strategies, particularly a hybrid model, was approached by dividing country participants into three groups. The following pros and cons for combining or integrating RTS,S roll-out according to malaria seasonality and together with SMC schedules were collated:Possible research questions in the context of malaria vaccine administration in conjunction with SMC were raised through the group work, these are listed in additional file New vaccine Post-introduction Evaluation (PIE) as tool .The scope of using the PIE  as tool As an exemplar, the experiences with PIE in the context of RTS,S piloting in Ghana were provided by Naziru Mohammed Tanko. The PIE was conducted using an adapted PIE tool with a mixed method approach to assess pre-implementation planning and vaccine introduction; social mobilization, advocacy and community engagement; healthcare worker training, understanding and application of eligibility criteria; logistics and resources for key MVIP activities; data collection, management and use; and vaccine safety surveillance. This allowed to bring out a number of strengths and areas for improvement in the vaccine implementation approach. The resulting recommendations centered around resource mobilization, capacity building, supervision and demand generation see Fig. . The oveCase control studies to evaluate vaccine strategy efficacy .Colleagues from Ghana presented on behalf of the Malaria Vaccine Pilot Evaluation (MVPE-CC) their approach to apply an embedded case–control study as evaluation tool for Malaria Elimination Programmes introducing the RTS,S vaccine . This approach was employed specifically to address the following questions across Ghana, Kenya and Malawi: (1) Are children who receive RTS,S vaccination at increased risk of meningitis compared to unvaccinated children? (2) Are children who receive RTS,S vaccine at increased risk of cerebral malaria compared to unvaccinated children? (3) Is the incidence of severe malaria increased in children who received 3 doses, but failed to receive a 4th dose, compared to children who did not receive the vaccine (the rebound effect)? (4) What is the effectiveness of RTS,S  in preventing severe malaria? and (5) Is there any evidence that RTS,S vaccine increases mortality in girls, or is less effective in preventing death in girls, compared to boys? The pilot evaluations have shown that the vaccine is safe in routine use, but was unable to show the added benefit of a 4th dose (question 4) and whether a 3-dose schedule is a cost-effective option in some situations. The added individual-level evidence on safety will provide added reassurance on the vaccine safety.Details of study design including governance structure, case definitions, data collection and analyses approaches, as well as ethical considerations were laid out, with provision of limitations and challenges. To date the study recruited over 1500 cases and 6000 controls. Whilst the work is ongoing, the team are preparing a manual for the case control approach, a budget template to help estimate the cost of such an evaluation, webinars to disseminate the findings when available as well as a publication.Implementation questions and funding for evaluation and implementation research questions .The pilot introduction of RTS,S in Malawi, Kenya and Ghana have provided many rich lessons on safety in routine use, acceptability, feasibility, impact, and the use of remote tools for effective delivery during a pandemic. Nonetheless, outstanding questions remain on seasonal vaccine delivery; programmatic optimization of delivery in different settings; increasing supply and reducing cost; optimizing impact, including combined use of vaccine and other malaria prevention measures; as well as behaviour change and communication, including optimal use of additional visits to catch up on other vaccines and child health measures.Notably, the pilots were conducted in areas with functional EPI and NMCP programmes, in areas with perennial transmission, and the vaccines were provided through routine EPI (age-based administration throughout the year). As discussed over the course of the workshop, the initial subnational introduction of the vaccine in highest need (category 1) areas may entail delivery through weaker health systems and areas with high malaria seasonality may select a seasonal delivery, thereby warranting a broader spectrum of implementation research questions. This could further be expanded to include research on vaccine escape mutant, immunogenicity, co-administration studies.The WHO and Gavi are engaging PMI Insights jointly with a research or public health institution in a malaria-affected country, to develop a comprehensive and vetted research agenda. A broad and inclusive stakeholder consultation process will be undertaken to gather input from ministries of health, research institutions and partners engaged in malaria vaccine introduction or research, CSO, regional and global bodies, and global health funding agencies. The aim is to conduct a rigorous evaluation process to prioritize and rank research topics and develop research agenda.The final discussion session focused on areas that needed additional clarification (for instance regarding Gavi application process) and further exploration of research questions. These discussion strands have been incorporated into the overall report including the additional files.The overall conclusions from the workshop are summarised below, highlighting more specific areas relating to supply and demand as well as lessons learned to date. Further, open questions and suggested topics for future research were collated , it is unlikely that countries will introduce RTS,S before beginning of 2024.Awareness of differences in RTS,S vaccine efficacy depending on epidemiological context and timing of vaccination in relation to the malaria transmission season can support informed decision-making by EPI and NMCP regarding the optimal implementation strategy. Training of personnel administering the vaccine should be carefully timed prior to the introduction of new vaccines, though not too far ahead.Community engagement from the outset is key to increase acceptability of the vaccine where it is introduced.Dialogue and integration of activities by key national stakeholders, e.g. the EPI and NMCP, is mutually beneficial, as evidenced in the RTS,S pilot countries. For countries with weaker EPI performance and/or where malaria transmission is highly seasonal, synergies between the two programmes through joint-up of implementation strategies has the potential to improve uptake of all available interventions.Achieving high coverage of dose 4 has been challenging, even under favourable conditions in pilot countries where EPI coverage overall is good. This suggests the need for additional efforts, or alternative approaches to cover the harder-to-reach and strengthened approaches to increase vaccine coverage during the second year of life.Optimal impact of RTS,S is achieved with four doses and – where applicable – by timing of vaccine administration prior to higher malaria transmission periods in highly seasonal areas .Findings from the pilot countries are extremely valuable towards informing the introduction of RTS,S in other settings.The workshop helped to collate an initial set of questions and more general areas for future investigation , and delivery of dose 4 through a campaign (or similar) approach before or at the onset of the rainy season. Fourth dose delivery could be coordinated with preparations for SMC delivery (or other mass drug administration programmes), e.g. advanced communications for SMC or using SMC as means to facilitate vaccine referrals. However, there is a great need for exploration of the feasibility, acceptability and cost effectiveness of this model through implementation research.Modelling approaches could be explored to provide guidance regarding this hybrid model in meantime particularly with respect to uptake of the vaccine in the 2nd year of life (4th dose) and optimal spacing between the 3rd and 4th dose of RTS,S.As recommended by the WHO, countries should document lessons learned from seasonal vaccination strategies, especially regarding operational feasibility, vaccine efficacy and safety. The ADP is considering provision of support for countries to develop such documentation.Continued information exchange between countries planning to introduce RTS,S should be facilitated, e.g. through future workshops. Further, suitable communication materials should be developed  for the community of implementers from vaccine and NMCP programmes.For the development of a comprehensive and vetted research agenda (facilitated by the WHO and Gavi) consultation with researchers and other stakeholders on the ground is essential, to capture relevant implementation priorities generated in the global south. This is planned for later in 2023.Additional file 1. Exemplar research questions and areas for investigation in the context of implementing a malaria vaccine and in settings with seasonal malaria transmission.Additional file 2. TDR workshop website including workshop presentations. A dedicated TDR website was set up in conjunction with this workshop, this includes a short report in English and French, as well as links to the workshop presentations and other relevant information. Additional file 3. Workshop agenda: Implementation strategies for the introduction of the RTS,S/AS01  malaria vaccine in countries with areas of highly seasonal transmission.


Setting mental health priorities helps researchers, policy makers, and service funders improve mental health services. In the context of a national mental health implementation programme in England, this study aims to identify implementable evidence-based interventions in key priority areas to improve mental health service delivery.A mixed-methods research design was used for a three step prioritisation approach involving systematic scoping reviews , expert consultations and data triangulation. Groups with diverse expertise, including experts by experience, worked together to improve decision-making quality by promoting more inclusive and comprehensive discussions. A multi-criteria decision analysis (MCDA) model was used to combine participants' varied opinions, data and judgments about the data's relevance to the issues at hand during a decision conferencing workshop where the priorities were finalised.The study identified mental health interventions in three mental health priority areas: mental health inequalities, child and adolescent mental health, comorbidities with a focus on integration of mental and physical health services and mental health and substance misuse problems. Key interventions in all the priority areas are outlined. The programme is putting some of these evidence-based interventions into action nationwide in each of these three priority mental health priority areas.We report an inclusive attempt to ensure that the list of mental health service priorities agrees with perceived needs on the ground and focuses on evidence-based interventions. Other fields of healthcare may also benefit from this methodological approach if they need to make rapid health-prioritisation decisions. There is a pressing need to address the burden of mental health and substance use disorders in England as mental health disorders, including anxiety, depression, and alcohol and substance use disorders, account for at least 21.3% of the burden of Years Lost to Disability in England . IneffecDespite recommendations to reform mental health systems , 7, progTo reliably prioritise the unmet needs that evidence-based interventions can address, a rapid, systematic, and transparent process is required for our health systems . TraditiThe aim of this paper is to identify implementable evidence-based interventions in key mental health priority areas in England. The MHIN national prioritisation exercise was grounded on individual and broader system needs and evidence, and carried out with a focus on implementation. It is critical to identify mental health priority areas with the greatest unmet needs, which are supported by evidence-based interventions and are fit for delivery, especially in the current political climate in England, where improving mental health care access and “levelling up” are significant discourses.Our study aimed to address the complex prioritisation process of improving mental health services in England by utilising a mixed-methods research approach , 18.The quantitative methods involved data collection through structured questionnaires and participant ratings. The data obtained through these methods were then analysed using descriptive statistics to summarise and interpret the numerical information. Additionally, we utilised MCDA to assess and weigh multiple criteria or factors involved in the decision-making process. On the other hand, the qualitative data collection focused on participant observations during decision conferencing sessions and narrative data gathered throughout the study. The qualitative analysis was guided by thematic analysis principles, which involved identifying recurring mental health priority areas, patterns, or concepts within the qualitative data.By combining both qualitative and quantitative approaches, we sought to achieve a more comprehensive understanding of the phenomenon under investigation. To guide the study, an eight-member multidisciplinary steering committee consisting of MHIN investigators , including experts by experience, was established.The steering committee determined the scope and process of prioritisation, which is illustrated in To achieve our aims, the steering committee developed a comprehensive and three-step priority-setting process . The process was used to comprehend diverse viewpoints and help build consensus among a group of stakeholders –22 and gIn the first phase of expert consultation process, an electronic scoping exercise was conducted to identify areas of perceived unmet needs in mental health service provision, with accompanying potential solutions. The steering committee used a structured survey to gather information from a wide range of relevant stakeholders across England, including mental health trusts, voluntary, social, and educational services, all 15 NIHR Applied Research Collaborations (ARCs), NIHR Translational Research Collaboration (TRCs), Academic Health Science Networks (AHSNs), NIHR Clinical Research Networks, and mental health-related charities and other third sector organisations. Perceived unmet mental health needs and service priorities were also identified via systematic scoping reviews (manuscript under process) which added to the list of priorities identified through this electronic scoping exercise. To reduce the number of priorities identified, the steering committee focused only on the largest mental health priority areas that aligned with national priorities and were endorsed by the National Institute for Health and Care Excellence (NICE), indicating that they were supported by evidence and could be considered for wider implementation.In the next phase of expert consultation process, we organised second consultation exercise with similar stakeholder groups to further identify mental health priority areas. This step was introduced to sift the mental health priority areas to a manageable number. The stakeholders ranked mental health mental health priority areas according to five key objectives, including clinical effectiveness, involvement of patients and community in developing and delivering interventions, addressing health inequalities in terms of access to mental health services, implementation outcomes , and susThe third and final step of expert consultations, we engaged with 11 experts, plus the MHIN team, for a facilitated virtual workshop, known as a decision conference , to deveThe social process of decision conferencing plus the technical modelling of MCDA enable a group of experts to move from implicit, qualitative perspectives about a problem to explicit, quantitative views in which preferences are constructed, debated, and agreed as new intuitions arise. They can lead to different, better differentiated, and more valid results than can be achieved by methods such as Delphi and Nominal Group Technique, as described in a comparison of the two approaches to research on the harm of drugs .The most preferred topic area for a given criterion was assigned a value of 100, the least preferred topic area a value of zero, and the other seven mental health priority areas were assigned reference scores between 0 and 100, inclusive. The group also assessed criteria weights representing the differences in clinical value between least and most preferred mental health priority areas across the five objectives. The scoring for each objective proceeded in three phases: (1) identifying the most and least preferred mental health priority areas; (2) discussing a score for the next most preferred topic area; and (3) revealing and discussing the scores they were thinking of. This “think, reveal, discuss” process was intended to prevent participants from anchoring on the number suggested by the first person in an open discussion, and has been proven to minimise bias in group assessments . The fivBelow we present the findings from the three phases of expert consultations.In April 2021, 78 stakeholders completed the survey, out of a total of 190 stakeholders contacted. The stakeholder groups included ARCs, Biomedical Research Centres (BRCs), mental health trusts, TRCs, Local Clinical Research Networks (LCRNs), AHSNs, NICE, NHS E/I, and charities and other third sector organisations. The majority of respondents (47.5%) worked for mental health trusts, while the rest were employed by various organisations. A significant proportion (12.8%) of the stakeholders worked across these organisations or did not specify their organisation see .The respondents identified 92 perceived mental health needs in mental health priority areas. Among these needs, 60% of respondents recognised one unmet mental-health need, 25% identified two, 5% three, and approximately 2% identified four needs. These 92 perceived mental health needs, along with additional mental health needs from systematic desk reviews, were categorised into 17 different mental health priority areas or priority topics. These mental health priority areas included mental-health system strengthening, severe mental illness, comorbidities, trauma and crisis care, autism spectrum disorders and intellectual disabilities, learning disabilities, continuity of care (including early interventions), talking therapies and peer support therapies, self-harm and suicide, common mental disorders, technology-driven mental-health-care, eating disorders, inequalities, maternal mental-health, personality disorders, mental-health of older adults, and addictions.Notably, all identified mental health needs where the proposed solutions did not align with NICE guidelines  and/or rThe results from phase 2, the second consultation exercise conducted in May 2021 are presented in this section. A total of 196 stakeholders were contacted, of which 61 completed the survey. The response rate varied across different regions, with the East of England region recording the highest response rate. Mental health trusts (39.29%) were the most significant contributors among the different types of respondents, followed by universities (16.07%), ARCs, (10.71%), AHSNs (10.71%), charities (7.14%), TRCs (3.57%), NICE (1.79%), MHIN (1.79%), and ARC Well-being collaboration network (1.79%). The remaining organisations did not respond, and about 11% of the respondents identified themselves as “Other”. These results are presented in detail in Mean priority scores for each of the five objectives, as well as an overall priority score, were calculated for each topic . Nine top-ranked topics were identified for the decision conferencing based on these scores, with mean scores ranging from 4.14 to 3.55 , followed by implementation (23%), reduction in health inequalities (20%), and engaging patients and the public (16%). The lowest overall weighting was given to the objective of capturing alignment with national priorities. A compendium of implementable interventions in each of the nine mental health priority areas was provided to the stakeholders for the decision conferencing to assist them in ranking the topics. These potential interventions are presented in detail in It is worth noting that the MCDA process was a complex one. For example, one participant in the decision conference suggested, at the start of scoring, that all nine mental health priority areas were valuable, so the least and most preferred couldn't be established. However, the group soon agreed that the relative values differed from one objective to the next and they also realised that if their expertise was lacking for a particular topic area, they weren't required to contribute to the scoring. After the first hour of assessing preference values for the clinically effective objective, the group found that they could indeed assess quantitative preference values. MCDA modelling helped the group to collectively distinguish the greater value mental health priority areas from the lower value ones. The preference values assessed by the stakeholder group, where 100 identifies the most preferred topic area(s) and 0 the least preferred, one of each for every row, are detailed in The MHIN network selected the three highest priority mental health priority areas and four sub themes for the next phase. The three priority areas are: mental health inequalities, child and adolescent mental health, comorbidities with a focus on integration of mental and physical health services and mental health and substance misuse problems. The sub theme selected in the mental health inequalities priority area is community engagement systems for people from ethnic minority groups to access mental health-care. The sub theme within the comorbidities priority area are two fold (a) physical health checks for people with severe mental illness and consequent intervention; integrated care protocols for patients with co-occurring serious mental illness and substance misuse. Finally, the sub theme within children and adolescents mental health priority area is IAPT-style services for children and adolescents. The MHIN steering committee decided to keep psycho-social support for caregivers of people with severe mental illness and mental-health-care access for people with long-term physical conditions as reserve mental health priority areas.Inequity in health and social care, along with a higher level of mental health problems as a result of the COVID-19 pandemic, has resulted in an even greater of the mental health treatment gap , 33, whiSimilar to other prioritisation exercises, three mental health priority areas such as mental health inequalities , 36, chiWhile each of these mental health priority areas can be a collection of multiple interventions, many of them are secondary and tertiary level public mental health interventions since they incorporate evidence-based treatment for mental disorders. Although outside the scope of the MHIN, primary public mental health interventions and mental well-being promotion interventions across the life course should be equally explored .In several contexts, failure to scale up evidence-based mental health interventions is due to a lack of awareness of unmet needs, poor anticipation of implementation challenges, costs, and impact of interventions . TheoretTherefore pivoting from traditional prioritisation approaches, this study made an inclusive attempt to incorporate some of these principles by (i) understanding perceived mental health needs on the ground by investigating the literature and incorporating stakeholders' views, (ii) focusing on evidence-based interventions which align with national priorities and provide information about implementation, and (iii) carefully selecting objectives for prioritisation that respond much better to implementation needs.To nest this theoretical perspective, we used a mix of methodological approaches, including repeated expert consultations, thematic analysis and data triangulation which other health services research initiatives can learn from and emulate. The strong consensus reached in the various steps regarding mental health needs and the respective evidence-based solutions provides a solid foundation for other research initiatives in health and social settings in England, which in turn (and over time with appropriate resource) may improve mental health systems performance, at the national level.Much of the prioritisation process convened various expert groups to elicit their judgments. However new research in the field of group decision-making  shows thOverall, this exercise provides new and relevant information for health service researchers, policymakers, and implementation scientists on how to rapidly prioritise mental health areas with evidence-based interventions or service delivery models that have the potential to improve mental health services in the NHS. This effort of transparency and developing transferable methods adds to the novelty of this study. Our rapid prioritisation process took about 6 months and each priority area identified are further developed into interventions (see appendix for more details on interventions) and implementation strategies with greater involvement of experts by experience in the coming phases of the programme. All the sites will create implementation and evaluation plans and methods that are appropriate for the topic and local context, based on a variety of change methods and implementation theories, taking into account the complexity of the change required, as well as the scale of the ambition of implementation in terms of spread and sustainability.This study presents several limitations that must be acknowledged. First, this study, like other prioritisation exercises , highligSecondly, respondents in the electronic scoping exercise and expert consensus survey were primarily asked to identify themselves with one organisation, which may have resulted in skewed response rates.Thirdly, certain mental health mental health priority areas with high unmet needs and little or no evidence of implementation were excluded from the programme's remit, potentially resulting in the exclusion of emerging priorities or important issues that are difficult to measure. Future research could investigate these priority areas to address unmet needs.Finally, the online nature of the prioritization process during the COVID-19 pandemic may have affected the response rate in expert consultations and potentially barred representation from digitally remote communities. For example, only 41% and 31%, respectively, of participants in the phase 2 and phase 3 of the expert consultations responded. It is important to consider all these limitations when interpreting the results of the study.This article reports on a prioritisation process designed to improve the evidence-based selection of mental health interventions. We have deployed a process for prioritising mental health interventions that embeds implementation thinking right from the beginning of the research cycle. The process includes assessing unmet needs, exploring implementation efforts of evidence-based solutions, engaging with key stakeholders, and providing specific recommendations to improve current prioritisation practices with a future goal of implementing these solutions within the life of the programme .Although some may argue that each step in the prioritisation process could stand alone as a prioritisation exercise, the combination of approaches used has reduced the risk of bias and achieved the determination of an acceptable and defensible set of priorities. The process has also allowed us to reach out to a range of local and national stakeholders, and as a result, the need for improved communication and dissemination methods for patient groups and communities was identified. These established relationships will be critical in the subsequent steps of implementing the priorities.We suggest that health-care organisations should recognise the relevance of research and appropriately allocate resources during the prioritisation of implementable health solutions. We also recommend that this approach can be used by others, not only in mental health, who need to make rapid and difficult health-prioritisation decisions with adequate resources.


The promise of digital health is principally dependent on the ability to electronically capture data that can be analyzed to improve decision-making. However, the ability to effectively harness data has proven elusive, largely because of the quality of the data captured. Despite the importance of data quality (DQ), an agreed-upon DQ taxonomy evades literature. When consolidated frameworks are developed, the dimensions are often fragmented, without consideration of the interrelationships among the dimensions or their resultant impact.The aim of this study was to develop a consolidated digital health DQ dimension and outcome (DQ-DO) framework to provide insights into 3 research questions: What are the dimensions of digital health DQ? How are the dimensions of digital health DQ related? and What are the impacts of digital health DQ?Following the PRISMA  guidelines, a developmental systematic literature review was conducted of peer-reviewed literature focusing on digital health DQ in predominately hospital settings. A total of 227 relevant articles were retrieved and inductively analyzed to identify digital health DQ dimensions and outcomes. The inductive analysis was performed through open coding, constant comparison, and card sorting with subject matter experts to identify digital health DQ dimensions and digital health DQ outcomes. Subsequently, a computer-assisted analysis was performed and verified by DQ experts to identify the interrelationships among the DQ dimensions and relationships between DQ dimensions and outcomes. The analysis resulted in the development of the DQ-DO framework.The digital health DQ-DO framework consists of 6 dimensions of DQ, namely accessibility, accuracy, completeness, consistency, contextual validity, and currency; interrelationships among the dimensions of digital health DQ, with consistency being the most influential dimension impacting all other digital health DQ dimensions; 5 digital health DQ outcomes, namely clinical, clinician, research-related, business process, and organizational outcomes; and relationships between the digital health DQ dimensions and DQ outcomes, with the consistency and accessibility dimensions impacting all DQ outcomes.The DQ-DO framework developed in this study demonstrates the complexity of digital health DQ and the necessity for reducing digital health DQ issues. The framework further provides health care executives with holistic insights into DQ issues and resultant outcomes, which can help them prioritize which DQ-related problems to tackle first. The health care landscape is changing globally owing to substantial investments in health information systems that seek to improve health care outcomes . DespiteThe promise of digital health is principally dependent on the ability to electronically capture data that can be analyzed to improve decision-making at the local, national , and gloTo assess DQ, scholars have developed numerous DQ taxonomies, which evaluate the extent to which the data contained within digital health systems adhere to multiple dimensions . Weiskopf and Weng  identifiFailure of health organizations to leverage high-quality data will compromise the sustainability of an already strained health care system . TherefoThis paper is structured as follows: first, we provide details of the systematic literature review method; second, in line with the research questions, we present our 3 key findings—(1) DQ dimensions, (2) DQ interrelationships, and (3) DQ outcomes; and third, we compare the findings of our study with those of previous studies and discuss the implications of this work.We followed the PRISMA  guidelines and the guidelines proposed by Webster and Watson  for systTo ensure the completeness of the review  and consThe papers returned from the search were narrowed down in a 4-step process . In the Inclusion criteriaSpecifically focuses on data quality in digital healthEmpirical papers or review articles where conceptual frameworks were either developed or assessedConsiders digital health within hospital settingsPublished in peer-reviewed outlets within any time framePublished in EnglishExclusion criteriaDevelopment of algorithms for advanced analytics techniques  without application within hospital settingsDescriptive papers without a conceptual framework or an empirical analysisFocused only on primary care Pre–go-live considerations Theses and non–peer-reviewed publications The relevant articles were imported to NVivo , where the analysis was iteratively performed. To ensure reliability and consistency in coding, a coding rule book  was deveIn the first step of the analysis, the research team performed open coding , where rIn the third step, through constant comparison , the 104After identifying the DQ dimensions and outcomes, the next stage of coding progressed to identifying the interrelationships (step 5) among the DQ dimensions and the relationships (step 6) between the DQ dimensions and DQ outcomes. To this end, the matrix coding query function using relevant Boolean operators (AND and NEAR) in NVivo was performed. The outcomes of the matrix queries were reviewed and verified by an expert researcher in the health domain.Throughout the analysis, steps for providing credibility to our findings were performed. First, before commencing the analysis, the research team members who extracted the verbatim codes initially independently reviewed 3 common articles and then convened to review any variations in coding. In addition, they reconvened multiple times a week to discuss their coding and update the codebook to ensure that a consistent approach was followed. Coder corroboration was performed throughout the analysis, with 2 experienced researchers independently verifying all verbatim codes until a consensus was reached . SubsequBMC Medical Informatics and Decision Making, eGEMS, International Journal of Medical Informatics, Applied Clinical Informatics, Journal of Medical Internet Research, Journal of the American Medical Information Association, PLOS One, BMC Emergency Medicine, Computer Methods and Programs in Biomedicine, International Journal of Population Data Science, JCO Clinical Cancer Informatics, Perspectives in Health Information Management, Studies in Health Technology and Informatics, Australian Health Review, MBC Health Services Research, BMJ Open, Decision Support Systems, Health Informatics Journal, International Journal of Information Management, JAMIA Open, JMIR Medical Informatics, Journal of Biomedical Informatics, Journal of Medical Systems, Malawi Medical Journal, Medical Care, Online Journal of Public Health Informatics, and Telemedicine and e-Health). A complete breakdown of the number of articles published in each outlet is provided in The vast majority of relevant articles were published in journal outlets , followed by conference proceedings  and book sections . The 169 journal articles were published in 107 journals, with 12% (n=13) of the journals publishing >1 study  systems overcoming physical and chronological boundaries associated with paper records by allowing access to information from multiple locations at any time  data within EHRs is crucial”  can jeopardise the integrity of a patient’s information” henreusability of EHR data for research purposes, hindering research-related outcomes and negatively impacting business processes and organizational outcomes. For example, Kim et al [clinical outcomes, reporting that the existence of inconsistencies in captured data “could explain the varying number of diagnoses throughout the encounter history of some subjects,” whereas Diaz-Garelli et al [clinicians in terms of increased workload.The consistency DQ dimension was related to all DQ outcomes. It was commonly reported that inconsistency in data negatively impacts the im et al  acknowleim et al  indicateli et al  demonstrclinical outcomes , business process outcomes , research-related outcomes , and organizational outcomes  [Incomplete EMR data were found to impact issions) . For exaissions)  found thissions)  reportedissions)  emphasizSummary of Key Findings section, these 3 findings led to the development of the DQ-DO framework. Subsequently, we compared the DQ-DO framework with related works. This leads to implications for future research. The Discussion section concludes with a reflection on the limitations of this study.The following sections describe the three main findings of this research: the (1) dimensions of DQ, (2) interrelationships among the dimensions of DQ, and (3) outcomes of DQ. As described in the In summary, we unearthed 3 core findings. First, we identified 6 dimensions of DQ within the digital health domain: consistency, accessibility, completeness, accuracy, contextual validity, and currency. These dimensions were synthesized from 30 subthemes described in the literature. We found that consistency, completeness, and accuracy are the predominant dimensions of DQ. Comparatively, limited attention has been paid to the dimensions of accessibility, currency, and contextual validity. Second, we identified the interrelationships among these 6 dimensions of digital health DQ . The litOur findings extend those of previous studies on digital health DQ in 3 ways. First, through our rigorous approach, we identified a comprehensive set of DQ dimensions, which both confirmed and extended the existing literature. For instance, Weiskopf and Weng  identifiSecond, unlike previous studies on DQ dimensions, we also demonstrate how these dimensions are all related to each other. By analyzing the interrelationships between these DQ dimensions, we can determine how a particular dimension influences another and in which direction this relationship is unfolding. This is an important implication for digital health practitioners, as although several studies have examined how to validate  and resoThird, although previous studies describe how DQ can impact a particular outcome , material world , and personal world  . These iAlthough prevention remains better than the cure (refer to implication 3), not all DQ errors can be prevented or mitigated. It is common for many health care organizations to dedicate resources to data cleaning to obtain high-quality data in a timely manner, and this will remain necessary . Some studies  advocatMethods section, with the objective of improving internal and external reliability [This review is scoped to studying digital health data generated within a hospital setting and not those generated within other health care settings. This is necessary because of the vast differences between acute health care settings and primary care. Future research should seek to investigate the digital health data of primary care settings to identify the DQ dimensions and outcomes relevant to these settings. In addition, this literature review has been scoped to peer-reviewed outlets, with “grey” literature excluded, which could have led to publication bias. Although this scoping may have resulted in the exclusion of some relevant articles, it was necessary to ensure the quality behind the development of the digital health DQ framework. An additional limitation that may be raised by our method is that because of the sheer number of articles returned by our search, we did not perform double coding . To mitigate this limitation, steps were taken to minimize bias by conducting coder corroboration sessions and group validation, as mentioned in the iability . To furtThe multidisciplinary systematic literature review conducted in this study resulted in the development of a consolidated digital health DQ framework comprising 6 DQ dimensions, the interrelationships among these dimensions, 6 DQ outcomes, and the relationships between these dimensions and outcomes. We identified four core implications to motivate future research: specifically, researchers should (1) pay equal consideration to all dimensions of DQ, as the dimensions can both directly and indirectly influence DQ outcomes; (2) seek to empirically assess the DQ-DO framework using a mixed methods case study design; (3) identify the root causes of the digital health DQ issues; and (4) develop interventions for mitigating DQ issues or preventing them from arising. The DQ-DO framework provides health care executives  with insights into DQ issues and which digital health-related outcomes they have an impact on, and this can help them prioritize tackling DQ-related problems.


Community and stakeholder involvement in decision-making to determine publicly-funded health services and interventions is advocated to fulfil citizens’ rights and improve health outcomes. The inclusion of public actors, particularly disadvantaged populations, in priority setting for universal health coverage (UHC) is also enshrined in guidance from the World Health Organization (WHO). However, challenges remain in operationalising this policy aim and ensuring that these approaches are effective and equitable. This study aimed to synthesise published evidence on the role of community and stakeholder participation in determining health service coverage.A systematic review was conducted, searching the Web of Science, Ovid Global Health, and PubMed Central databases from 2000 onwards, including all study types. A framework synthesis approach was used for charting and synthesising data on mechanisms, effectiveness , and barriers and facilitators for engagement.Twenty-seven relevant studies were identified that involve community actors and other stakeholders in priority setting and decision-making processes for defining health benefit packages and UHC, health technology assessment, and pharmaceutical coverage. Mechanisms of engagement include a wide variety of consultation approaches; participation in decision-making committees, advisory councils, and local planning meetings; and appeals mechanisms. Participation occurs primarily at Data and Dialogue stages of decision-making processes, and we found limited depth of engagement among identified cases. Limited consideration of equity was observed in planning and reporting on community involvement in priority setting. A number of challenges are identified in the literature, which we typologise as institutional, procedural, technical, and structural / normative barriers to meaningful participation.This systematic review identifies key gaps and opportunities in the literature and practice related to effective and equitable community and stakeholder participation in determining health service coverage. It offers essential considerations for planning and executing inclusive approaches to priority setting for publicly-funded health services and interventions and defining health benefit packages for UHC. Decision-makers around the world face the challenge of setting priorities and determining the publicly-funded health services and interventions available to a population. Increasingly, calls for greater involvement of community and other stakeholders in these processes are being made by academics, policy actors, and practitioners globally, including in the global policy agenda for social participation for universal health coverage (UHC) . Framed Understandings of community and stakeholder participation (CSP) are contested however, in part due to ambiguity and different normative understandings of “community” and “participation”. In their framework for community engagement, the World Health Organization (WHO) defines community participation as “a process of developing relationships that enable stakeholders to work together to address health-related issues and promote well-being to achieve positive health impact and outcomes” . DespiteIn our conceptualisation of health service coverage, we include priority setting and decision-making processes for determining the health services and interventions available to a population that are covered by public funding, also referred to as a “benefit package” or “essential service”. These decision-making processes vary across countries and are discussed across a range of literature on priority setting, including health technology assessment (HTA) and health benefit package design for UHC. Literature on the former (HTA) tends to be concentrated in high-income countries (HICs), however examples in low- and middle-income countries (LMICs) are emerging . DefininSetting priorities for UHC has been characterised as an inherently political and value-laden process, which requires not only technical inputs but also deliberation among stakeholders with diverging values and interests ,13,14. IAbelson et al. 2016) identify four broad rationales for CSP, categorised as serving democratic, methodological, instrumental, and developmental goals  identify,8,17-20,Given the many rationales for CSP, and the need for further guidance, this review aimed to synthesise existing evidence in the literature on CSP. While the literature examining public participation in priority setting and HTA is relatively dense, the aim of this review is distinctive in synthesising evidence about CSP in the specific context of priority setting and decision-making for health service coverage.Systematic reviews provide an essential function in synthesising evidence to support decision-making in health policy . In thisThe research team selected electronic databases and search terms in consultation with an information specialist. Three databases were selected for the review: Web of Science, Ovid Global Health, and PubMed Central. These databases were searched on September 1, 2020, with no limits applied on the type of publication or the country of study. Limits were applied on language (English only) and publication date (2000 onwards). Detailed search strategies for the three databases are outlined in Table S2 in the Citations for all retrieved studies from each of the databases were imported into a reference manager  and all The two analysts double screened all the records and compiled a list of studies included by both, and studies for which there were discrepancies. The full texts of discrepant studies were reviewed and discussed by the research team with reference to relevance criteria. This process led to further refinement of the study scope and discussion of emerging themes. One additional study was identified for inclusion during the data extraction stage from reviewing the reference lists of key studies.Quality criteria were drawn from the systematic review by George et al.  which weAll the studies that were included in this review used qualitative methods of data collection and analysis. One approach to cope with the large volume and complexity of information resulting from qualitative research is framework synthesis -34. ThisIn the first of the five stages of framework synthesis , familiaThe framework includes three key components of the effectiveness of CSP: equity, depth of engagement, and stage of engagement, and captured how they were reported in studies included in our review ,16,24,36The second component considers the depth of engagement in decision-making processes. Here, we employed a comprehensive five-degree spectrum that identifies a continuum of progressively deeper engagement in decision-making processes, encompassing and expanding upon Rowe and Frewer’s commonly cited typology ,7,37 froFinally, we examine the stage at which engagement occurs, corresponding with the aspects of the decision-making process that engagement is intended to inform. We adapted the WHO guidance  that invDuring the third and fourth steps of framework synthesis, indexing and charting, we used a data charting form in Excel for structured extraction of textual data from articles. This charting form included the categories of effectiveness from our conceptual framework, alongside categories to capture the type of decision-making activity, country, governance level , population, mechanism(s) of engagement, and barriers, facilitators, and recommendations. We coded types of decision-making activities inductively and identified three types: priority setting and decision-making processes for health service coverage, HTA, and decision-making processes for pharmaceutical coverage specifically. For the population, we adopted an inclusive approach to conceptualising relevant actors, adapting an existing framework. We used Mitton et al.’s categorisation of “multiple publics” into three groups: 1) “individual citizens speaking on their own behalf”, 2) “organized interest groups supposedly speaking on behalf of their membership”, and 3) “the public as patients or consumers of services” . Our revData extraction was conducted independently by the two analysts (RS and MA), with each reviewing half of the studies, communicating regularly to discuss uncertainties. Each analyst also conducted data extraction of two of the other’s assigned papers, and one analyst (MA) subsequently reviewed data extraction for all of the included studies, to verify consistency.In the fifth and final mapping and interpretation stage of the framework synthesis, we reviewed the extracted data and generated aggregative quantitative summaries of some categories, and conducted interpretive synthesis of other categories, such as barriers and facilitators . PrelimiOur initial search strategy yielded 3238 results, which was reduced to 2871 once duplicates were removed, and 44 following initial title and abstract screening  processes to set priorities and determine health service coverage and packages (12 studies), 2) processes using HTA in particular (11 studies), and 3) decision-making to determine public coverage of pharmaceuticals specifically (four studies). The majority of cases focus on decision-making at the national level (27 cases), while 12 examine sub-national levels (such as provinces and districts), and five were undertaken at the local level . While priority setting for health service coverage usually occurs at the national level, cases at sub-national or local levels are from devolved or decentralised decision-making contexts. Five studies examine multiple levels in a given country, including decision-making at national, sub-national, and / or local levels ,40-43. TA wide range and number of CSP mechanisms are reported, often with more than one mechanism used in each case. The most common mechanism across both HICs and LMICs is consultation activities (34 cases), including a wide range of in-person and written methods. Examples of these are found in Box 1• surveys and polling• workshops• interactive seminars• policy roundtable• citizen laboratory• citizen dialogue• citizens’ jury• public / community forums and debates• focus groups• public hearings• liaising with facility committees• evidence statements• “consumer impact” assessments• written submissions (often to public consultation documents)• deliberative decision-making simulations• media and internet-based methods• soliciting priorities and nomination of technologies and servicesSee Table S4 in the The second most common engagement mechanism is participation in dialogue and decision-making committees and boards (14 cases), which was identified in both HICs and MICs. These groups are distinct from CSP in councils, committees, boards, and panels that serve an advisory function rather than direct engagement in decision-making. The latter mechanism was identified in seven cases, all in HICs. Participation in district health planning processes, through district- and local-level meetings, was identified in five cases, all in decentralised systems in MICs. Finally, mechanisms for appealing health service coverage decisions are discussed in eight cases, across both HICs and MICs.Most of these are formal mechanisms occurring within government institutions and structures, however studies were often not clear about whether they were ad hoc and one-off or institutionalised and consistent over time. One factor identified as contributing to the institutionalisation of engagement mechanisms is the existence of legislation and the extent to which the voices of community actors are seen as critical to decision-making in the country’s constitutional and political processes ,45. Two Engagement mechanisms identified in this review often involve participants from across the four categories of actors found in our conceptual framework. Examples of “other stakeholders” are found in Box 2• government ministry and departmental officials• political and parliamentary representatives• health service managers, administrators, and staff• health care professionals• health system experts / analysts• health economists• academics and researchers• insurance company representatives• health technology company representatives• pharmaceutical company representativesSignificant gaps in the consideration, and / or reporting, of representative and distributional equity in CSP were identified based on our review. Only 12 cases discuss representativeness, with six explicitly reporting on the inclusion of vulnerable, marginalised, or special-needs groups in priority setting processes for health service coverage and a further six discussing the need for diverse representation in CSP ,44,46-54Consideration of distributional equity implications of CSP in decision-making about health service coverage is briefly discussed in only five of the 27 studies in this review, and no studies actually assessed or evaluated these ,47,52,57In terms of depth, most mechanisms identified in this review enable engagement at the level of consultation or involvement. The consultation level entails obtaining public opinions and experiences regarding priorities, values, services, and intervention preferences for health service coverage. Involvement entails working directly with community actors and other stakeholders in decision-making processes. No studies were identified that involve collaborative or empowered engagement, which can be defined respectively as partnering with the public on each aspect of a decision-making process (collaborative), and giving final decision-making authority to community actors and other stakeholders (empowerment).Studies included in this review describe engagement at all stages of health service coverage decision-making, though most commonly the data and dialogue stages. Engagement at the data stage includes involvement in nominating and selecting services and interventions for assessment, as well as providing information about patient and public needs and experiences to serve as evidence in assessing options for health service coverage. This was found across income levels and across types of decision-making, including determining health service coverage, HTA, and pharmaceutical coverage.The dialogue stage usually entails engagement in appraising services and interventions for inclusion in health service coverage. This includes both defining values and selection criteria for appraisal, as well as conducting the appraisal itself by discussing and prioritising among options. Engagement in this stage of decision-making was found across LMICs and HICs, and across types of decision-making. The National Institute for Clinical Excellence (NICE) in the UK is an example of the former, involving patient advocacy groups in determining assessment criteria for HTA . An examIn ten cases, CSP in decision-making committees and boards entails a role in the decision stage of priority setting for health service coverage. For example, patient representatives’ collaboration in the NICE Appraisal Committee represents a distinctive case in which the committee’s decisions are binding ,59. AnotFinally, we identified nine cases of CSP in the review stage, primarily through the establishment of mechanisms for complaints and appeals of health coverage decisions ,48-50,61A wide range of barriers and facilitators to CSP were identified in the studies included in our review, though most studies did not include formal evaluation, and many did not report on challenges. We developed a typology to summarise these and conceptualise the different domains of common challenges for engagement, including structural / normative, institutional, procedural, and technical barriers, as presented in Table S5 in the While a range of factors are identified as hindering meaningful engagement, power differentials and divides across different stakeholder groups emerged as a key issue ,58,60,62In their survey of cancer control decision-makers in Canada, Regier et al. found that their professional and disciplinary background influenced the value they placed on patient-generated evidence and its uptake in decision-making processes. Health care professionals with medical degrees were less likely to use patient input as evidence compared to decision-makers with social science backgrounds. The authors therefore emphasise the use of multi-disciplinary decision-making committees to increase patient and public input in priority setting .Among the 27 studies included in this systematic review, a wide range of mechanisms of engagement, populations, effectiveness outcomes, and barriers and facilitators to meaningful CSP were identified. Studies were concentrated in HICs, which may reflect not only a higher rate of CSP in these countries, but also more formalisation of decision-making processes and institutions for priority setting of health service coverage. Indeed, in a comparative study of four LMICs, Holtorf et al. found that only one had clearly defined HTA processes (Brazil) . As greaOne of the challenges for clear guidance and identification of best practices to date is the inconsistencies in how CSP is conceptualised and what constitutes meaningful participation. Reporting on who is involved in engagement mechanisms and equity considerations within this is a major gap in the literature; both among studies included in this review as well as the wider literature on participation in health governance. Central to this is how communities are conceptualised and the extent to which studies make a distinction between various stakeholders engaged in decision-making processes. This review revealed a need for further disaggregation of Mitton et al.’s categorisation of three “publics”, including individual citizens, organised interest groups, and individual patients . In partIndeed, a number of barriers and challenges for CSP reported in studies relate to equity in the representation of interests within health service coverage decision-making, including de-legitimising patients’ and public views and experience in relation to “experts”. Other studies reported concerns about patient group representatives advocating for the interests of private industry ,55,61,63The representation of private sector / industry stakeholders in decision-making processes is of particular concern, as the concept of “the public” is “prone to manipulation by those with strong interests” . Due to The distributional equity implications of conflicting interests are evidenced in a study by Yassoub et al. about stakeholders’ views on a health benefit package in Lebanon. They found that the weakest support for improving equity and access to health services came from the private health sector, which unanimously opposed the absence of out-of-pocket payments in the package . A studyOne strategy to mitigate conflicts of interests (among others) in order to facilitate effective CSP may be to increase the representation of community-based actors, including particularly marginalised and vulnerable populations. Our study found limited discussion of equity considerations in articles included in our review, and even less successful examples of equitable inclusion in practice. Indeed, the characterisations of “lay” participants and comments about participants’ education levels seem to demonstrate a biased perception of the “right” kind of community actors for participation, and potential inequities and failures in reaching out to the “left-behind” marginalised communities. This suggests at least a failure in tackling, and possibly further contributing to exacerbating, social inequities prevalent in health service recipient communities. We echo other authors in arguing that greater attention needs to be paid to diversity and representativeness within CSP, including key considerations such as clear and transparent nomination and selection criteria for representatives, rationales for the inclusion of particular actors and their role and practices in priority setting and decision-making, and methods to ensure equitable inclusion within participatory spaces to overcome power and resource imbalances ,19. ThesAn important facilitator for effective depth and decisive stage of CSP identified by our study is legislative provisions that compel the institutionalisation and continuity of CSP mechanisms. The examples provided by Thailand and Brazil demonstrate the role played by legal and constitutional provisions in ensuring CSP and promoting its effectiveness. Community engagement in defining the essential package of services for UHC in Thailand is widely recognised as a leading model. The success of engagement mechanisms in this context is understood to stem from the stipulations for engagement and accountability set out in the UHC legislation . This is ultimately the result of civil society advocacy, which led to the inclusion of these provisions through the submission of a draft bill during the drafting of the legislation . In BrazIn addition to the institutionalisation of engagement mechanisms, the stage at which they occur provides a measure of meaningful CSP. Our findings indicate engagement in all stages of determining health service coverage, however primarily at data and dialogue stages. When engagement did occur in the decision stage, the role of community actors and other stakeholders in contributing to final decision-making was unclear. As one study from Israel noted, the national advisory committee (which includes public representatives) that makes final recommendations on adding health technologies to the national list operates on a consensus basis, therefore the impact of any one committee member is unclear .Across all stages of engagement, evidence of the impacts of CSP on the actual outcomes of decision-making processes for health service coverage was lacking in the literature. Some studies cited anecdotal evidence, including one from Thailand in which the authors note that inputs from annual public hearings led to changes to the organisation and eligibility of services  . The deaThis study was limited by the inclusion of only English language articles. Moreover, while the use of three different databases is comprehensive, it is possible that relevant articles were missed if they were not indexed in one of the selected databases. Grey literature sources were also not included and might have added relevant empirical findings. Our search terms included “health care” or “health services”, which may have missed some relevant HTA studies as that literature may not explicitly mention these terms in titles or abstracts.Moreover, synthesising qualitative research is challenging, including deciding what to abstract from qualitative studies while navigating the complexity and contextual nature of their findings . We haveOur systematic review of the literature on CSP in decision-making processes for determining health service coverage found a dearth of relevant studies. Among those that were identified, most were descriptive and provided limited evaluative information. Our assessment based on a conceptual framework of engagement effectiveness identified limited consideration of representative or distributional equity, limited depth of engagement, and a focus on data and dialogue stages of decision-making processes, with some involvement in the decision stage. Overall, our results highlight a need for further research and evidence, for which our conceptual framework offers some key parameters.This review indicates that while global policy guidelines and resolutions commit to community engagement, such commitment has not translated widely to planning and design processes at country level. Countries that are taking promising steps in this direction seemingly do not pay adequate attention to researching the effectiveness of the mechanisms adopted, and conditions in which CSP may flourish. We call on the international health community to prioritise research in this area to generate evidence that can facilitate the creation of transformative governance and fulfil policy commitments.Some progress and growing interest in CSP in health system priority setting is reflected in the emergence of new research collaborations and guidance for participation, particularly in the field of HTA ,14,67-70Specific to the priority setting and decision-making context of determining health service coverage, our study contributes a first step in systematically reviewing and synthesising evidence of the effectiveness of CSP mechanisms. This may contribute to the inclusion of CSP considerations within technical support for priority setting for UHC, such as that provided by the international Decision Support Initiative (iDSI) ,14, as w


In the last 100 years, the world has experienced four influenza pandemics, with an occurrence every 15-30 years resulting in an annual probability of 3% to 7% [Developing strategies to promote robust basic and applied research with continuous monitoring of the human/animal interface to generate new knowledge applicable to public health policies and measures is critical to enhancing rapid responses to these outbreaks. Early detection and fast response to an epidemic or pandemic can be improved through an effective communication structure between countries worldwide. Such a structure should include strategically placed clinical research programmes and associated infrastructures.The United States (US) National Institute of Allergy and Infectious Diseases (NIAID) supports collaborative, government-to-government research partnerships with the Ministries of Health (MoH) of Mexico, Indonesia, Mali, Guinea, Liberia, and the Democratic Republic of the Congo (DRC), designed to both improve long-term research capacity and promote rapid response to disease threats.One key goal of these international collaborations is the establishment of human subjects’ research protocols to identify the early emergence of pathogens with pandemic potential. These collaborations also strive to determine the most effective treatments through randomised controlled trials (RCTs) designed and executed with adequate rigor to determine the safety and efficacy of potential therapies . During The World Health Organization (WHO) has a research and development blueprint identifying diseases and pathogens that can cause a public health emergency due to their pandemic potential and/or a lack of effective treatments and vaccines. This list includes “Disease X”, representing the need to prepare for an unknown pathogen that could cause a significant public health emergency . During The successful implementation of a clinical trial involves different actors, including representatives of affected communities, clinicians, biostatisticians, regulatory experts, laboratorians, pharmacists, regulatory experts, logisticians, and experts in communication. A clearly defined regulatory process, harmonised whenever possible on a global basis, and communication among regulators are crucial to their rapid implementation. International clinical research networks have substantially strengthened countries’ capabilities to implement clinical trials effectively and quickly.Since the H1N1 pandemic, we have witnessed how pandemic response has been a model of ensuring the implementation of state-of-the-art science to inform and influence public health policy . The recSuccessful evidence of the concepts mentioned above ie, surveillance, RCTs and regulatory communication is the The Mexican Emerging Infectious Diseases Clinical Research Network (LaRed), established in response to the 2009 H1N1 pandemic through a letter of intent signed between Mexican Minister of Health José Ángel Córdova and NIAID Director Anthony Fauci. The initial aim of LaRed was to develop a coordinated bi-national effort to conduct clinical research on influenza and potentially other infectious diseases. LaRed complements existing healthcare structures in Mexico and the US-generating scientific knowledge focused on treatment of emerging infectious diseases and respiratory viruses, with its mission being the generation of knowledge to improve health outcomes ..Since 2009, LaRed has conducted over 11 multicentre clinical studies and trials and is currently recruiting for two additional protocols. It has enrolled over 8617 study subjects, focusing on pathogens with pandemic potential and other causes of febrile illnesses Mexico is strategically located for the rapid identification of pathogens with pandemic potential. It has also become an immigration corridor for individuals from Africa, Asia, the Middle East, and Central and South America traveling to the North, who potentially bring new diseases that could pose a public health threat. Therefore, LaRed can rapidly mobilise in multiple clinical research sites in Mexico to implement surveillance, observational and clinical trials, as with influenza H1N1 pdm09, Zika, and COVID-19. Also, it has the capability for data collection on the natural history, vaccination, and breakthrough infections for various pathogens within crucial geographic regions.During the COVID-19 pandemic, LaRed has implemented two observational studies (NCT04385251 and NCT05096091) and participated in two RCTs: an interventional study for the treatment of outpatients in the early stages of COVID-19 (NCT04910269) and the Adaptive COVID-19 Treatment Trial (ACTT) (NCT04280705). ACTT generated scientific knowledge on the use of potential treatments and allowed some subjects to benefit from treatments not originally available as standard of care in Mexico in the early phases of the pandemic. The platform design of ACTT allowed four different drugs  to be evLaRed has also simultaneously collaborated with other networks across different continents. The group of six networks, alongside the ONUM (Sanskrit for spreading knowledge) Foundation in Mongolia, is conducting an observational cohort study of COVID-19 vaccine immunogenicity and allows the comparison of responses to different vaccines in different environments .Pamoja Tulinde Maisha (PALM) (Together Save Lives” in the Kiswahili language) Consortium at the Democratic Republic of the Congo (DRC), which operated through a multi-partner governance structure coordinated by WHO, the Ministry of Health, the Alliance for International Medical Action (ALIMA), and Médecins Sans Frontières (MSF) with the Institut National de Recherche Biomédicale and NIAID. They launched a RCT testing multiple investigational Ebola therapies, demonstrating that even during a conflict, clinical research can be conducted in an ethical, scientifically sound manner, and can help inform the outbreak response [Another example of how collaborative, government-to-government research partnerships have provided outstanding support to pandemic response is the response , contribPALM consortium (which were key in the detection and implementation of treatment protocols against Ebola), and the Indonesia Research Partnership on Infectious Diseases (INA-RESPOND) in Indonesia and the University of Clinical Research Center (UCRC) in Mali, studying causes of acute febrile illness, respiratory diseases, and tuberculosis, a critical re-emerging infectious disease. During the COVID-19 pandemic, clinical research programs in these countries were in place, acting as an essential structure to facilitate the needed rapid initiation of research protocols.Other host country – US partnerships have been developed, usually established during an outbreak and maintained within efforts directed toward preparedness. These include the Partnership for Research on Ebola Vaccines in Liberia (PREVAIL), the Partnership of Clinical Research in Guinea (PREGUI), and the aforementioned International research networks are a potent tool for delivering high-quality and productive research, encouraging co-operation and harmonisation to boost research delivery efficiency and facilitate preparedness to conduct clinical trials for health emergencies.The cooperation and collaboration with the regulatory entities in these countries have led to effective clinical research efforts that have generated scientific knowledge, supporting prompt and inclusive public health decisions with a social perspective. For example, during the COVID-19 pandemic, regulatory entities around the global developed specific policy responses, including shortened administrative procedures and new forms of coordinating committees . On NoveThe world must continue to support efforts on scientific research to prevent and/or effectively manage future pandemics caused by emerging infectious diseases. The “restless tide” of the microbial world, coupled with enhanced human mobility and environmental changes, necessitates a continued set of challenges from infectious diseases. Research partnerships and International Clinical Research Networks such as LaRed are essential to enhancing the ability of the global community to respond to the threat of emerging infectious diseases and to foster health diplomacy.