Ever since the development of the first vaccine more than 200 years ago, vaccinations have greatly decreased the burden of infectious diseases worldwide, famously leading to the eradication of small pox and allowing the restriction of diseases such as polio, tetanus, diphtheria, and measles. A multitude of research efforts focuses on the improvement of established and the discovery of new vaccines such as the HPV (human papilloma virus) vaccine in 2006. However, radical changes in the density, age distribution and traveling habits of the population worldwide as well as the changing climate favor the emergence of old and new pathogens that bear the risk of becoming pandemic threats. In recent years, the rapid spread of severe infections such as HIV, SARS, Ebola, and Zika have highlighted the dire need for global preparedness for pandemics, which necessitates the extremely rapid development and comprehensive distribution of vaccines against potentially previously unknown pathogens. What is more, the emergence of antibiotic resistant bacteria calls for new approaches to prevent infections. Given these changes, established methods for the identification of new vaccine candidates are no longer sufficient to ensure global protection. Hence, new vaccine technologies able to achieve rapid development as well as large scale production are of pivotal importance. This review will discuss viral vector and nucleic acid-based vaccines (DNA and mRNA vaccines) as new approaches that might be able to tackle these challenges to global health. The world population has grown to 7.6 billion people in 2018, more than half of which live in densely populated urban settings. Travel habits have changed radically; the number of people traveling by plane is growing each year and amounted to a total of 3.7 billion in 2016HIV), the causative agent of acquired immunodeficiency syndrome (AIDS), represents a zoonosis from non-human primates in West-central Africa and has claimed more than 35 million lives since its discovery in 1983Human immunodeficiency virus  on the one hand, that led to a phase 6 pandemic alert declared by the WHO in 2009 but caused relatively mild symptoms and the 1918 influenza A H1N1 pandemic (“Spanish flu”) on the other hand, that resulted in the deaths of around 50 million people (ne flu” HN1pdm09 oSevere acute respiratory syndrome (SARS) first occurred in China in 2002 and was caused by a novel coronavirus (CoV) that likely originated in bats (Middle East respiratory syndrome (MERS). Like SARS CoV, the virus originated in bats and likely spread to humans via infected dromedary camels. According to the WHO, there have been 2,143 confirmed cases of MERS, with 750 deaths in 27 countries since 2012. in bats . SARS Co in bats . A notab in bats . In 2012Ebolaviruses belong to the family Filoviridae (consisting of the two genera Ebolavirus and Marburgvirus) that cause hemorrhagic fever with a high mortality rate and whose natural reservoir is believed to be in bats  organisms is ever-growing, favored by the misuse and overuse of antibiotics. This holds true for the use of antibiotics in both humans and, even more problematically, in animals, where antibiotics are routinely used for prevention of disease and promotion of growth in livestock. MDR organisms, such as methicillin-resistant Staphylococcus aureus (MRSA) or multidrug-resistant tuberculosis (MDR-TB) are becoming a serious threat to global public health. According to WHO estimates, 490,000 new cases of MDR-TB were registered in 2016, of which only 54% could be successfully treated. Again, the solution to this growing threat could be the development of efficient vaccines to prevent MDR organisms from further spread.In addition to pandemic threats, the list of in vitro conditions or the requirement of a high biosafety level and specialized labs for cultivation. Hence, new and highly versatile approaches that are independent of whole pathogen cultivation are required to effectively and quickly combat outbreak situations.Conventional vaccines, developed by attenuating or inactivating the respective pathogen, have successfully decreased the burden of a number of infectious diseases in the past, leading to the eradication of small pox and significantly restricting diseases such as polio, tetanus, diphtheria, and measles. However, established methods may not always be suitable or even feasible in outbreak situations. Live attenuated vaccines generally bear the risk of reversion, rendering this approach unfavorable for highly pathogenic, possibly largely uncharacterized organisms. Inactivation may not induce protective responses, as is the case for Ebola  or can eIn order to proof effective against an upcoming pandemic, these new technologies need to overcome a number of challenges. The unpredictable nature of emerging pathogens represents one of the pivotal problems for pandemic preparedness. Zoonoses present a constant threat to introduce a previously uncharacterized pathogen into the population, as was the case for HIV as well as for SARS and MERS CoV. The outbreaks caused by pandemic influenza virus demonstrate the potential of a known pathogen to mutate and adapt to a new host or environment, with unpredictable outcomes for its immunogenic properties and the severity of symptoms it induces. As demonstrated by the recent epidemics and pandemics, the risk of such events is highest for RNA viruses, whose high mutation rates favor adaptability.Since the vaccine targets remain undefined before an outbreak occurs, time remains one of the major hurdles for effective vaccine development. Currently, the average development time for conventional vaccines from preclinical phase is more than 10 years , highligA further major problem is the cost associated with vaccine development and production: using established technologies, development of a new vaccine candidate is estimated to amount to more than 500 million USD, with further expenses to establish facilities and equipment ranging from 50 to 700 million USD . While sA further issue is production capacities of established methods, which are often insufficient to support global vaccination. Even if the potential threat is known and vaccine manufacturing technologies are established, like for pandemic influenza vaccine, production capacity to meet peak demands during a pandemic remains problematic. Thanks to efforts coordinated by the WHO, the potential production capacity for pandemic influenza vaccines in 2015 could in theory support the vaccination of 43% of the population with two doses of vaccine . HoweverEfforts to meet these challenges are made by monitoring viruses with high pandemic potential and programs, most notably Coalition for Epidemic Preparedness Innovations (CEPI), that finances and develops vaccines against likely pandemic threats.The past decades have witnessed the development of a wide array of new vaccination technologies ranging from targeted attenuation techniques of live pathogens to the delivery of biologically engineered protein and peptide antigens as well as viral vector and nucleic acid based antigens. Many of these technologies have yielded highly promising results which are discussed in excellent reviews elsewhere –21. HereViral vector based vaccines, that rely on the delivery of one or more antigens encoded in the context of an unrelated, modified virus, represent a highly versatile platform that offers many advantages over more established vaccine technologies. This technology either employs live (replicating but often attenuated) or non-replicating vectors. Research conducted since the 1980s has established a variety of viruses as vaccine vectors by engineering them to encode for heterologous antigens that are shuttled into the host cells by the vector. Upon delivery, antigens are expressed and the host is able to induce immune responses against the respective target pathogen .A wide array of different viruses has been employed as a basis for constructing viral vector based vaccines . Among oAdenovirus (Ad) vectors are among the most commonly employed viral vectors, with vast amounts of both preclinical and clinical studies assessing their protective efficacy against a variety of infectious diseases available. Adenoviridae are non-enveloped viruses with an icosahedral capsid and a linear double-stranded DNA genome, whose size ranges from 30 to 40 kb. Next to a multitude of adenoviruses occurring in different animal species, there are 57 identified human adenovirus that are classified into seven species A–G. Adenoviral receptors are expressed on the surface of most human cells, allowing a broad tissue tropism of the virus  genomic region by an antigen expression cassette, thereby abolishing the viral ability to replicate . In addiin vitro . Adenoviin vitro .+ T cell as well as antibody responses , a common human pathogen, belongs to the family of Paramyxoviridae. MV is an enveloped virus with a non-segmented, negative-sense, single-stranded RNA genome of ~16 kb. Measles virus vaccines have been generated by serial passaging of infectious virus through different cell lines resulting in a live attenuated virus that is replication deficient in humans. The introduction of numerous mutations in this process has established a highly stable vaccine for which reversion to pathogenicity has never been observed  complex, the viral genome is highly flexible and accepts insertions of up to 6 kb, as long as the total number of nucleotides in the genome can be divided by 6 (“rule of six”). The ability to accept relatively large transgenes offers the opportunity to generate multi-pathogen or multivalent vaccines . However+ phenotype, unlike the more CD8+ dominated response to adenoviral vectors, which might be a consideration for vaccine development. Since live attenuated MV is routinely used as a vaccine in child immunization programs in many countries, pre-existing immunity to MV as a viral vector has been raised as a concern. However, studies in mice and macaques showed no impact of previous MV exposure on transgene immunity , a member of the Rhabdoviridae family, is an enveloped virus containing a single stranded, negative-sense RNA of ~11 kb. The virus naturally infects livestock with sporadic infections found in humans , strong antigen-specific cellular and humoral immune responses against the target antigen can be induced. Strategies to achieve replication incompetency or attenuation of modern viral vectors generally ensure a good safety profile of viral vector based vaccines. For most commonly employed viral vector based vaccines, high yield production processes with means of upscaling have been established, supporting the use of these technologies for pandemic settings.Despite many advantages, several aspects have to be considered when developing a viral vector based vaccine. Firstly, viral vectors are genetically modified organisms (GMOs) and are therefore considered a potential risks to human health and environment associated with the release of these organisms. European regulatory agencies require environmental risks assessment (ERA) to evaluate potential environmental and health risks posed by the GMO . In the In terms of vaccine manufacturing, each viral system requires different cellular systems for high yield propagation, necessitating different manufacturing facilities for each viral vector platform. As viruses may undergo recombination during production, great care must be taken to keep cell cultures free of material that can lead to the emergence of recombined and uncharacterized pathogens . In geneViral vector based vaccines have been employed for the development of vaccines against many different pathogens in a vast number of preclinical and clinical studies. However, so far only one viral vector based vaccine, i.e., Dengvaxia, which is a recombinant Dengue vaccine based on the yellow fever attenuated strain 17D, has been licensed for human use. More comprehensive summaries of their applications in the context of prophylactic vaccines are published elsewhere , 29. In Ebolaviruses started as early as the 1990s. However, most approaches were still in preclinical stages when the Ebola pandemic emerged in 2014. Viral vector based vaccines against Ebolaviruses have been tested in the context of non-replicative vectors such as modified vaccinia strain Ankara (MVA), human adenovirus (Ad) and replication-defective recombinant chimpanzee adenovirus type3 (ChAd3 vaccine) as well as replication competent vectors including VSV-EBOV, human parainfluenza virus type 3 (HPIV3), recombinant cytomegalovirus (rCMV), and recombinant rabies virus (RABV). Clinical trials were conducted for VSV-EBOV, ChAd3 vaccine, Ad26-EBOV, Ad5-EBOV, HPIV3, and MVA-vector vaccine . Vaccination was found to be protective but required long vaccination schedules . This vaesponses . Furtheresponses . Given tesponses . This stesponses . Importaesponses . Importaesponses . A phaseesponses , results10 viral particles. However, immune responses waned several months after prime vaccination which could be prevented by boosting with MVA encoding for GPs from EBOV and SUDV . Especially ChAd3 is among the most widely evaluated vectors for the development of a vaccine against Ebola. Two vaccines developed by the NIAID VRC, i.e., replication defective ChAd3 encoding for EBOV GP alone or in combination with SUDV GP, were tested in preclinical studies which demonstrated complete protection in NHPs for both vaccines 5 weeks after single injection, using 10and SUDV . Startinand SUDV –63 . Reduction of viral titers employed for vaccination yielded reduced adverse events. However, while the frequency of GP specific antibody induction remained similar to cohorts vaccinated with higher doses (94%), levels of antibody responses were reduced.rVSV-ZEBOV currently represents the most promising candidate for the development of an effective vaccine against Ebolaviruses. This vaccine consists of a live attenuated VSV in which the VSV glycoprotein is removed and replaced with the GP from a 1995 EBOV strain. rVSV-ZEBOV was developed by the Canadian National Microbiology Laboratory and is now licensed to Merck. Preclinical studies published in 2004 and 2005, respectively, demonstrated complete protection from a lethal EBOV challenge infection in mice using a mouse-adapted strain  and NHPsOf note, further phase I clinical trials  Table  employin7 PFU in a cluster randomization design with a ring vaccination approach  was initiated in Guinea in March 2015 assessing vaccine efficacy upon vaccination using one dose of 2 × 10ch Table , 75. No 3 was iniNucleic acid based technologies employ either antigen encoding plasmid DNA or RNA, as messenger RNA or viral replicons. Upon their cellular uptake and expression, nucleic acid encoded antigens can elicit humoral as well as cell-mediated immune responses. Both technologies are extremely versatile due to the ease of antigen manipulation they allow. The production of antigens in the target cells offers the advantage of mimicking protein synthesis during an infection, i.e., protein localizations such as presence in the plasma membrane and modifications such as glycosylation patterns can be formed with a high degree of faithfulness. Importantly, they support the delivery of any antigen of choice, regardless of whether it was derived from a virus, bacterium or parasite, supporting vaccine development against a wide array of pathogens. Since vaccine characteristics are independent of the encoded proteins, development of different vaccines can take place without the need to establish new production, purification and validation methods as well as manufacturing facilities. Hence, nucleic acid based technologies support fast and flexible vaccine development and production. Since all vaccines can be produced using the same basic components, manufacturing of several vaccines can take place in one established facility cutting both costs and time of vaccine production dramatically. Lastly, their synthesis mostly relies on chemically synthesized material, supporting large-scale production with relative ease.Escherichia coli, i.e., an origin of replication that supports high yields of the plasmid during bacterial growth and a selectable marker, mostly the bacterial antibiotic resistance gene against Kanamycin, which allows stable inheritance of the vector. Since regulatory safety concerns have been raised against the presence of non-functional sequences, especially the antibiotic resistance marker, for human use, the marker has been replaced or removed in new generations of DNA vaccines (DNA vaccines are generated by insertion of a eukaryotic expression cassette encoding for the antigen(s) of choice into a bacteria-derived plasmid. The plasmid backbone generally contains elements that permit propagation and selection of the vector in vaccines . In addivaccines  and the vaccines , have bevaccines .in vivo electroporation, which is among the most widely used and has been shown to yield promising results in both preclinical and clinical trials  or intradermal (ID) injection using a conventional needle. However, vaccination with a DNA vector alone generally leads to relatively low immunogenicity, especially in large animal models and humans. A factor that may play a role is the need for DNA vaccines to cross two cellular membranes, i.e., the plasma, as well as the nuclear membrane, in order to achieve protein expression. Of note, this does not hold true for RNA vaccines, which are translated upon crossing the plasma or endosomal membrane, respectively. Hence, additional methods have been developed that are able to enhance DNA uptake, expression and immunogenicity. These include various delivery devices such as gene gun, needle free injection devices (jet injection) and l trials , 80. Furl trials . These ml trials . In addi+ cytotoxic and CD4+ helper T cells, respectively, the exact mechanism of action remains to be evaluated. Upon entry in the cell, DNA vaccines are sensed by a variety of innate immune receptors. While TLR9 is not critical for DNA vaccine efficacy, the STING/TBK1/IRF3 pathways and the AIM2 inflammasome are involved in DNA vaccine mode of action and other factors might additionally be involved , to induce neutralizing antibodies as well as nucleoprotein (NP) as a target for antibody as well as T cell responses. Induction of both humoral and T cell-mediated immunity as well as protective efficacy against rodent adapted viral strains was demonstrated in guinea pigs and mice, upon vaccination with DNA encoding for GP and NP using intramuscular injection or intradermal delivery using a gene gun, respectively , one of the viral surface antigens and the main target for neutralizing antibodies against influenza. In terms of pandemic preparedness in DNA only vaccination strategies, Vical Inc. has developed and tested a vaccine that targets the highly pathogenic avian H5N1 influenza endemic in poultry. Its ability to cross the species barrier, that was first discovered in 1997 and caused rising numbers of human infections between 2003 and 2008, renders this virus a high pathogenic risk. So far, the virus is not able to spread efficiently and sustainably from human to human but H5N1 bird to human infections have caused the death of 453 people worldwide until 2017.A large number of preclinical and clinical studies have assessed the ability of DNA vaccines to mediated protection against Upon emergence of a novel H1N1 influenza that originated in pigs and became pandemic in humans in spring 2009 , effortsZika crisis in 2016. Leveraging knowledge generated in the context of other flaviviruses, these approaches rely on the expression of the precursor membrane and envelope (Env) (prM-E) proteins which are known to form subviral particles with Env being the target of virus neutralizing antibodies. The first approach developed by Inovio employed a consensus prM-E derived from African and more recent Asian and American strains modified to contain an IgE signal peptide with a putative glycosylation site removed (GLS-5700)  . This vaLS-5700)  demonstrA preclinical study published in October 2016 demonstrated the induction of neutralizing antibodies and protection from challenge infection in 17 of 18 NHPs upon two IM vaccinations using a needle free injection device. This study employed two different prM-E constructs based on the sequence of French Polynesian and early Brazilian ZIKV isolates in which the Zika prM signal sequence alone (VRC5283) or in combination with the stem and transmembrane regions (VRC5288) were exchanged with the corresponding sequences from Japanese encephalitis virus (JEV). Both vaccine candidates are evaluated in clinical studies by The Vaccine Research Center (VRC), National Institute of Allergy and Infectious Diseases (NIAID) Table . Both trBoth DNA based approaches for the development of an effective Zika vaccine appeared safe for human use and yielded promising results. Importantly, they were initiated within months after sequences became available, highlighting the versatility and speed provided by DNA vaccine platforms.Non-replicating mRNASelf-amplifying mRNAmRNA is an intermediate carrier of genetic information used as template for endogenous protein production in the vaccinated subject. Two major types of RNA have been utilized as prophylactic vaccines against pathogens that cause infectious diseases:Non-replicating mRNA contains the sequence of the antigen of choice flanked by 5′ and 3′ untranslated regions (UTRs). The advantages of using non-replicating mRNA vaccines compared to self-amplifying mRNA are rooted in the simplicity of the construct, the small size of the RNA, and the absence of any additional encoded proteins that could induce unintended immune responses  produced in E. coli. The pDNA template is linearized using restriction enzymes and is transcribed in vitro into mRNA in a mixture containing recombinant phage DNA-dependent RNA polymerase  and nucleoside triphosphates (NTPs)  flanked by elements essential for the function of mature eukaryotic mRNA: a cap structure, joined to the 5′ and a poly(A) tail at the 3′ end, as well as 5′ and a 3′ untranslated regions (UTR) (esponses . The desesponses –107. Bris (NTPs) . Upon pus (NTPs) . Notablyied mRNA . The in ns (UTR) –113. Thens (UTR) , 114. Thns (UTR) , or subsns (UTR) . The UTRns (UTR) –120. Thens (UTR) , 122. Thns (UTR) –125 or bns (UTR) . Utilizans (UTR) .Self-ampifying mRNA vaccines are most commonly based on the alphavirus genome  are efficient transfection reagents for mRNA in cell lines and primary cells  primarily targets the liver  injection delivers mRNA vaccines directly into the skin, an organ densely populated with professional APCs such as Langerhans cells in the epidermis and various dendritic cells (DC) subtypes in the dermis. The ID route of administration has been shown to effectively induce a balanced immune response including antibodies as well as Th1 type and cytotoxic T cells for mRNA vaccines formulated in protamine or LNP  injection of vaccines is the most often practiced route of administration in humans. Since this route of vaccination is simple to carry out and does not require much training for its implementation, it may be the preferred route of administration by the physicians carrying out immunization in regions affected by a pandemic. However, the need for educated personnel to vaccinate people might represent a limiting factor in the face of a pandemic. The induction of strong immune responses after IM injection of mRNA represents a high hurdle, due to lack of co-stimulatory molecules and optimal antigen presentation on muscle cells and low infiltration of the muscle tissue by immune cells. Thus, potent IM mRNA vaccines must allow high antigen expression and presentation and simultaneously induce strong immunostimulatory signals to recruit immune cells to the injection site. The IM administration of non-replicating nucleoside-modified mRNA-LNP vaccines against the Zika virus, as well as influenza A H10N8 and H7N9 viruses proved to be immunogenic and provide protection in preclinical studies in mice, ferrets and NHPs  which results in a slowdown and eventually inhibition of protein translation. Pepini et al. report that a self-amplifying mRNA vaccine elicits an inflammatory response within a few hours indicated by the upregulation of several IFN-stimulated genes and that antigen expression and immunogenicity were both enhanced in the absence of IFN-α/β signaling, suggesting that reduction of early type I IFN responses could improve RNA vaccine potency . Several in mice , 170. Thleosides . HoweverIn recent human clinical studies, mild to moderate and in rare cases severe local and systemic reactions were reported for different mRNA platforms , 159. Fuin vitro systems and does not require amplification in bacteria or cell cultures, manufacturing of mRNA vaccines is a comparably short and simple to monitor process. As mRNA vaccines do not interact with the host-cell DNA, they avoid the potential risk of genomic integration posed by DNA-based vaccines. Since mRNA vaccines represent a minimal vector containing the ORF encoding the antigen of choice flanked by specific regulatory elements, they do not induce anti-vector immunity as observed for certain viral vector-based platforms  were immunogenic in ferrets, facilitating containment of viral replication in the upper respiratory tract upon influenza infection and conferred protection against homologous and heterosubtypic viral challenge in mice (+ T cell responses against Zika virus ( in mice , 178. A  in mice . RNA rep in mice . Chahal ka virus , 181. Honon-replicating mRNA vaccines to induce immune responses and confer protection against pathogens with pandemic potential such as ZIKV, EBOV and influenza. Importantly, some of these approaches are currently being tested in clinical trials. Pardi et al. demonstrated that ID immunization with LNP-encapsulated modified mRNA encoding the prME glycoproteins of ZIKV elicited potent and durable neutralizing antibody responses that were protective in mice and NHPs (in utero transmission of ZIKV in mice (A variety of preclinical studies have demonstrated the ability of and NHPs . A subse in mice , 163. A Ebolavirus vaccines, LNP-encapsulated modified mRNA encoding EBOV GP delivered IM was shown to induce EBOV-specific IgG and neutralizing antibody responses and protected guinea pigs against lethal infection and signs of clinical illness (In the context of  illness . Howeverinfluenza. Petsch et al. were the first to demonstrate that ID administration of protamine-complexed non-replicating sequence-optimized mRNA vaccines encoding influenza HA was protective in mice upon homologous challenge with influenza H1N1, H3N2, and H5N1 and was immunogenic in ferrets and pigs (Several studies have demonstrated to ability of mRNA vaccines to elicit protective immune responses against and pigs . Furtherand pigs . A recenand pigs . A singlOverall, these data show that non-replicating LNP-encapsulated mRNA vaccines can induce functional antibody titers at levels associated with protection with acceptable tolerability profiles upon parenteral administration. Future studies that employ LNPs for encapsulation of non-replicating mRNA targeting diverse and more complex antigens are required to demonstrate the broad applicability of this vaccine platform against pathogens posing potential pandemic threats.Pandemics such as HIV, Ebola, and Zika have raised the awareness of global threats to human health posed by known as well as newly emerging pathogens and can provide the impetus to prepare against future pandemics by promoting the development of vaccine platforms that can tackle the challenges of outbreak situations. New platforms, such as viral vector and nucleic acid based vaccines meet the prerequisites to provide solutions for some of these challenges by representing highly versatile technologies that allow fast vaccine manufacturing. Each vaccine technology has its own advantages and disadvantages related to its ability to induce certain immune responses, manufacturing capacity and safety for human use Table . Viral vWhile it seems unlikely that a single technology will be able to provide a solution for each future outbreak situation, the combination of present knowledge, ongoing development and the growing understanding of human immunology can provide tools to successfully combat emerging global threats.SR responsible for writing and coordination of the manuscript. EJ wrote part of the manuscript. KS wrote part of the manuscript. BP discussed the manuscript and reviewed the document.SR, EJ, KS, and BP are employed by CureVac AG.


We employed an agile, user-centered approach to the design of a clinical decision support tool in our prior integrated clinical prediction rule study, which achieved high adoption rates. To understand if applying this user-centered process to adapt clinical decision support tools is effective in improving the use of clinical prediction rules, we examined utilization rates of a clinical decision support tool adapted from the original integrated clinical prediction rule study tool to determine if applying this user-centered process to design yields enhanced utilization rates similar to the integrated clinical prediction rule study.MATERIALS & METHODS: We conducted pre-deployment usability testing and semi-structured group interviews at 6 months post-deployment with 75 providers at 14 intervention clinics across the two sites to collect user feedback. Qualitative data analysis is bifurcated into immediate and delayed stages; we reported on immediate-stage findings from real-time field notes used to generate a set of rapid, pragmatic recommendations for iterative refinement. Monthly utilization rates were calculated and examined over 12 months.We hypothesized a well-validated, user-centered clinical decision support tool would lead to relatively high adoption rates. Then 6 months post-deployment, integrated clinical prediction rule study tool utilization rates were substantially lower than anticipated based on the original integrated clinical prediction rule study trial (68%) at 17%  and 5% . User feedback at 6 months resulted in recommendations for tool refinement, which were incorporated when possible into tool design; however, utilization rates at 12 months post-deployment remained low at 14% and 4% respectively.Although valuable, findings demonstrate the limitations of a user-centered approach given the complexity of clinical decision support.Strategies for addressing persistent external factors impacting clinical decision support adoption should be considered in addition to the user-centered design and implementation of clinical decision support. Yet, these objectives have been constrained by well-documented barriers to CDS effectiveness, including poor usability, alert fatigue, and low utilization.35 Together, these barriers have impeded the impact of CDS on health outcomes, quality of care, and cost reductions.68Clinical decision support (CDS) tools have been at the forefront of digital health solutions for more than 10 years.9 In these approaches, CDS tools are created in collaboration with users to best reflect needed content, workflow, and ease of use.11 Commonly used in digital marketing and online commerce, user-centered design principles are increasingly used by the healthcare industry to increase usability, acceptability, and effectiveness of healthcare information technologies.13 In a systematic review comparing models of adoption of CDS, the most significant factors driving adoption was a system’s ability to be dynamic, launch “multiple assumptions,” and incorporate “new information in response to changing circumstances,” all factors heavily dependent on an explicit understanding of system end users.14To address these barriers, CDS adoption studies have increasingly used an agile, user-centered design (UCD) approach with a focus on human-computer interaction (HCI).15 The four-phase model, executed by a team of multidisciplinary stakeholders, uses a process of discovery paired with product definition and development phases characterized by rapid cycle agile testing—an implementation involving wireframing, workflow analysis, and usability testing—to develop and update tool design based on feedback regarding user needs and shifting contexts.17 Importantly, the model takes into consideration the CDS module lifecycle, including the need to consider adaptations or further “optimization” as additional user needs are identified or changes in practice or technology, such as electronic health record (EHR) functionality, are encountered.18 The design and workflow of the tool was created through iterative cycles of design, function, and usability testing, and content creation and revision with stakeholders; this resulted in a CDS tool with an adoption rate five times greater than previous reports.20 In nearly two-thirds of opportunities, clinicians used the offered iCPR1 tool, an antibiotic evidence-based CDS pathway, in more than half of all iCPR1 encounters; they went on to use the associated bundled order set . This represents a substantial improvement from earlier studies of CDS tools for acute upper respiratory infection, which demonstrated utilization rates as low as 6%.21 The success of the original iCPR1 tool points to the strengths of the agile, user-centered approach to CDS tool design and highlights the value of our tailored application of this industry practice for academic research.2224Our previous study ) employed this UCD framework to develop an iCPR CDS tool for reducing antibiotic prescribing for upper respiratory infections.26 Findings highlight an understanding of general clinic workflows, specific workflows, policies, and practices around rapid strep and chest x-ray testing, and clinic organizational structure as related to implementation process and tool requirements.25 Near-live and live usability testing was conducted, and the iCPR2 tool iteratively revised accordingly, as reported in previous publications.2628Our objectives for the adapted iCPR project (iCPR2) were to apply this approach to adapt and scale up the implementation of the iCPR CDS tool to a diverse group of 33 primary care clinics across multiple institutions. To understand if applying a user-centered process to adapt CDS tools is effective in improving the use of clinical prediction rules, we assessed resulting utilization rates, and ultimately, the tool’s impact on antibiotic prescription rates for acute respiratory infection in two academic health systems. Prior publications from this project outline the initial user-centered tool design process including initial tool development based on feedback from assessment-focused key informant interviews with providers, clinic managers, and medical assistants at each site.The objective of this study was to examine utilization rates of the adapted tool (iCPR2) across sites over 12 months post-deployment to determine if applying this user-centered process to the design of the tool yields enhanced utilization rates similar to that in iCPR1.As part of a large, randomized controlled trial involving over 40,000 visits in which we adapt and scale up the implementation of the iCPR1 tool to diverse clinics in two academic health systems, we collected qualitative user feedback during tool development, and quantitative and quantitative data on tool use over the first 12 months of implementation. Utilization data were reviewed biweekly for 12 months post-deployment to assess ongoing engagement with the tools by site and to determine the need for and direction of further user feedback, data collection, and tool iteration, with the goal of identifying barriers and facilitators to use as indicated by differences in utilization rates post-deployment.pre-deployment using: 1) key informant interviews with CDS and clinical content experts and local clinical leaders, 2) think-aloud, 3) near-live, and 4) live usability methodologies to gather information for initial tool development and iteration; the use of these methodologies in collecting usability data for the purposes of adaptive design is described in detail in our previous publications and summarized below.28Qualitative usability data were collected n=12) were presented with simulated cases and asked to “think-aloud” by verbalizing thought processes as they interacted with the prototype iCPR2 tool to examine usability aspects of the tool.The initial prototype of the iCPR2 tool was subjected to multiple rounds of usability testing in increasingly realistic clinical simulations. During these usability tests, providers (Post-deployment usability data were collected via semi-structured group interviews with providers across a convenience sample of the 33 intervention clinics (n=14) to collect qualitative user feedback 6 months post-deployment. All providers  who may have been exposed to the iCPR2 tool in practice at intervention sites were invited via clinic administrators to participate in a 30–45 minute group interview session in which they were asked questions related to usability aspects of the tool  with provider workflows. Observational “live” usability was conducted with three providers using the tool in six real clinical encounters to assess previously unidentified barriers to tool use. Morae® and Camtasia® software. Post-deployment group interviews were recorded with detailed field notes and then summarized. Qualitative usability data analysis is bifurcated into immediate and delayed stages; this paper reports immediate stage findings aimed at generating rapid, pragmatic recommendations for iterative refinement as they apply to design decisions and tool utilization. Participant comments were placed into a priori (according to key usability principles) as well as inductively derived coding categories, and analyzed for generalizable themes to translate into tool design recommendations. The delayed phase of analysis features deeper, systematic thematic analysis of transcribed field notes to identify generalizable insights for CDS.Full screen capture and audio was recorded for each think-aloud, near-live, and live-usability session using To determine the rate and variability in utilization of the iCPR2 tool across settings, monthly utilization rates  at each participating health system were calculated and compared at 6 and 12 months post-deployment to determine if utilization increased with tool iteration. To accomplish this, weekly reports were generated on the utilization of each step of the iCPR2 tool pathways for each intervention site. These reports identified rates of tool triggering by the relevant chief complaint and actions taken (or not taken) by the providers .The study was conducted at primary care clinics associated with two large academic health systems: one Midwestern  and the other in the Intermountain United States . All general internal medicine (GIM) and family medicine (FM) primary care clinics at the two institutions were invited to participate. A total of 33 individual clinics  participated in the study. Each site used the same EHR system  and its native functionality  to develop the iCPR tools in their EHR. Each site was supported by an information technology department that adapted and tested the components of the iCPR tool before deployment.19 In the pre-deployment phase of the current study, the iCPR2 design was developed by an interdisciplinary team of experts in primary care, usability, and clinical informatics at each institution .25The iCPR2 tool and workflow were adapted from those developed in our original iCPR study, as outlined above.The initial iteration of the new iCPR2 tool was triggered by relevant activity in any of the following EHR fields: chief complaint, diagnosis, or diagnosis along with antibiotic ordering. When triggered, the clinician is presented with an alert offering the iCPR2 tool upon opening the chart. If the alert is accepted (versus dismissed or ignored), the clinician is taken to a screen with a list of clinical questions, each of which contributes to a total risk score  longer list of more specific but potentially mismatched chief complaints, ultimately prioritizing alert sensitivity over specificity. To balance out the low specificity triggers and minimize clinician frustration and alert fatigue, we chose to use non-interruptive alerts  for the initial iCPR2 chief complaint-based trigger.By 6 months post-deployment, it had become apparent that iCPR2 tool utilization rates were substantially lower than anticipated based on the original iCPR trial (68%), with calculator completion rates of 17% at Health System A and 5% at Health System B. These results were the impetus for the additional site visits and post-deployment provider group interviews described below. At 12 months, after additional refinements based on user feedback at 6 months, utilization rates remained similarly low at 14% and 4% respectively.Post-deployment, provider-group interviews with over 75 providers in 14 intervention clinics across sites highlighted several persistent usability issues and other potential drivers of the relatively low utilization rates of iCPR2 , which wTo address the potential for lack of timely training for new providers as a potential driver of low utilization rates, additional academic detailing was held for all physicians and specifically newly on-boarded physicians 6 months post-deployment. An analysis of click counts required for use of Health System A’s calculator and order set for each condition was performed to explore opportunities to relieve burden by reducing clicks .As indicated in This study sought to adapt an innovative CDS tool across diverse ambulatory settings using a user-centered, agile approach to design and implementation. Our user-centered approach allowed for a rapid yet rigorous tool-adaption plan involving substantial engagements with study providers in multiple settings, from pre-deployment, think-aloud sessions to post-deployment group interviews with users. Based on pre- and post-deployment user feedback, the iCPR2 tool was modified iteratively and site specifically and, ultimately, most of the original iCPR CDS tool and its underlying evidence was conserved, while accommodating new workflows in the adapted iCPR2 model.We hypothesized that the integration of a well-validated, highly usable CDS tool into the EHR would lead to relatively high adoption rates and, as a result, reduced antibiotic prescription and diagnostic test ordering as in the original iCPR study. However, our findings demonstrate the complexity of CDS tools and the likelihood that, even with the use of this state-of-the-art approach to digital health design and testing, there can be persistent external factors impacting adoption rates that we could not address. Process measures from the intervention arm indicated substantially lower utilization rates than hypothesized at the study outset despite our rigorous adaptive design process.Adoption rates appear to demonstrate a stepped pattern. Essentially, the lower the adoption rate, the more divergent it is from the original iCPR tool design. As indicated in 32 Since that time, the EHR has become inundated with alerts vying for providers’ attention. It is certainly plausible that the reduction in adoption seen in iCPR2 is in part due to the higher level of CDS alert fatigue creating a higher threshold of importance required for providers to engage with a tool like iCPR2. Unfortunately, the design changes that were recommended for decreasing less clinically relevant triggers were not possible in the current EHR environment; this finding points to the urgency of identifying methods for “smarter” alerting whether through machine learning, personalization by role, or exploring potential to build and incorporate CDS tools outside of the EHR.Key user feedback throughout the iCPR2 tool development and iteration cycle thus far has highlighted alert fatigue and the potential for the tool to contribute to already heavy demands on provider attention as barriers to acceptance. The original iCPR trial was conducted early after the implementation of the EHR at that health system, a time when CDS tools were less pervasive. At the time of the original iCPR study, just a few CDS tools and alerts were implemented within its EHR, and the providers had likely not yet built up significant “alert fatigue.”33 Should the professional level of the user be a significant factor in CDS usage, the lower concentration of trainees at iCPR2 study sites could be a driver of the lower rates of tool usage compared to the original study.Changes in study setting related to professional level of the pool of users , but less proactive engagement as clinicians gain experience with the tool and their practice behaviors maintain acceptable clinical standards.Low adoption rates may also be an unintentional impact of over-design of the tool to accommodate clinical workflow. A purposeful decision to guard against burden, the user-centered approach may have, particularly in the case of Health System B, allowed the workflow to overcorrect by completely sacrificing visibility for the sake of accommodating expressed provider preference for low burden, passive alerts. This fact limited the tool’s ability to influence provider behavior and engagement with the CDS tool. We are revising Health System B’s workflow to more closely align with Health System A and the original iCPR workflow to understand if the low use reflects design or other factors such as provider culture. This speculation does not suggest that user-centered design is inappropriate for the adaptation of CDS tools in established EHR workflows. Instead, it calls attention to the need for a user-centered design approach in the context of CDS, and potentially other healthcare-related technologies, to balance the priorities of minimizing disruption to workflow and user burden with an understanding that some level of workflow disruption may be required to achieve sufficient user engagement with new tools.This study is limited by some key challenges. Although we can make some alterations to workflows regarding CDS, we cannot change the actual content of the tool as clinical decision support’s primary role is to provide evidence-based content to users. Additionally, the user interface of the CDS tool is constrained by the EHR vendor software, which limits the design changes able to be made.Although our research indicates users agree the tools are clinically relevant, they may not agree there is a clinical need for these tools. To the degree to which provider buy-in may be associated with level of training, results from iCPR1 and comparing overall adoption rates in iCPR1 versus iCPR2 (in which study sites contain more experienced providers), suggest that buy-in may be a driver of adoption. However, the design of this study does not provide a way to determine the degree to which buy-in affects adoption rates. Similarly, we cannot adjust for changes in practice of CDS and that providers’ experience with or attitude toward the iCPR2 tool may be shaped by experiences with CDS tools, including “alert fatigue.” A greater breadth of feedback from live usability sessions may have been possible with more than the three providers/six encounters studied; given the resource-intensive nature of this type of data collection and analysis, however, this is a reasonable sample size for live usability studies.Another limitation may be our choice of common patient symptoms of cough and sore throat for triggers. Not only are more of the triggers not applicable to the clinical situation contributing to increased alert fatigue, but as evaluation and treatment of these common symptoms occurs so often during primary care medical training and practice, more senior residents and attending physicians will have already developed a routine of history, physical exam, and treatment methods. Because they “know what to do,” changing clinical behavior by adding the iCPR2 tools may be less effective. Lastly, our slow response to poor adoption rates posed an additional limitation, born of the fact that unlike a “start-up,” we must work within parameters of a large healthcare organization, which requires great preparation and evidence to implement change.This study provides an example of how to leverage the principles and techniques of UCD and HCI approaches to tailor CDS interventions to site-specific workflows to support utilization rates necessary to positively impact clinical outcomes, such as reducing rates of inappropriate antibiotic prescription in the case of iCPR. Although there are limitations to CDS as illustrated above, the adaptability of these tools, when designed and implemented within user-centered frameworks, contributes to CDS tools that are more likely to fit within site-specific workflows and, subsequently, be used more often. We found that although the tools implemented in iCPR2 were designed to meet the specific needs of sites’ workflows and provider preferences as in iCPR1, utilization did not reflect the rates expected from the implementation a similar user-centered process. Our findings offer evidence that although necessary, user-centered design approaches are not sufficient for effective CDS design; our results suggest that additional factors, including structural ones such as the EHR environment and predominance of alert fatigue, are driving the degree to which a CDS tool such as iCPR2 is used. Novel questions and approaches are needed to explore the potential and limitations of UCD approaches to positively impact CDS. Our results suggest examining CDS adoption more broadly to ask the question: how might we address the problem of poor EHR usability more broadly and examine how that might affect adoption of clinical prediction rules such as iCPR2? Additionally, given the digital demands of physicians, exploring adaption of the iCPR2 tool for use by those other clinical roles such as nurses may provide insight into the factors impacting CDS adoption.


EDA disrupting the prenatal development of ectodermal derivatives. Cardinal symptoms are hypotrichosis, lack of teeth, and hypo- or anhidrosis, but the disease may also evoke other clinical problems. This study aimed at investigating the clinical course of XLHED in early childhood as the basis for an evaluation of the efficacy of potential treatments.X-linked hypohidrotic ectodermal dysplasia (XLHED) is caused by pathogenic variants of the gene 25 children (19 boys and 6 girls between 11 and 35 months of age) with genetically confirmed XLHED were enrolled in a long-term natural history study. Clinical data were collected both retrospectively using parent questionnaires and medical records  and prospectively until the age of 60 months. General development, dentition, sweating ability, ocular, respiratory, and skin involvement were assessed by standardized clinical examination and yearly quantitative surveys.All male subjects suffered from persistent anhidrosis and heat intolerance, although a few sweat ducts were detected in some patients. Sweating ability of girls with XLHED ranged from strongly reduced to almost normal. In the male subjects, 1–12 deciduous teeth erupted and 0–8 tooth germs of the permanent dentition became detectable. Tooth numbers were higher but variable in the female group. Most affected boys had no more than three if any Meibomian glands per eyelid, most girls had fewer than 10. Many male subjects developed additional, sometimes severe health issues, such as obstructive airway conditions, chronic eczema, or dry eye disease. Adverse events included various XLHED-related infections, unexplained fever, allergic reactions, and retardation of psychomotor development.This first comprehensive study of the course of XLHED confirmed the early involvement of multiple organs, pointing to the need of early therapeutic intervention. The term ectodermal dysplasia (ED) refers to a heterogeneous group of rare congenital conditions affecting the normal development of ectodermal structures including skin, teeth, hair, nails, and eccrine glands . X-linkeEDA; NM_001399.4) leading to loss or dysfunction of the signaling protein EDA1 [EDA1R, a gain-of-function allele associated with hair thickness and shovel-shaped incisors that predominantly occurs in the Native American and East Asian population, was found to attenuate the severity of symptoms in at least one familial case of XLHED [XLHED is caused by variants of the X-chromosomal ectodysplasin A gene  and completed this study. The protocol included the retrospective collection of clinical data using parent questionnaires and medical records, standardized systematic clinical examinations both at the time of enrolment and at the age of five years, repeated facial photographs, and yearly interrogations of the parents with a quantitative phone survey plus gathering of all relevant medical documents during the course of the study. Final data analysis was conducted at the age of 5 years when all deciduous teeth should have erupted and tooth germs of the permanent dentition except molars M3 (wisdom teeth) are expected to be calcified sufficiently to be detectable in panoramic radiographs.19 male and 6 female patients between 11 and 35 months of age with genetically confirmed diagnosis of XLHED were enrolled in a long-term natural history study at our site (Written informed consents of both parents (if available) to the participation of their child in this study were obtained. The study was approved by an independent institutional ethics committee and conducted according to national regulations and GCP/ICH guidelines. Subjects were included only if they had not received any investigational treatment prior to enrolment. Further exclusion criteria were congenital anomalies outside of those considered to be associated with XLHED, known hypersensitivity to pilocarpine or pilocarpine-like drugs, and implantable electronic devices.EDA mutations and screening for the polymorphism rs3827760 in EDA1R are available upon request.Standard gene variant analysis including DNA extraction, polymerase chain reaction, and Sanger sequencing was performed as described previously . Specifi2 were compared with standard percentiles for the normal population [Anthropometric measurements were obtained retrospectively from the child’s medical records or during the pediatric examinations. Body length in cm, weight in kg, and body mass index (BMI) in kg/mpulation . Dentiti2 was determined by confocal laser scanning microscopy with the VivaScope 1500  and extrapolated to an area of 1 cm2. Sweating ability was assessed by quantification of pilocarpine-induced sweating (volumetry) in an area of 57 mm2 of the forearm for 30 min using the Wescor 3700 device  as described previously [The number of plantar sweat ducts in a skin area of 36 mmeviously .Symptoms of XLHED-related chronic dry eye were assessed by an experienced ophthalmologist as described previously. This included corneal examinations for signs of keratitis by staining of the ocular surface, Meibography  , measurePulmonary function tests for the detection of asthma and bronchial obstruction included measurements of forced vital capacity (FVC), forced expiratory flow over 1 s (FEV1), and exhaled nitric oxide (eNO) level , a scoring system originally established for the assessment of extent and severity of atopic dermatitis. The EASI score for the whole body is the sum of scores for each of four defined body regions  and can range from 0 to a maximum 72 . The particular body scores were obtained by multiplying the sum of severity scores  of the four key symptoms  with the area score  and by multiplying the result with the constant value for each body region proportionate to its percentage of the body surface –26.The patients’ parents were asked once a year to fill in a questionnaire referring to different clinical and social aspects of XLHED. These surveys focused on the evaluation of the child’s sweating ability and (impaired) heat tolerance, XLHED-related issues with hair, teeth, eyes, nose and airways, voice, nutrition, skin, and nails. Continuous records of the various parameters including all data collected at the scheduled clinical examinations were analyzed. Box-and-whisker plots were used for the graphical depiction of the following statistical values: minimum, first quartile, median, third quartile and the maximum.EDA genotypes of all subjects are listed in Table EDA1R. Parent-reported details on the medical history of each child (parent questionnaires) confirmed the known spectrum of XLHED symptoms; information from medical records was added if required  with clearly fewer sweat ducts, and their brother  who does not seem to have sweat glands at all. Average body temperatures measured during clinical examinations at the study site  but in none of the 6 girls Table . All malSufficient tear film stability as indicated by a normal break-up time (BUT) on both eyes was observed only in three male subjects Table ; 11 of 1Any adverse events (AEs) including serious adverse events (SAEs) that occurred during this study were documented thoroughly. In our cohort of male subjects with XLHED, 85 AEs (thereof 17 SAEs) were recorded, between 0 and 6 per year and individual, most of them XLHED-related infections  and four of them  developed asthma. Furthermore, 22 children suffered from hoarseness of the voice (irrespective of sex). With the exception of two girls, all subjects were afflicted with dry skin, 15 of them also with eczema or atopic dermatitis. Every male but no female subject was reported to have a constantly blocked or running nose. Nosebleeds occurred frequently; 11 subjects had more than ten nosebleeds per year (ranging up to 36 events). In three children with eczematous skin lesions, the EASI scoring system was used and a maximum score of 10.1 was observed. Body plethysmography of 22 subjects and evaluation of their lung function parameters indicated obstructive airway conditions in seven boys with XLHED but in none of the female subjects.So far only one large collection of ED-related clinical issues has been published, the web-based Ectodermal Dysplasia International Registry (EDIR). It summarizes patient-reported medical data from 141 male and 82 female patients of different age groups . Self-assessment revealed life-long relevant health problems, not restricted to the typical symptom triad but also regarding growth, skin, nails, respiratory, and ocular issues . The natEDA mutations [EDA variant. Skewed X-chromosome inactivation is likely to explain phenotypic differences between females with similar EDA variants and has been discussed in a few case reports [As expected, our female subjects with XLHED showed milder phenotypes than affected boys, but none of them was asymptomatic. Although females with XLHED used to be considered only as carriers in numerous publications, it has become well known that most of them are actually affected by XLHED-related symptoms like hypotrichosis, hypodontia, and conically shaped teeth (carrier detection is possible in at least 70% of cases) . Additioutations . Therefoutations , 20. The reports –33.Although this natural history study was completed in 2018, medical care of the patients at our center will continue, enabling further data collection until adulthood. Due to the sparse availability of data for rare diseases like XLHED, those studies may not only be valuable for genotype-based prediction of the course of disease and attempts to prevent complications, but also serve as comparators allowing the evaluation of future drug therapies , 34–36. This first comprehensive natural history study characterized the course of XLHED in male and female patients during the first five years of life, confirming the early involvement of multiple organs and pointing to the need of early therapeutic intervention. The data emphasize that null mutations in EDA consistently result in anhidrosis and severe heat intolerance, while the phenotypes of individuals who carry such mutations heterozygously show a remarkable variability. Although female subjects with one normal X chromosome are often less severely affected, they are not asymptomatic and should receive more attention by researchers in the field. Besides the classic symptom triad of XLHED, patients usually suffer from a number of additional clinical issues, such as skin, eye and respiratory problems that may also be amenable to early treatment.Additional file 1. Pulmonary Function Test (selected parameters). FVC (L), FEV1 (L) and eNO level (ppb).Additional file 2. ED-related medical history. Reported abnormalities.


Stem cells are multipotent cells that maintain the skin epidermis including skin appendages such as hair follicle, sebaceous glands, and sweat glands. There is evidence that reciprocal signalling between the epidermis and the dermis plays an important role in skin development, homeostasis, wound repair, and skin cancer. The origin of skin cancer that derive from skin appendages is still controversial, including basal cell carcinoma and even more of rare tumours such as sebaceous carcinomas and whether those tumours originate from resident tissue stem cells. To investigate whether markers reported to label dermal progenitor cells are preserved in the tumour including the tumour stroma of skin adnexal tumours, we tested 45 human basal cell carcinomas, including superficial, nodular, adenoid, infiltrating, and sclerosing types, and further 38 human tumours of skin appendages including 13 sebaceous adenomas and carcinomas, 20 eccrine sweat gland tumours, and 5 pilomatricomas, syringomas, and hair follicle tumours for the expression of the potential dermal and epidermal cell markers CRABP1, Nestin, and Ephrin B2 and compared these findings with healthy, age-related human epidermis. We detected that CRABP1, Nestin, and Ephrin B2 are expressed in the intratumoural stroma as well as the tumour invasive front of skin tumours of appendages and BCCs. The skin is the outermost layer of the human body, and it protects from physical or biological harm. It is a multilayer epithelium, which contains the interfollicular epidermis and adnexal structures such as the hair follicle, sebaceous glands, or sweat glands . The haiβ-catenin was expressed in sebaceous gland tumours, and CRABP1 as part of retinoic acid signalling enhanced malignancy of human mesenchymal cells [β1-induced myofibroblast activation [Lineage-tracing experiments have identified that although retinoic acid (RA) signalling is essential for epidermal differentiation, the RA-binding protein CRABP1 is dynamically expressed in the embryonic dermis as well as in the stroma of skin tumours  and playal cells  and invaal cells , 9. Nestal cells . It is ral cells . In the al cells , 12. In al cells . Erythroal cells . Eph recal cells , 15. Mesal cells –20. Ephrtivation . Furthertivation . In thisIn this study, we addressed the question whether different regulators of the dermal compartment such as CRABP1, Nestin, and Ephrin B2 are markers preserved directly in the surrounding tumour stroma of skin appendages tumours. In the last years, experiments have shown that cells are capable of exhibiting plasticity and changing fate through dedifferentiation and transdifferentiation upon reciprocal signalling between the epidermis and the dermis. Dermal responses to skin appendage tumorigenesis reflect a combination of changes leading to different microenvironments that might be driven by multilineage progenitor cells of the mesenchymal compartment. To ensure that these cells are part of the compartment of the cancer-associated stroma, we performed Herovici's collagen stain to differentiate between young newly generated collagen of the tumour stroma and mature collagen that represent the unaffected dermal compartment. We also tried to address the question whether they are associated with an either well or poorly differentiated skin tumour state. We have shown before, that within cells of the most frequent nonmelanoma skin cancer basal cell carcinoma, and other adnexal benign or malignant proliferations such as sebaceous adenoma or carcinoma and eccrine sweat gland tumours such as sebaceous carcinomas and porocarcinomas, stem cell markers such as Lrig1-, Lgr5-, Sox9 are preserved.In order to test whether several markers reported to be important in the dermal and epidermal embryonic and postnatal development, such as CRAPB1, Nestin, Ephrin B2C (C-terminus marker), and B2N (N-terminus marker), are conserved in various human tumours of skin appendages, the number of cells, amount of expression, and localization of these markers were analysed in 45 human basal cell carcinomas , 13 sebaceous adenomas and carcinomas, 20 eccrine sweat gland tumours, and 5 pilomatricomas.We used the sections from paraffin-embedded material of skin tumours that were taken with informed consent and stored at the DermPath Laboratory of the Department of Dermatology, Otto-von-Guericke University Magdeburg, and used for staining in accordance with the recommendations of the IRB and GMC. Sections were deparaffinised, and antigen retrieval was used with either citrate buffer pH 6 (Nestin) or EDTA pH 9 . Primary antibodies Nestin , CRAPB1 , Ephrin B2-N-terminus corresponding to amino acids 118-133 of Human Ephrin B2 , 1 : 50), and B2-C-terminus corresponding to amino acids 1036-1051 of Human Ephrin B2  were incubated for 3 h, and Zymed histostain plus kit using AEC  was utilized for detection and counterstained with hematoxylin. In addition to differentiating between the tumour and the stroma, we performed Herovici's collagen stain  to identify young newly generated collagen of the tumour stroma and mature collagen that represent the unaffected dermal compartment.Skin sections were evaluated by 2 independent dermatohistopathologists for the markers tested. First, normal epidermis and hair follicles or skin appendages were evaluated, and expression in different anatomical important areas was noticed . EvaluatIn unaffected skin, CRABP1 staining was detected at the epidermal-dermal interface and the papillary dermis  where thIn unaffected skin of the scalp, Nestin-positive cells are located at the outer root sheath cells of the hair follicle , Table 1All tumours except pilomatricoma were positive for the expression of Ephrin B2 Figures . Ephrin In addition to the long debate on the origin of tumours that are derived from the skin appendages of basal cell carcinoma, the most frequent one, in particular, we have previously reported that Lgr5, Lrig1, and cytokeratin 15 were expressed in all skin appendage tumours tested, and expression of Lgr5 and Lrig1 was generally lower expressed in more aggressive tumour types such as sclerosing basal cell carcinoma and late porocarcinoma stage than in less aggressive superficial or nodular basal cell carcinoma or early porocarcinoma and sebaceous gland tumours . In thisIn previous studies, CRABP1 has been detected in murine papillomas and SCCs within the tumour stroma and at the junctions between the epidermal Keratin 14-positive cells and the dermis . FurtherIn this study, we have tested for the first time the potential markers of dermal progenitor cells in a wide range of skin appendage tumours, including BCCs, as the most frequent skin cancer and sebaceous carcinomas, porocarcinomas, and pilomatricomas.In addition to the usual analysis of immunohistochemistry staining results, we used an analysis method that has been established in the analysis and prognostic evaluation of gastrointestinal tumours. It incorporates the differences in the distribution of cells, whether the cells are allocated to the invasive margin (IM) of the tumour, the inflammatory infiltrate surrounding the tumour (IIF), or the tumour-stroma interaction (TSI) site within the core of the tumour.CRABP1 was expressed at the IM in early formed superficial BCCs indicating a role in the formation of BCCs. It was also expressed in the TSI and IIF of more aggressive BCC including sclerosing and infiltrative types indicating its role in the progression and invasion of the tumour. Since we used Herovici's collagen stain to allocate CRABP1-positive cells in the tumour stroma and the preexisting dermis, we could verify that CRABP1-positive cells are located in the tumour stroma and located in the environment of tumour cells. In sebaceous tumours, CRABP1 was expressed in adenomas more than carcinomas and in early well-differentiated porocarcinoma more than in more aggressive and progressed porocarcinoma indicating a role in the formation and the transition of these tumours rather than in progression and invasion.Nestin was expressed in the tumour stroma of all skin adnexal tumours. It was higher expressed within the stroma of low-grade tumours such as superficial BCC or sebaceous adenoma or poroma and less expressed in more progressed skin tumours such as sclerosing or infiltrative BCC, sebaceous carcinoma, or progressive porocarcinoma. Interestingly, Nestin main expression was observed in the stroma interacting with tumour cells (TSI) and the invasive margin (IM), indicating an active role in reciprocal signalling between the tumour and the stroma cells. In contrast, it was low expressed within the immune infiltrate surrounding the tumour indicating its role in the tumour stroma.We further observed that Ephrin B2, a marker known to be upregulated not only in the mesenchymal compartment but also in SCCs, was expressed in the tumour stroma and within skin appendage tumours in more advanced BCCs (infiltrative BCCs and ulcus terebrans), sebaceous carcinomas, and early porocarcinomas.In conclusion, we could detect several markers that have been reported as progenitor cells in the dermal compartment within the stroma of tumours of skin appendages indicating that they have a role in the formation of these tumours.


Critical program elements to improving voluntary contraceptive use among married youth included: (1) use of a socioecological intervention model of behavior change; (2) engaging both women and men; and (3) calibrating interventions to different moments in the life cycle of adolescents and youth. Trade-offs between intensive NGO-led models and less intensive government-led models occurred in effectiveness, scale of interventions, and sustained behavior changes. Critical program elements to improving voluntary contraceptive use among married youth included: (1) use of a socioecological intervention model of behavior change; (2) engaging both women and men; and (3) calibrating interventions to different moments in the life cycle of adolescents and youth. Trade-offs between intensive NGO-led models and less intensive government-led models occurred in effectiveness, scale of interventions, and sustained behavior changes. Evidence on effective behavior change programming for sexual and reproductive health among married youth aged 15–24 in developing countries is lacking. To address this gap, we examined monitoring, evaluation, and special study data from the Promoting Change in Reproductive Behavior of Adolescents (PRACHAR) Project, which was implemented between 2001 and 2012 in Bihar, India, over 3 phases using 2 different implementation models (NGO- and government-led).We conducted a synthesis of evidence from multiple PRACHAR studies to identify key findings on intervention effectiveness, scalability, and sustained effects on behaviors. Data were triangulated from quantitative population-based quasi-experimental evaluations from each project phase; the project's performance monitoring database; and qualitative studies with beneficiaries.PRACHAR's original comprehensive NGO-led model, which included behavior change elements and multiple overlapping communication channels , increased contraceptive use among young married couples, and these outcomes were sustained 4–8 years after project interventions ended. Several program elements were critical to the effectiveness of PRACHAR, including use of a socioecological intervention model with emphasis on behavior change efforts; use of a gender-synchronized approach that engages both male and female partners; and intensity of interventions calibrated to different moments in the life cycle of adolescents and youth. While the hybrid government-NGO model of PRACHAR implementation reached greater scale than the original NGO-led model, comparison of results suggests trade-offs in effectiveness of interventions and sustained impacts.A decade of learning from the PRACHAR Project in Bihar, India, finds that comprehensive programming with gender-synchronized interventions tailored to specific life stages and aimed at different levels of the socioecological model can effectively increase contraceptive use among married young people in a conservative context. Shifting from a more intensive NGO-led model to less intensive government implementation enhances scalability but may have diminished impact on reach and long-lasting effectiveness. At the time PRACHAR began, Bihar had few programs to address the contraceptive needs of the population, including adolescents and youth. The modern contraceptive prevalence rate was low (22%) for all women of reproductive age, with almost no contraceptive use among married adolescents aged 15–19 (1%) and young women aged 20–24 (5%).Supplement for a detailed description of how PRACHAR activities were implemented.)PRACHAR was designed using a life-stage tailored social and behavior change approach, based on the socioecological framework.PRACHAR was implemented in 3 phases with different coverage levels, intervention combinations, and durations . Phase IThe PRACHAR Project was implemented in 3 phases between 2001 and 2012 with different coverage levels, intervention combinations, and durations.––From 2014 to 15, Pathfinder International staff conducted a review of existing PRACHAR Phases I, II, and III evaluation reports, special studies, presentations, and project monitoring data to synthesize evidence related to the following research question: “What is the evidence from PRACHAR around programming to increase contraceptive use among young married women and men?”. The studies we reviewed are briefly described here, and their methodologies are described in the source files referenced. First, we reviewed reports of quasi-experimental studies conducted for each project phase, i.e. population-based surveys among young married women and men in intervention and control areas at baseline and endline.Following the review of existing PRACHAR evaluation and monitoring reports, we identified key gaps in knowledge that would be important to address for program design purposes and conducted secondary analyses of PRACHAR evaluation data to answer specific questions around effectiveness, intensity, and duration of program interventions. We generated bivariate frequency distributions and conducted multivariate logistic regressions with contraceptive use as the outcome variable and a variety of independent variables .The findings from our review and the secondary analysis were categorized according to key program learning themes as described in the Results below.Through our review of the PRACHAR evaluation studies, special studies, and our secondary analysis, we identified evidence around 4 key themes: (1) project effectiveness in achieving attitudinal and behavioral outcomes; (2) effectiveness of selected program components and the intensity required to produce effects; (3) scalability and effectiveness at scale; and (4) sustained project impact.P<.001, adjusted for age, education, caste, and parity) ,P<.001, adjusted for age, education, and caste).The results of our analyses indicate that comprehensive intervention models of longer duration are most effective in increasing contraceptive use among married youth. The 3-year comprehensive NGO-led PRACHAR model in Phase I, which included behavior change elements and multiple overlapping communication channels, had the greatest magnitude of effect on contraceptive use. As published in Daniel et al. (2008), the odds of current contraceptive use increased nearly 4 times as much from baseline to endline among young married women in Phase I intervention areas than among those in comparison areas  .15 The aComprehensive intervention models of longer duration are most effective in increasing contraceptive use among married youth.–Bivariate findings on current contraceptive prevalence rate from all 3 PRACHAR Phases also show the greatest baseline-endline increase in the 3-year Phase I model, from 4% to 21% in intervention areas .15–17 WhContraceptive method mix remained relatively consistent in Phases I and II, reflecting the limited availability of several methods in Bihar at that time, particularly for young married women. Condoms and pills were the most commonly used contraceptive methods among young married women aged 15–24 of zero and single parity in the intervention and comparison areas. Condoms ranged from 62% to 85% of the method mix, and pills from 11% to 27% of the method mix. Use of IUDs was negligible (less than 1%), and there was virtually no use of female or male sterilization as expected with a young population . In PhasP<.001, adjusted for education, caste, and standard of living index). Among the Phase II “single-intervention” models, the home visit model (plus service linkages and community-level interventions) had the highest magnitude of effect for current contraceptive use .P<.001, adjusted for education, caste, and standard of living index).Phase I evidence also sheds light on the relative contributions of specific PRACHAR interventions, as well as the intervention timing and intensity required to influence contraceptive use. Home visits by NGO change agents conducting interpersonal communication were found to be effective in increasing contraceptive use among young married women when implemented in tandem with community-level activities that aimed to change attitudes and behaviors. In the Phase I comprehensive model, young married women in PRACHAR intervention areas who were exposed to home visits had 2 times higher odds of currently using contraception than those who did not receive home visits  had the highest ever use of contraception after their first birth and initiated contraception more quickly compared with women reached at fewer life cycle stages. In addition, a relationship was observed between the number of home visits and ever use of contraception among young married women, with 7 to 12 visits as the “tipping point” where more than half of contraceptive users had initiated use . While mP<.001).We also found that contraceptive initiation earlier in life is correlated with future contraceptive use. Results from the Adolescent Follow-up Survey show that married young women with 1 or more children (in both intervention and comparison areas) who had used contraception before their first birth had nearly 14 times higher odds of using contraception after their first birth compared with women who had not previously used contraception , whereas couples in which only the woman was exposed to PRACHAR had lower odds of contraceptive use , and there was no significant effect on contraceptive use for couples in which only the husband was exposed   . SimilarThe PRACHAR data do not shed light on the effectiveness of the other PRACHAR intervention components. No significant associations were found between exposure to newlywed ceremonies or cultural programs and current use of contraception by young married women. The added impact of engaging mothers-in-law and other key gatekeepers could not be determined, as young married respondents were not asked if other family members besides their partner  participated in PRACHAR activities, and gatekeeper and community engagement to shift attitudes occurred across all Phase I and II intervention arms.P<.01, adjusted for age, education, and caste). Effect sizes varied by parity: aOR=1.89 among women with 1 child (P<.05), aOR=1.67 among women with more than 2 children , and no significant effect among women with zero or 2 children. The smaller effect sizes in Phase III may reflect several implementation factors. First, women in comparison and intervention areas had comparable rates of exposure to ASHAs . Second, among women contacted by ASHAs, only 28% of women in intervention areas and 17% of women in comparison areas reported that the ASHA discussed family planning.  Third, ASHAs had lower coverage rates for zero-parity women, which was a key target population for PRACHAR. Only 43% of zero-parity women were reached at least once in both intervention and comparison groups, compared with 78% to 86% of women of parity 1 or higher. Only 44% of zero-parity women visited by ASHAs received 13 or more visits in the past 3 years, compared with 61% to 74% of women of parity 1 or higher. Finally, Phase III also lacked significant behavior change interventions with gatekeepers such as mothers-in-law and other community influencers.As per an internal evaluation report by the Population Council, the scalable PRACHAR Phase III hybrid government-NGO model (using ASHAs instead of change agents and reducing or eliminating other activities) had significant but smaller contraceptive use gains than Phases I and II.The hybrid government-NGO model, while scalable, had significant but smaller contraceptive use gains than the NGO-led model.P<.001) (P<.05), with 4.1% of women with zero children and 5.9% of women with 1 child initiating contraception in this time frame . Married women with 1 child in former Phase I areas had 3 times higher odds of initiating contraceptive use within 3 months of their first birth (aOR=3.13), with 10.2% of women with 1 child initiating contraception during this time frame (vs. 2.9% in comparison areas). Married women with 2 children in former PRACHAR Phase II areas had 61% higher odds of initiating contraception within 3 months of their first birth (aOR=1.61) than women in comparison areas (both P<.05), with 15.1% of women with 2 children initiating contraception during this time frame (vs. 9.8% in comparison areas).As published in Jejeebhoy et. al (2015), the current contraceptive use gains achieved in PRACHAR Phases I and II persisted several years after the interventions ended, both among those directly exposed to PRACHAR as well as those living in the intervention areas at the time of the survey but not directly exposed to PRACHAR.P<.001) . BivariaP<.001) . SustainThe contraceptive use gains achieved under the NGO-led PRACHAR model persisted several years after the interventions ended.P<.001).In addition to the contraceptive behavior changes sustained after the intervention ended, there is some evidence that PRACHAR led to sustained attitudinal shifts around healthy timing and spacing of pregnancies. Women in former PRACHAR intervention areas had significantly greater odds of preferring an ideal age at first birth of 21 or older (aOR=1.60) and preferring a birth interval of at least 36 months (aOR=1.46) than women in comparison areas  relative to the effect size of Phase I (the full comprehensive model) raises important considerations for program design and scale up.First, the shift from NGO change agent (in Phases I and II) to ASHA (in Phase III) enhanced scalability. ASHAs are a scalable platform through which interventions can potentially reach many people and be sustained through government funding, whereas local NGOs have limited geographic reach and depend on external funding sources. However, the shift to ASHAs resulted in reduced quality and reach of interpersonal communication during home visits under Phase III. There are several possible reasons for this. Like community health workers in many other countries, ASHAs are responsible for promoting a wide range of maternal and child health care practices and are regularly tasked with additional priorities. During PRACHAR implementation, incentivization schemes that encouraged ASHAs to seek out pregnant women and refer them for institutional delivery diverted ASHAs from reaching young nulliparous women and inadvertently disincentivized discussion of family planning. Some states in India have explored incentivization of community health workers for helping couples to achieve outcomes related to delaying and spacing births, but ASHAs—like community health workers worldwide—remain a cadre with many responsibilities and competing incentives covering the gamut of maternal and child health. The PRACHAR experience under Phase III demonstrates the challenges of relying on government-supported multipurpose community health workers to conduct intensive behavior change focused on young married women, amidst their other competing priorities.,In addition, PRACHAR further demonstrates that shifting the main intervention to government implementation resulted in less funding and diminished prioritization of the other elements of the intervention, including reduced engagement of male partners and key gatekeepers. Considering the quantitative and qualitative evidence from Phases I and II of PRACHAR that points to the importance of the engagement of male partners and the broader community, it is possible that the reduced engagement of men and limited community social norm change interventions under Phase III contributed to the reduced magnitude of impact. This finding aligns with behavior change theory and reinforces existing evidence from a range of contexts that suggests that the use of a socioecological framework with multiple reinforcing interventions at different levels can change behavior related to contraceptive use among young married couples.The PRACHAR experience suggests that there are trade-offs for behavior change approaches when moving from a more intensive, NGO-implemented approach to an approach that may be more easily scaled but relies on overburdened government workers or systems. These trade-offs are not unique to PRACHAR and are likely to be found when seeking to scale up behavior change programs across a range of target groups and outcomes.The PRACHAR experience also raises important questions on potential tensions between less intensive implementation approaches that can reach more people in the context of constrained funding but may have diminished impact on intractable behaviors, and intervention models that may be more intensive but catalyze change in behaviors with long-lasting and intergenerational effects. The PRACHAR long-term studies indicate that the more intensive interventions of PRACHAR Phases I and II had sustained impacts for 4 to 8 years after the program ended. While the PRACHAR studies did not include normative measures, the sustained behavior change and corresponding attitudinal changes regarding healthy timing and spacing of pregnancies suggest that norms surrounding use of contraception and fertility among young married couples may also have shifted due to PRACHAR. While direct comparison of the sustained impact of Phases I and II with Phase III is not possible (because no long-term study has been done following Phase III), it is important to consider whether the intensity and/or quality of Phase I and II interventions might have contributed to the sustained effects observed. Would we see the same sustained effects from the less intensive ASHA-led intervention with limited community engagement, especially given that the immediate effects on contraceptive use at endline were modest (and negligible among zero-parity women)? We hypothesize that we would not. This begs the question of whether a more intensive model that creates sustained impacts after 3 years of implementation may offer better value for money than a less intensive model that reaches more people but must continue for many more years to create long-lasting effects.PRACHAR represents a decade of investment in implementation, learning, and evaluation of social and behavior change efforts to increase voluntary contraceptive use for married adolescents and youth. It is essential that we use the learning from PRACHAR to inform the design and implementation of current and future programs for married youth. In addition, to advance the field and better meet the needs and rights of married adolescents and youth, future programs should seek to answer the questions that PRACHAR raises in relation to effective models to scale up multilevel behavior change efforts and around potential trade-offs between intensity, quality, and sustainability of impacts over time. Answering these questions will require implementation of thoughtfully designed programs for married youth that draw on the learnings from PRACHAR and improve upon the PRACHAR model. In addition, it will require robust and mixed-method evaluations and thoughtful implementation science approaches that assess not only effectiveness but also how programs are implemented, why programs are or are not impactful, and what the scale-up pathways and processes are. Furthermore, as the use of social norm measurement and theory advances in global public health, programs for married youth will benefit from more theory-driven normative change interventions and measurement.