This article presents the structure and function of the Health Equity Collective in developing a systemic approach to promoting health equity across the Greater Houston area. Grounded in Kania and Kramer's five phases of collective impact for coalition building, The Collective operationalizes its mission through its backbone team, steering committees, and eight workgroups; each has goals that mutually reinforce and advance its vision. To date, Phase I (generating ideas), Phase II (initiating action), and Phase III (organizing for impact) have been completed. Phases IV (implementation) and Phase V (sustainability) are currently underway. This article presents the Collective's structure and progress in this space.18 To circumvent these potential challenges, the Collective adopted Kania and Kramer's conditions of common agenda, shared measurement systems, mutually reinforcing activities, continuous communication, and backbone support, which are currently cemented across five phases: (I) generating ideas and host dialogues, (II) initiating action, (III) organizing for impact, (IV) beginning implementation, and (V) sustaining action and impact.17According to Schuler and Koka, coalitions must first address “problem ambiguity, user dimensionality, structural barriers, and a mindset of competition among social service organizations” as common barriers to effectively tackle societal issues.In Phase I, coleads from the University of Texas Health Science Center at Houston, School of Public Health (UTSPH), American Heart Association, and Harris County Public Health conducted over 200 one-on-one listening meetings with prospective Collective members and community stakeholders from December 2018 to December 2019. An SDoH screening and data-sharing landscape scan between Greater Houston health care and community-based organizations (CBOs) were also produced to better understand and establish possible data-sharing relationships that would benefit the community.In August 2019, the Collective also convened 85+ key stakeholders to help shape its focus. Deliberations revealed care coordination and data-sharing capacity across the health and social services sectors as the approaches best suited to address SDoH needs across the Greater Houston. During this period, our mission to achieve health equity was also cemented with unanimous consent. A second convening in February 2020 with executive leadership from 25 Greater Houston organizations resulted in recommendations for an information exchange ecosystem starting first with the development of a Community Information Exchange (CIE), which would allow for referral and care coordination between social service providers in the Greater Houston area.Backbone coleads also reached out to local and national philanthropic entities to develop a sustainable funding strategy for the Coalition. Activities conducted in Phase I not only solidified consensus on the Coalition's mission, goals, structure, and strategic direction, but also laid the groundwork for Phases II and III of collective impact action.19Central to Phases II and III was the formation of the Collective's structure and functions . The thrDuring Phases I and II, membership grew from 100 members and 75 organizations to over 375 members representing 130 organizations sectors across the Greater Houston area.(1)Developing and implementing a resource data infrastructure to convene disparate sources of data to address SDoH needs.(2)Developing and implementing the referral network infrastructure to support referrals between social service organizations and to track the impact of referrals on clients' social needs and behavioral outcomes.(3)Developing a vendor agnostic infrastructure to link the CIE and health care organizations to facilitate care coordination. The Collective workgroups and its steering committee will work in tandem to support CIE efforts. As immediate next steps, the Collective will also initiate a governance process for the CIE as well as facilitate its 12-month proof-of-concept. As we enter Phase V of collective impact in 2021, the principal focus will be to launch the CIE proof-of-concept and related governance elements.The Collective is now in its Phase IV (2021–2022) of the collective impact approach, and has initiated the development of cloud-based CIE. Using a federated model approach, the CIE will allow multiple organizations with varied technologies to connect, coordinate care, and serve their populations' social needs. Once established, the CIE will connect with the health care sector and will allow for bidirectional referrals and care coordination with health care and behavioral health entities. The CIE architecture will include the following:20 and a recognized focus for many health, community, institutional, and business entities across the Greater Houston area.15 Achieving health equity demands an accelerated pivot toward improved systems interoperability and a clear focus on upstream prevention and integration of health and social services. These transformational shifts have been operationalized by the newly formed Health Equity Collective—a coalition that has convened cross-sector organizational support to address and promote population well-being.22Addressing key SDoH indicators is rightfully a global health priorityFormed in 2018, the Collective has prioritized the development and implementation of a sustainable, data-driven ecosystem of care to achieve health equity. This article adds to the current body of literature on SDoH by describing the structure and functions of this coalition's work across the Greater Houston area and details our collective impact-driven framework to address our mission and future work in this area.17 The success of the collective impact approach has been mirrored in other existing U.S. coalitions; their successes have informed the Collective's work and our roadmap for looking ahead.The collective impact approach to coalition-building posits that organizations from diverse sectors must work together around a shared vision to make lasting social change. Working toward this shared vision is greatly enhanced because of common measurement approaches, activities that reinforce the Collective's agenda, ongoing communication, and the continuous support of a backbone team, keeping the participating institutions grounded in a shared vision.24An example of a multipronged collective impact initiative is “Memphis Fast Forward,” which was established in 2008 to launch “Operation: Safe Community.”24 Similarly, our coalition has learned to adapt and integrate an iterative process of learning and action, which has collectively moved us closer to satisfying goals that meet our shared agenda.The coalition made significant headway on its prosecution and policing metrics, but struggled to achieve progress on the third metric: violence prevention. To recalibrate, coalition members worked in unison to identify and apply for funding mechanisms and were successfully awarded a grant by the U.S. Department of Justice.25Another collective impact initiative is the “Communities that Care” Coalition, launched in 2003 by Franklin County in Massachusetts to address teen substance abuse. The shared vision was to improve parents' behaviors and attitudes on teen substance abuse through parental training and “parents teach other parents” sessions. To inform coalition efforts, members engaged in various data collection and sharing activities  that summarily found family dinner messaging to be the best approach for achieving the coalition's goal.25 Efforts not only demonstrated a willingness to be quick-thinking and flexible but also kept the coalition on track with its overall goal, which remained unchanged.Armed with this information, the Coalition retooled its messaging strategies to improve outreach efforts.Indeed, insights from these coalitions have provided directions for navigating challenges—many of which we have since turned into opportunities for growth and innovation. Through these and our own experiences, we found that convening institutions—that had their own priorities and way of doing things—around a shared vision to be occasionally difficult, yet manageable. For example, grant proposals and technology investments that were pursued as a coalition had better funding and sustainability prospects versus organizations that acted independently and within silos. Ongoing participation across backbone, steering committee, workgroup, and quarterly sessions greatly reinforced the importance of interoperability and “buy in” to the Collective’ shared agenda.Second, an inherent challenge to building and maintaining a healthy coalition is the demand for committed leadership, effort beyond primary job responsibilities, and time investment. We experienced these “organizational capacity challenges” as a coalition. However, in our case, we found that an evolving and growing coalition of new members at the executive level or within workgroups brought a much-needed infusion of support, enthusiasm, expertise, and new ideas that pushed the vision of the Collective forward.In particular, the work by the Food Security workgroup and Framework and Common Metrics workgroup members has proven transformational in advancing the Collective's health equity vision. As examples, the Food Security workgroup's conduct of (1) a landscape scan of food security and healthy food intake interventions/programs and (2) a comprehensive qualitative assessment of organizations' capacity to address food security  will inform best practices and understandings for how organizations and the Collective can better support food security efforts across the county.Similarly, the Frameworks and Common Metrics workgroup has worked diligently to develop a framework that identifies four main disciplines of health action that will influence identified “drivers of health” in the county. This model serves as a railroad for how the Collective will operationalize its vision and is quintessential for keeping the Collective grounded in its mission. These successful workgroup efforts and ongoing equally vital work by other workgroups have been supported by continuous communication, support, and iterative feedback from backbone coleads, steering committee leadership, and our executive ambassadorship team.26 the Health Equity Collective is uniquely positioned to leverage its partnerships to create standardized SDoH screening tools for various needs  to facilitate both screening efficiency and care coordination efforts across health and social care sectors.Lastly, through listening sessions with CBOs, qualitative assessments (Food Security workgroup), and ongoing networking and strategy sessions, we also realized that improvements in interoperability in screening protocols as in technology platforms are equally critical. Like other coalitions,In conclusion, as the Collective expands, the work of diverse-sector coalition members will continue to influence how we implement an integrative framework for SDoH in the Greater Houston area. Looking ahead, this work could prove transformational in serving as a blueprint for how diverse sectors can work together to leverage their health and social services in tackling upstream QOL predictors.


The goal of newborn screening is to improve health outcomes by identifying and treating affected newborns. This manuscript provides an overview of a data tool to facilitate the longitudinal collection of health information on newborns diagnosed with a condition through NBS. The Newborn Screening Translational Research Network (NBSTRN) developed the Longitudinal Pediatric Data Resource (LPDR) to capture, store, analyze, visualize, and share genomic and phenotypic data over the lifespan of NBS identified newborns to facilitate understanding of genetic disease and to assess the impact of early identification and treatment. NBSTRN developed a consensus-based process using clinical care experts to create, maintain, and evolve question and answer sets organized into common data elements (CDEs). The LPDR contains 24,172 core and disease specific CDEs for 118 rare genetic diseases, and the CDEs are being made available through the NIH CDE Repository. The number of CDEs for each condition average of 2200 with a range from 69 to 7944. The LPDR is used by state NBS programs, clinical researchers, and community-based organizations. Case level, de-identified data sets are available for secondary research and data mining. The development of the LPDR for longitudinal data gathering, sharing, and analysis supports research and facilitates the translation of discoveries into clinical practice. Each of the approximately 4 million babies born in the United States each year receives neonatal screening for over thirty-six rare genetic diseases . NewbornNewborn screening is a system of interconnected activities that begin before a baby is born. This multi-stakeholder system’s key components include prenatal education, neonatal screening, short-term follow-up and diagnosis, and clinical care and management. Each year at least 12,905 newborns are diagnosed with a condition through NBS, and the majority enter life-long care management . HoweverThe phrase “newborn screening long-term follow-up” applies both to the longitudinal treatment and management of diagnosed newborns as well as the longitudinal data collection of health information to inform assessments of outcomes. The Newborn Screening Saves Lives Reauthorization Act addressed LTFU as “follow-up activities, including those necessary to achieve rapid diagnosis in the short-term, and those that ascertain long-term management outcomes and appropriate access to related services.” . Both acThe Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) Hunter Kelly Newborn Screening Research Program was created to support investigations and newborn screening innovations . Recent NBS in the US is a multi-component, multi-stakeholder system of prenatal education, hospital and state-based public health laboratory screening, clinician and state-based laboratory confirmation and diagnosis, clinical treatment and management, and health outcome analysis. The NBSTRN data tools, resources, and expertise are designed to facilitate all stakeholders’ efforts and leverages each component of NBS to advance research. As a translational research network, NBSTRN was conceptualized to develop and share resources and infrastructure to support NBS researchers. The collective research infrastructure provides mechanisms for an unbiased understanding of rare genetic conditions across the lifespan, population-based pilot testing of new NBS tests, new technology development and application, and new therapeutics development. This research infrastructure also strengthens and enables the expansion of NBS programs in a more systematic fashion. ACMG also operates the National Coordinating Center (NCC) for the Regional Genetics Networks (RGNs) . In 2015The goal was to create a catalog of CDEs to enable investigators and public health teams to systematically collect, analyze, and share data across the NBS community. The development of the CDEs was guided by ACHDNC’s publications and Follow-Up and Treatment Subcommittee discussions . The CDENBSTRN was charged with creating a data tool using the resulting consensus based, standardized CDEs, organizing the CDEs into case report forms, and making the forms available through a secure online portal facilitating data collection and management. NBSTRN also worked with subject matter experts to develop CDE sets for conditions that are candidates for newborn screening. CDE dissemination was accomplished through a collaboration with the National Institutes of Health (NIH) National Library of Medicine (NLM) and NBSTRN via the NIH CDE Repository (Repository) . CurrentThe neonatal screening of newborns each year leads to the diagnosis of infants with a genetic condition that requires referral to clinical care and, in most cases, lifelong management. This unselected cohort of newborns reflects our nation’s racial, geographic, economic, and educational diversity. This may be the perfect cohort to help advance disease understanding because although every newborn receives essentially the same screen, other factors vary, including treatment choice and disease course. Also, many of the screened conditions have comorbidities, including intellectual disabilities. These children receive various interventions that could be tracked and analyzed to identify critical periods of development and intervention. Because the NBS system in the United States effectively screens over 99% of newborns, it can provide a unique platform for understanding rare diseases and lifelong outcomes. The process of neonatal screening followed by a coordinated transition to clinical care facilitates collecting health information beginning just hours after birth. And because the majority of NBS conditions require life-long care and management, we have the opportunity to conduct prospective, longitudinal natural history studies on a population basis with unbiased ascertainment. https://cde.nlm.nih.gov, accessed on 12 April 2021). The LPDR provides data dictionaries to create electronic data entry forms and features de-identified, case-level data sets for data mining. The secondary use of accrued LPDR data may help to (1) establish the efficacy of new treatments and management approaches, (2) inform the community about the value of early identification and treatment through newborn screening, and (3) identify areas for improvement in disease management throughout the lifespan.Over the last decade, NBSTRN has developed a data tool, the Longitudinal Pediatric Data Resource (LPDR), to support several landmark natural history studies that have contributed to a better understanding of the etiology, pathophysiology, and phenotypic heterogeneity of NBS conditions and provided an assessment of health outcomes . The LPDThe LPDR is accessible through the NBSTRN website at nbstrn.org and operates in a cloud environment that utilizes the NIH Science and Technology Research Infrastructure for Discovery, Experimentation, and Sustainability (STRIDES) Initiative and Amazon Web Services (AWS) . The NBShttps://nbstrn.org/tools/lpdr (accessed on 12 April 2021). CDE sets are organized into REDCap formatted data dictionaries and text files. The LPDR enables users to adopt the consensus based CDEs, generate case report forms, and suggest additional CDEs for new conditions, technologies, and treatments.A total of 24,172 core and disease specific CDEs have been developed for use in longitudinal health information data collection efforts for 118 conditions that are part of, or candidates for, nationwide NBS. The joint committee created 15,685 CDEs for 46 RUSP conditions. The NBSTRN developed 8487 additional disease specific CDEs for 72 diseases for use by researchers conducting natural history studies or pilots of conditions recently added to, or candidates for, the RUSP . The CDESince its launch in 2013, the LPDR has been utilized by several research teams, either conducting longitudinal studies of both RUSP and candidate conditions, exploring the use of genome sequencing in the newborn period, and executing NBS pilots of candidate conditions. The LPDR has 312 registered users, 151 individuals approved to enter or upload data, 23 established longitudinal projects, ~12 Million data points, and 8842 subjects with an average of 3.62 data collection time points per subject. The LPDR has data displays that describe the types of data and populations available for secondary use . Our datLongitudinal care management of newborn screen identified individuals coupled with longitudinal capture of health information are vital cornerstones of a successful NBS system in the United States. The NBSTRN led an effort to develop the LPDR, a suite of tools and resources to facilitate longitudinal data capture using question and answer sets developed by clinical experts. This effort resulted in the identification of key data elements for long-term follow-up (LTFU) efforts conducted by state NBS programs. These data elements could also be used by researchers, advocates, policy makers, and clinicians to evaluate the health outcomes of newborns identified by NBS. The LPDR has been utilized in a variety of efforts, including a ten-year effort to collect, analyze and disseminate health information on individuals with one of 42 RUSP conditions in 30 clinical sites located in 22 states, multi-state pilots of 4 conditions collectively screening over 1.2 M births, use of genome sequencing in four cohorts of newborns including neonatal intensive care unit and healthy, studies expanding the diagnostic window of NBS both beyond and before the neonatal period, and efforts to consolidate disparate patient registries into a single data dictionary that supports pilots. As the data sets contained in the LPDR grow, the NBS community can utilize this information to inform the care and management of newborns identified with a condition through NBS, guide policy and funding decisions, assess the benefit of NBS, and further our understanding of these rare genetic conditions.www.nbstrn.org (accessed on 12 April 2021)  see . By beco


The pathology in neurodegenerative diseases is often accompanied by inflammation. It is well-known that many cells within the central nervous system (CNS) also contribute to ongoing neuroinflammation, which can promote neurodegeneration. Multiple sclerosis (MS) is both an inflammatory and neurodegenerative disease in which there is a complex interplay between resident CNS cells to mediate myelin and axonal damage, and this communication network can vary depending on the subtype and chronicity of disease. Oligodendrocytes, the myelinating cell of the CNS, and their precursors, oligodendrocyte precursor cells (OPCs), are often thought of as the targets of autoimmune pathology during MS and in several animal models of MS; however, there is emerging evidence that OPCs actively contribute to inflammation that directly and indirectly contributes to neurodegeneration. Here we discuss several contributors to MS disease progression starting with lesion pathology and murine models amenable to studying particular aspects of disease. We then review how OPCs themselves can play an active role in promoting neuroinflammation and neurodegeneration, and how other resident CNS cells including microglia, astrocytes, and neurons can impact OPC function. Further, we outline the very complex and pleiotropic role(s) of several inflammatory cytokines and other secreted factors classically described as solely deleterious during MS and its animal models, but in fact, have many neuroprotective functions and promote a return to homeostasis, in part via modulation of OPC function. Finally, since MS affects patients from the onset of disease throughout their lifespan, we discuss the impact of aging on OPC function and CNS recovery. It is becoming clear that OPCs are not simply a bystander during MS progression and uncovering the active roles they play during different stages of disease will help uncover potential new avenues for therapeutic intervention. Multiple sclerosis (MS) is an inflammatory and neurodegenerative disease of the central nervous system (CNS) characterized by loss of myelin, oligodendrocytes, and axons  and passive induction via adoptive transfer of activated, myelin-specific T cells from an immune-primed donor into a naïve recipient without the use of adjuvants , a member of the enveloped Coronaviridae family , a member of the non-enveloped + cells are likewise present in elevated numbers in spinal cord lesions. Interestingly, lesion-associated NG2+ OPCs can co-express GFAP or CNPase suggesting potential alternative routes of OPC differentiation, which may affect remyelination capacity , allowing OPCs to differentiate into myelinating OLs  and aged (>1 year) rats undergoing EtBr-mediated demyelination revealed that OPCs in aged animals were recruited to the lesion site more slowly. Transcript levels of the OPC markers Pdgfra and MyT1 were comparatively retained in aged rats and mature OL markers of Mbp and Gtx were delayed, indicative of impaired differentiation (Sim et al., Age-related remyelination failure has been linked to suppression of OPC differentiation as large pools of undifferentiated OL lineage cells were present in chronically demyelinated MS lesions in post-mortem tissue (Boyd et al., Cdkn2a (Neumann et al., + OPCs resulted in inhibited differentiation suggesting that functional inhibition of OPC recruitment and differentiation may be partially explained by a shift toward a senescent phenotype (Nicaise et al., Mmp3, p53, and p21 in older astrocytes compared to those from younger rats. Similarly, there was a decrease in aged cells undergoing the G2 and S phases of the cell cycle, which was accompanied by an increase in cells associated with the G1 phase, demonstrating inhibited astrocytic proliferation (Willis et al., The intrinsic changes associated with both OPC differentiation and recruitment may be linked to cellular senescence, or the cessation of the cell cycle and a deviation from normal cell function toward an inflammatory state (Gorgoulis et al., in vivo, while also promoting transcript signatures of mature OLs in human OPCs (Ito et al., Astrocytes are crucial for maintaining CNS microenvironments conducive to homeostasis and proper OPC function (Liedtke et al., While it is clear that resident CNS cells contribute to ongoing inflammation during MS, the complexity of the signaling events in various cells types, the interactions between those cells, and the role these processes have in neurodegeneration are only beginning to be understood. This intricate cellular and molecular symphony can also vary widely depending on the subtype of MS, the stage of disease, and the age of the patient. Since remyelination failure is a significant roadblock in the treatment of MS (Franklin, In addition to factors intrinsic to OPC function during neuroinflammation and neurodegeneration in MS, there are a multitude of factors that impact the MS lesion microenvironment that critically regulate OPC migration, proliferation, and maturation ; Table 1JW generated the concept design. MP, BS, RT, and JW wrote and edited this manuscript. MP created the figures. MP, BS, and RT created the table. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Next-generation sequencing (NGS) and decreased costs of genomic testing are changing the paradigm in precision medicine and continue to fuel innovation. Integration of NGS into clinical drug development has the potential to accelerate clinical trial conduct and ultimately will shape the landscape of clinical care by making it easier to identify patients who would benefit from particular therapy(ies) and to monitor treatment outcomes with less invasive tests. This has led to an increased use of NGS service providers by pharmaceutical sponsors: to screen patients for clinical trials eligibility and for patient stratification, expanded Companion Diagnostic (CDx) development for treatment recommendations and Comprehensive Genomic profiling (CGP). These changes are reshaping the face of clinical quality considerations for precision medicine. Although some clinical quality considerations do exist in Health Authorities (HA) guidances and regulations , there is currently no holistic GxP-like detailed framework for pharmaceutical sponsors using NGS service providers in clinical trials, or for the development of CDx and CGP. In this research, we identified existing and applicable regulations, guidelines and recommendations that could be translated into clinical quality considerations related to technology, data quality, patients and oversight. We propose these considerations as a basis for pharmaceutical sponsors using NGS service providers in clinical drug development to develop a set of guidelines for NGS clinical quality.The online version contains supplementary material available at 10.1007/s43441-021-00308-6. Next-generation sequencing (NGS) refers to large-scale, fast and efficient DNA (and RNA) sequencing technology. For DNA sequencing all NGS platforms perform sequencing of millions of small fragments of DNA in parallel  and then rely on bioinformatic analysis against a reference genome to infer useful insights from detected variations. This technology can be utilized in many research areas, from molecular biology to human genetics, but is also used as part of clinical care in various therapeutic areas  –4. NGS aNature Medicine [Establish[ing] good genomic practice to guide medicine forward". The scope was broad, from analytical to clinical validation of NGS, but also included ethical considerations. Furthermore, it provided only high level principles that needed to be further developed and explained. Although some clinical quality considerations do exist in Health Authorities (HA) guidances and regulations , there In this review, we aimed to identify existing and applicable regulations, guidelines and recommendations that could be consolidated into clinical quality considerations for pharmaceutical sponsors using NGS service providers in clinical development. Integration of clinical quality standards for NGS can lead to unified standards for the generation of high quality data, ethical use of data with increased transparency of testing limitations and ultimately better outcomes for patients. Through this review we sought to highlight requirements  that are present, and to provide pragmatic GxP recommendations for further consideration when no requirement was available.clinical quality aspects of data primarily generated by NGS service providers, and direct variables derived from NGS datause of these NGS data in the context of clinical trials use of NGS for development of solutions to guide treatment recommendations, which are subject to regulatory approval, for example CDx and CGPThe scope of this review included:NGS methods and standards, except elements relevant for pharmaceutical sponsors clinical quality oversightClinical quality aspects for secondary use of (genomic) dataApplicable Good Manufacturing Practices (GMP) regulationsInvestigational Device regulationsLocal/national laws or regulations for clinical laboratories )We considered the following to be out of scope:Although out of scope, this review will touch upon relevant ethical topics in using genomic data in clinical development, but will not provide detailed considerations. NGS use by pharmaceutical sponsors is not restricted to a specific therapeutic area, but increased use of precision medicine in oncology presents a significant benefit for patients. For example advances in the use of circulating tumor DNA (ctDNA) technologies offer less invasive testing options for HealthCare Providers (HCPs) to monitor disease progression or patients in high risk populations. Additionally development of new CDx's helps balance treatment risks by identifying targeted treatment options based on genomic profiles . GenomicWe started by examining the NGS landscape using case studies to identify risk areas related to ensuring safety/safeguarding patients and data integrity in clinical trials. Through this examination we highlighted key clinical quality risk factors across the NGS lifecycle and grouped them into technology, data quality, patients (that undergo tumor testing) and oversight  considerations Fig. .Fig. 1KeWe focused on standards and regulations applicable to the International Conference on Harmonization (ICH) regions , European Union (EU) and Japan).From the ICH guidelines, we selected the one applicable to clinical drug development —ICH-E6 Good Clinical Practices From ISO standards, we considered the ones relevant for Quality Management Systems (QMS) and the use of medical devices.We reviewed major HA regulations and guidance, and selected the ones that pertain to the development, use and validation of NGS in the context of use in clinical trials.We also included major HA regulations applicable to clinical drug development, for example the US Code of Federal Regulations (CFR) that encompass requirements for quality management, use of data and validation of electronic systems in clinical trials.Finally, we reviewed a selection of genetics scientific journals for relevant articles and reviews that could inform clinical quality considerations. We only considered scientific journals where institutions  or scieAs explained in the Background section, we limited our search up to 2010 as NGS was an emerging technology and did not want to include outdated information.A list of standards, regulations and journals included for review can be found in the Supplementary Material.While this paper was not intended to be a systematic review, we applied the following strategy and criteria to screen and identify relevant regulations, standards and scientific journals.To identify relevant information for clinical quality considerations, we applied inclusion/exclusions criteria to all documents screened and reviewed  and oversight . These topics were based on the respective needs of various quality assurance professionals within a pharmaceutical sponsor organization (the primary audience for this review), for example these considerations can be leveraged to:As this was not intended to be a systematic review, we acknowledge that the scope of the search strategy could have been broader; therefore, this can be considered as the main limitation of this review.Bioinformatics algorithms executed in a predefined sequence to process NGS data are collectively referred to as an NGS bioinformatics pipeline . This fiData controllers, processors and accountabilities should be traceable through a combination of contractual agreements, and data integrity controls  throughout the data lifecycle  , 15. DocData generated through NGS may support clinical decisions for patient care, patient eligibility for a clinical study, patients stratification and/or other research activities. This set of considerations is critical to ensure quality while using NGS service providers in clinical drug development. Requirements were already established in ICH and various HA regulations and ISO standards, but can be complemented by recommendations issued by medical genetics societies. The data clinical quality considerations are detailed below and summarized at the end of this section Fig. .Fig. 2DaTo protect patient rights and wellbeing it is critical that the integrity of genomic data and derived insights provided to HCPs is maintained , 12, 16.A defined and fit-for-purpose quality management system (QMS) and are accredited, if applicable, to ensure full NGS-based test workflow (from sample collection kit preparation to DNA extraction through to results interpretation) , 18–20Well-defined and validated sample tracking processes to provide traceability of results to the patient –21Processes in place to ensure scientifically justified analysis parameters are used and documented , 20NGS Platforms, including both Laboratory and Computational Methods, that are validated for compliance against quality metrics , 18–20Computerised system validation and/or qualification policies, processes, controls and monitoring in place to ensure systems are fit for their intended use , 15, 18Audit trails implemented (as required) and assessed as fit for purpose and regularly reviewed , 12, 15A data workflow, including data processing and analysis procedures clearly documented and followed. Raw sequencing requirements, post-sequencing bioinformatics pipeline updates and data management practices should be considered , 15, 19Adequate procedures or monitoring controls in place to demonstrate oversight and resolution of data integrity risks , 12, 15Data archival procedures in place to ensure reproducibility of analyses , 12, 15Use well defined or industry standard biomarker definitions, if available , 22–24Controls in place where test limitations exist , the impact on data is communicated to HCP and patients in a manner that supports clinical use of data , 25, 26The ability to use widely accepted standard open file formats to support future use of the data , 20, 27Controls in place to assess and address data standard requirements for data recipients. For example pharmaceutical sponsors may have specific requirements to facilitate useability. Additionally genomic data intended to be submitted to HA should comply with the applicable data standards, such as CDISC In summary, when genomic data are intended for submission to a HA or to be provided to a HCP, pharmaceutical sponsors should ensure genomic laboratories have:In clinical drug development, all GxP computerised systems , sample management systems, IT development tools) must be qualified or validated for their intended use , 12, 15.Similarly, analytical test methods used in clinical drug development must also be validated for their intended use. Such analytical validation requires the careful prospective end-to-end study of appropriately justified method parameters and performance characteristics  from sampling through to data reporting. In order to demonstrate safety and effectiveness, developers of NGS-based tests must provide assurance of accurate and reliable detection of clinically relevant variants through careful design, development and analytical validation of such techniques , 20, 26.NGS tests can be quite complex, involving many elements to achieve proper collection, and handling of clinical and laboratory samples, selection of adequate sampling materials, accurate laboratory analysis, and correct data processing through use of appropriate bioinformatics pipelines. Therefore, robust validation policies, processes, controls and monitoring must be in place to ensure NGS-based tests, including individual test elements and methods, that directly impact on the reliability of data are fit for purpose throughout the data processing and testing life cycle , 20, 26.In order to aid in the development of such robust controls, HA recommendations are available that outline key considerations for test design, development and validation ; AnalytiSome NGS applications require the capture of a high volume of genomic data that is processed to generate valuable clinical insights –6. PatieWhile manufacturers are responsible for privacy controls for tests they develop and distribute, pharmaceutical sponsors must ensure NGS data use is disclosed and privacy risks are considered and addressed in clinical trial consents and commercial agreements , 26, 31.Manufacturers and pharmaceutical sponsors are responsible for ensuring genomic data is stored in a secure manner. Controls should be in place as required by applicable local laws to address security requirements, and access to data should be limited, not only to protect the integrity of the results, but also the privacy of the patients. These controls should be routinely reviewed against applicable local laws and security tests should be performed to assess weakness in security measures and to identify new threats , 29, 30.The return of genomic individual research results  and theiPatient considerations are clearly stated in clinical trials regulations and HA applicable guidelines , 32, 35;Identifying pathogenic variants through genomic testing can guide treatment recommendations , 4, 6. TProcesses, controls and monitoring are in place to ensure that generation of sequencing data and derived variables are validated , 25, 26;The pharmaceutical sponsors have oversight on methods being verified (and reviewed on an ongoing basis) for analytical performance metrics  and compliance ;Contextual information, test limitations and potential risks  are communicated in a timely and transparent manner to the patient and their HCPs , 38.Through the process of genomic testing, additional variants  or biomarkers  status) could be inferred using algorithmic methods . VariantPatients participating in clinical trials must be provided with written information and consent forms that have been approved by an IRB/IEC . For conConsent form development ensures inclusion of sufficient NGS testing and data collection disclosure information, including the breadth of testing, use of data , potential use of data by third parties, and any unused and/or unreported sequencing data, that participation is voluntary, and that the subject may choose not to participate or may withdraw at any time , 38, 39Implications  of discovering a potential pathogenic germline variant are explained and contextualized , 25, 39Storing anonymized patients data and secondary use of data  is clearly stated and explained , 40Rights for patient privacy, withdrawal of consent  are clearly stated and explained , 39Disclosure of limitations of data  and explanations provided as applicable For genomic testing, whether used in clinical trials or in commercial settings, the above considerations can be transcribed into the following recommendations—to ensure patient’s rights, well-being and safety are protected:Requirements for oversight of NGS service providers do not differ from the ones applicable to other types of service providers  used in clinical drug development. Pharmaceutical sponsors retain the ultimate responsibility for the GxP activities conducted, as well as the quality and integrity of trial data generated on their behalf . Hence, In clinical trials, NGS service providers can perform various levels of GxP type activities under different types of relationships such as a standard service provider relationship, co-development relationship or collaborative model relationship. Irrespective of the relationship type the pharmaceutical sponsors must ensure clear oversight of the GxP service provider (and their third parties) including assurance they are fulfilling contractual obligations and regulatory requirements to ensure end-to-end sequencing data integrity and data credibility , 15, 18.NGS service providers used by pharmaceutical sponsors must be qualified to perform contracted activities. The selection process shall be documented to demonstrate credentials of the provider. Contracts and agreements should reflect the expected quality of the services based on applicable local and national laws and regulations, including national or regional medical laboratory accreditation, where required . Such coIn this paper, we reviewed applicable HA requirements, international standards and recommendations from medical genetics societies to identify relevant clinical quality considerations to ensure data integrity and patient safety in clinical drug development using NGS service providers.Demonstrate that data integrity was maintained through well defined, documented and understood quality standards for genomic testsProvide assurance to patients that their data is safeguarded and is used in an ethical and responsible wayDemonstrate that developed tests lead to better patient outcomes and increased patient safety through generation of transparent, accurate and reliable genomic results to assist HCPs in making informed decisions on next course of actionDemonstrate patient rights were maintained, and sufficient controls were in place to identify and address quality implications, which could have an effect on patient care Demonstrate that up-to-date information on the technology was disclosed to healthcare professionals, including clear communication of limitations of the data  so they can confidently use the data to guide clinical decisionsWhen using NGS in clinical trials it is recommended the quality considerations in this review be evaluated and applied in a fit for purpose manner by pharmaceutical sponsor clinical quality organizations to strengthen the ability to:Clinical quality assurance for NGS service providers will continue to be essential to ensure data integrity and patient safety as NGS testing is well established to guide clinical trials  while liSupplementary file1 (PDF 115 kb)Below is the link to the electronic supplementary material.


Artificial intelligence (AI) is a broad discipline that aims to understand and design systems that display properties of intelligence. Machine learning (ML) is a subset of AI that describes how algorithms and models can assist computer systems in progressively improving their performance. In health care, an increasingly common application of AI/ML is software as a medical device (SaMD), which has the intention to diagnose, treat, cure, mitigate, or prevent disease. AI/ML includes either “locked” or “continuous learning” algorithms. Locked algorithms consistently provide the same output for a particular input. Conversely, continuous learning algorithms, in their infancy in terms of SaMD, modify in real-time based on incoming real-world data, without controlled software version releases. This continuous learning has the potential to better handle local population characteristics, but with the risk of reinforcing existing structural biases. Continuous learning algorithms pose the greatest regulatory complexity, requiring seemingly continuous oversight in the form of special controls to ensure ongoing safety and effectiveness. We describe the challenges of continuous learning algorithms, then highlight the new evidence standards and frameworks under development, and discuss the need for stakeholder engagement. The paper concludes with 2 key steps that regulators need to address in order to optimize and realize the benefits of SaMD: first, international standards and guiding principles addressing the uniqueness of SaMD with a continuous learning algorithm are required and second, throughout the product life cycle and appropriate to the SaMD risk classification, there needs to be continuous communication between regulators, developers, and SaMD end users to ensure vigilance and an accurate understanding of the technology. Artificial intelligence (AI) is a broad discipline that aims to understand and design systems that display properties of intelligence . MachineML techniques incorporate training, validation, and test data sets at different stages of model development. Algorithms are executed in a training data set and results compared with a target value. Parameters of the model are adjusted accordingly as part of this process. Identifying potential data biases  is critical, but not limited to this point. At the validation stage, the fitted model is used to predict responses for observations in the validation data set, a process of fine-tuning the model. In the test stage, the ML model is exposed to a test data set, independent of training or validation data sets, providing unbiased evaluation of the final model. AI/ML includes either “locked” or “continuous learning” algorithms. Locked algorithms consistently provide the same output for a particular input. Such algorithms may be modified to optimize performance, requiring “episodic” regulatory review if the algorithm requires additional inputs or changes in intended use or performance. Continuous learning algorithms, in their infancy in terms of SaMD, modify in real-time based on incoming real-world data, without controlled software version releases. Continuous learning algorithms pose the greatest regulatory complexity, requiring seemingly continuous oversight in the form of special controls to ensure ongoing safety and effectiveness.Although systems with continuous learning may appear conceptually similar to systems that self-calibrate to the local environment , continuous learning algorithms using modern ML techniques are qualitatively different in that portions of their algorithms, in the form of their trained networks, are being modified autonomously. This continuous learning has the potential to better handle local population characteristics, but with the risk of reinforcing existing structural biases, potentially without adequate oversight. Thus, special regulations are needed to classify these risks and accordingly, ensure appropriate human oversight.Medical device regulatory agencies such as the US FDA, EU Notified Bodies, and the UK Medicines and Healthcare products Regulatory Agency (MHRA) have responsibility for protecting public health by only enabling market access for safe and effective products. Further down the line, importantly, health care budget holders then need to assess cost-effectiveness and budget impact, a potential rate-limiting step for successful market access. Lessons on successful AI/ML adoption in other industries are limited in their value given the unique health risks and benefits that health care regulators must assess. To verify claims of safety and effectiveness in the form of submitted evidence, regulators must keep pace with the complexity of algorithm models, including validation and testing stages, selected use of software of unknown pedigree, and real-world performance .The FDA has outlined its proposed framework for SaMD in a total product life cycle approach  and releTo promote rigor and transparency in design and reporting of AI-based interventions (underpinning regulatory submission evidence claims), reporting guidelines and checklists include Consolidated Standards of Reporting Trials–Artificial Intelligence (CONSORT-AI), Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence (SPIRIT-AI), The Transparent Reporting of a multivariable prediction model of Individual Prognosis Or Diagnosis-Artificial Intelligence (TRIPOD-AI), and Minimum Information About Clinical Artificial Intelligence Modeling (MI-CLAIM) ,10. In tGlobally, the International Medical Device Regulators Federation (IMDRF) aims to accelerate medical device international regulatory harmonization and has drafted key SaMD policies to complement existing international standards, particularly in terms of risk classification, converging terminology, a risk-based framework, and quality management systems. The Institute of Electrical and Electronic Engineers (IEEE) has Artificial Intelligence Medical Device Working Groups on terminology and recommended practice for the quality management of data sets. United Nations agency collaboration between the World Health Organization and the International Telecommunication Union: Focus Group on Artificial Intelligence for Health (FG-AI4H) was established to use AI to advance health care for all, and to benchmark AI models using secure and confidential, globally representative data sets .It is recognized that patient-centered data and engagement play a fundamental role in regulatory assessment of SaMD. The “patient-centered” approach referred to by the FDA addresses usability, equity, trust, and accountability. Engagement with both developers and end users occurred at a February 2020 Public Workshop on the Evolving Role of Artificial Intelligence in Radiological Imaging. At the latter event, The American College of Radiology (ACR) and Radiological Society of North America (RSNA) questioned  the abilSaMD has great potential to improve health and health care at individual and system levels. To optimize on the benefits associated with SaMD, patient safety and effectiveness need to be aptly assessed for which 2 key steps are necessary. First, international standards and guiding principles addressing the uniqueness of SaMD with a continuous learning algorithm are required , outlini


COVID-19 social distancing guidelines caused a rapid transition to telephone and video technologies for the delivery of mental health (MH) services. The study examined: (a) adoption of these technologies across the MH service continuum; (b) acceptance of these technologies; and (c) intention of providers to use these technologies following the pandemic based on a sample of 327 MH organizations from 22 states during May–August 2020. There was widespread use of technology, with greater than 69% of organizations reporting using telephone or video for most services. For all video services and just three telephone services, organizations reported significantly greater odds of intending to use technology to deliver services post-COVID-19. Use of video was seen as more desirable as compared to telephone. The overall perceived ease of use and usefulness for video-based services and certain telephone services provide a promising outlook for use of these services post the COVID-19 pandemic. The World Health Organization’s declaration of the Public Health Emergency of International Concern (PHEIC)  databases. The MHTTCs are Substance Abuse and Mental Health Services Administration (SAMHSA)-funded technical assistance centers whose purpose is to provide training and technical assistance to the mental health prevention, treatment, and recovery workforce in the United States. Four regional MHTTCs and four regions representing 22 states distributed this survey. Survey links were sent to respondents using an email script approved by the University of Wisconsin’s Institutional Review Board (IRB). Individuals sent the survey link were either administrators of mental health programs or mental health clinicians .The survey included the following components and scales:Locations included rural, small city, suburban, and urban. Organizational types included specialty behavior health and health systems.There were two categories of survey respondents: administrators and individuals who provide clinical services. These groupings were included because of their role in the adoption continuum. Administrators are often the decision-makers on whether telehealth will be offered as a service delivery option, and those who provide clinical services influence the ongoing daily use of telehealth for mental health services.(a) The use of telephone and video-based services was assessed for the following services: screening and assessment, case management, multi-disciplinary team-based services , peer supports, group therapy, individual therapy, medication management, psycho-education, therapy services during partial hospitalization, and therapy sessions during residential treatment with a binary Yes/No response option; and b) The projected use of telephone and video services for each these services following COVID-19 was assessed asking respondents to what extent they plan to use telephone or video services beyond use to maintain COVID-19 safety measures, with response options of less than before, about the same, little more than before, or much more than before.The Organizational Readiness for Technology Use predictive tool developed by Gustafson et al.  was applThe survey included scales from the Technology Acceptance Model (TAM) , participant job roles, use of telephone and video technologies for different SUD services, and intent to use telephone and video technologies to deliver the different services post-COVID-19. Linear mixed-effects models (LMM) were conducted to determine if there were differences in intent to use telephonic and video technologies based on organizational location or setting and the survey respondent’s role. For this analysis, a composite measure of the intent to use telephone and video technologies was created by averaging the intent to use scores across the different mental health services for each technology. The Intent to Use Telephone and Video technologies for the different services was analyzed using generalized linear mixed-effects models (GLMM) comparing “More Use” and “Little More Use” of the technology post-COVID-19 to “About the Same” and “Little Less” post-COVID-19. The Organizational Readiness for Technology Adoption variables were analyzed by comparing the Organizational Readiness for Technology Use factor scores between Telephone and Video using generalized linear mixed-effects models (GLMM) to determine factor differences between the technologies. Lastly, the TAM data was analyzed by conducting a mediational analysis of the Perceived Ease of Use and Perceived Usefulness variables compared to the composite Intent of Use variables for telephone and video services using linear regression. The analyses were conducted using the lme4 package in R Studio.Three hundred and twenty-seven organizations were represented in the survey. Surveys were distributed to 1790 organizations that provide mental health services for a return rate of 18%. The respondents’ job categories were 50.8% administrators and 49.2% clinical services providers  Telephonic/video counseling is affordable to patients , and (b) Most of our patients can access telephonic/video technology   or involving multiple clients  were more challenging to deliver remotely and were less likely to be offered via telehealth.While the transition to telemental health has been challenging for some (Murphy et al., The survey findings support the continued use of telehealth services offered by mental health providers and organizations, as respondents indicated a desire to use these services more following the pandemic. Results suggest that clinic leadership and staff support the implementation and sustainment of telephone and video counseling. A review or update of telehealth reimbursement guidelines and associated regulations may be warranted to support providers and organizations in these efforts. The survey results also highlight factors that mental health providers and organizations believe allow for increased patient accessibility; specifically, telephone services are affordable and easily accessible. As mental health providers and organizations seek to incorporate patient-centered approaches, telehealth services may allow for greater patient choice in seeking and receiving services. Although findings from this survey indicate video-based telehealth services are viewed more favorably than telephone-based services, additional research is needed to identify which service modality can be delivered the most effectively. Similarly, because of its perceived patient ease of use and accessibility, telephone delivery may be supported for some clients. Hence, a diverse mix of service delivery modalities, including in-person services, may be needed to maximize flexibility, outcomes, and patient preferences.The survey findings should be understood in the context of at least three limitations that affect generalizability. First, the mental health providers and organizations responding to the survey were from a convenience sample pulled from four regional MHTTC databases representing 22 states. Second, the return rate for this survey was 18%. Third, the survey’s email distribution was the only recruitment modality used and could have limited reach and response had multiple distribution approaches been used.Previous research on telemental health has established that it can be effective, improve client satisfaction, and reduce the overall cost of care (Hilty et al., In summary, the results demonstrate a large shift in telehealth use and provide an encouraging outlook for the use of telephone and video-based services after the COVID-19 pandemic. Future studies should continue to review the acceptance of these different service delivery approaches and their impact on care outcomes. In particular, integrating in-person, telephone, and video-based services to make them more patient-centered and achieve optimal outcomes should be studied in practice and research.


Each year he GSA publications team sponsors a symposium to assist authors who wish to publish in GSA’s high impact and influential journals. The first part of the session will include five brief presentations from the editors of The Gerontologist, Innovation and Aging, and the Journals of Gerontology Series A and B plus GSA’s managing editors. We will integrate practical tips with principles of publication ethics and scholarly integrity. The topics will be as follows: (1) Preparing your manuscript: strong and ethical scholarly writing for multidisciplinary audiences, (2) common problems that affect peer review, (3) addressing translational significance and fit to journal expectations, (4) transparency, documentation, and Open Science; and (5) working with Scholar One. Following these presentations, we will hold round table discussions with editors from the GSA journals portfolio. At these round tables, editors will answer questions related to the podium presentations and other questions specific to each journal. Intended audiences include emerging and international scholars, and authors interested in learning more about best practices and tips for getting their scholarly work published.


In collaboration with the ECHO  Institute since 2012, the Army, Navy, and Air Force have developed medical teleECHO programs to address various health and safety issues affecting military personnel. This article describes and compares the current state of military teleECHOs as well as the growth and change over time.This study evaluated continuing education units (CEUs) offered, average session attendance, and number of spoke sites for current military teleECHO programs across the service branches.Between 2012 and 2019, the military teleECHO initiative grew from one program to seven different teleECHO programs, covering topics from pain to diabetes to amputee care. Military ECHOs now provide training to 10 countries and 27 states in the United States. Between October 2018 and September 2019, the military ECHO programs provided a total of 51,769 continuing medical education (CME) hours to a total of 3,575 attendees from 223 spoke sites.The military has successfully used the ECHO model to improve the health and safety of active-duty military, retirees, and dependents. The ECHO model is now addressing over 70 health-related and non-medical conditions.2Project ECHO  began in 2003 at the University of New Mexico (UNM) to leverage technology and improve specialty care access in New Mexico’s rural and medically underserved communities. The ECHO model features an interprofessional team of specialists at the “hub” site, training primary care clinicians at the “spoke” sites remotely using videoconferencing technology. The spoke clinicians present real, de-identified cases in an “all teach, all learn” format, while the participants are encouraged to engage in the bi-directional best practices learning environment. Short lectures and no-cost continuing medical education (CME) credits are provided in all teleECHO sessions and outcomes are evaluated.3 Project ECHO’s Chronic Pain and Opioid Management program (ECHO Pain) is staffed by an interprofessional team of subject matter experts including neurology, myofascial pain, addiction psychiatry, behavioral health, physical therapy, chiropractic medicine, and advanced practice clinicians. Founded in 2009, ECHO Pain teaches non-opioid pain management, safe opioid prescribing, and screening for opioid use disorder and consults on pain and opioid misuse cases in weekly 90-min sessions.The opioid epidemic is a serious public health issue that has cost the USA $1.02 trillion.4 was invested in maximizing the role of primary care providers in managing chronic pain. Three Army Pain ECHO hubs were created in the continental USA, one in Hawaii, and one in Germany. The Hub Readiness Replication Model4 was created during this process and was the first example of a widely used replication tool. The goals of this tool were to ensure: (1) fidelity to the ECHO model during each MEDCOM teleECHO Pain launch, (2) consistency in the replication process, and (3) adaptability for use during all future clinical and non-clinical replications.Beginning in 2012, the U.S. Army collaborated with the ECHO Pain team to launch a large-scale replication of ECHO Pain across five U.S. Army Medical Command (MEDCOM) regions. The Army Comprehensive Pain Management Program4 as the U.S. Army, the U.S. Navy launched two large Navy Pain ECHOs  as part of their comprehensive pain program.5 The Navy Polypharmacy ECHO started as an onboarding effort for 21 new pain-focused clinical pharmacists serving primary care medical home clinics. This program focuses on treating complex pharmacy pain patients and has grown to become a virtual network for the Navy and other military branches both in the continental USA and overseas. The Navy also developed a Traumatic Brain Injury (TBI) ECHO to expand and increase resources associated with TBI medical knowledge across the military healthcare system. The TBI ECHO program was initially created to support the Naval Primary Care Clinicians located within the operational commands to better enable them to treat sailors and marines who suffer TBI-related injuries. Finally, the Navy most recently launched the Navy Sexual Assault Medical Forensics Examination ECHO to standardize the documentation processes and requirements associated with conducting sexual assault medical forensic examinations across Navy Medicine.Using the same Hub Readiness Replication Model6 This telementoring program provides the ability to reach deployed personnel, supporting force readiness regarding diabetes prevention and pre-diabetes.Air Force Diabetes ECHO was launched at the Diabetes Center of Excellence at Wilford Hall Ambulatory Surgical Center and Lackland Air Force Base to address limited endocrinology access at small military treatment facilities (MTFs).The Amputation Care ECHO was originally launched as an Extremity Trauma and Amputation Center of Excellence initiative designed to address educational gaps and facilitate a knowledge sharing network across the Military Health System with clinicians involved in amputation care. Initially, Amputation Care ECHO was established with one hub, Walter Reed National Military Medical Center, and four spokes, including the Veterans Health Administration (VHA).Please see 7 The Anatomy of an ECHO training tool has been used as a guide to illustrate the process of presenting a case, asking for clarifying questions; developing a differential diagnosis; and making non-pharmacological, pharmacological, and interventional recommendations.7 MOCK ECHO is a simulation training that allows clinicians to practice presenting cases with an interprofessional hub team and multiple spoke clinicians. It assists with new military ECHO launches as well as with refresher trainings.7During the U.S. Army–Project ECHO collaboration, MOCK ECHO and Anatomy of an ECHO were both created as important tools to ensure fidelity and standardization across the six large Army Pain ECHO hubs, as well as to aid in the training of hub and spoke clinicians.Because ECHO programs rely on evidence-based and evidence-guided treatment guidelines and recommendations, the development of a core ECHO curriculum is essential to training primary care clinicians in best practices to ensure the highest quality of care for their patients. One example of an ECHO curriculum that is being used by the Army and Navy Pain ECHOs is the Joint Pain Education Project (JPEP)—A collaboration between the Department of Defense (DoD) and VHA’s joint strategy on addressing the unwarranted variability of pain management practice. The JPEP common core curriculum consisted of 18 modules targeting the most common and challenging pain care conditions treated in primary care . DevelopMilitary ECHO refresher courses are offered to each of the current Air Force, Army, and Navy ECHOs by the faculty of the ECHO Pain team in order to provide ongoing continuing education related to the ECHO model. These refresher courses are provided both in-person and through synchronous and asynchronous videoconferencing. The content is not limited to, but may include MOCK ECHO simulation trainings, how to deliver Anatomy of an ECHO, helping hub teams strategize options for recruitment of spoke clinicians, and videoconferencing etiquette. Refresher courses allow the military ECHOs to preserve fidelity, maintain consistency, and assure standardization as clinicians and support teams rotate on and off of service.Since the DoD ECHO programs began in 2012, there are now hubs and spokes located in 27 U.S. states and 10 countries. In addition, many DoD ECHO programs collaborate across federal agencies to hold joint ECHO programs with the VHA and civilian academic medical centers. Some military ECHO clinics are well established and have been running for 8 years, whereas others are newer and have only been running for a year . Hub teaThe number of attendees and spokes for the Navy reflects all Navy programs, including Navy Pain, Navy TBI, Navy PPI, and Navy SAFME. During the fiscal year 2019, the military ECHO programs provided a total of 51,769 CME hours to attendees. These are reflective of CMEs made available to, but not necessarily collected by, attendees. Each military service has a point of contact in their training and education section, who reviews and approves CME. The Army Pain ECHOs, which have been running the longest, made 40,041 CMEs available to attendees.8 Using central Defense Health Agency (DHA) pharmacy data, new annual opioid prescriptions for patients whose clinicians participated in ECHO Pain versus the comparison group had a 23% (vs. 9%) reduction, P < .001.8 In addition, other military Pain ECHO programs that were originally trained using the Hub Readiness Model have been evaluated. Flynn et al. demonstrated that patients whose primary care providers participated in the telementoring program also had lower doses of prescribed opioids.9The continued success of the original Army and Navy Pain ECHO programs are most likely due to deliberate programmatic pre-planning and initial trainings used in the Hub Readiness Replication Model as well as continued refresher courses. In a 4-year observational cohort study, the patients of clinicians who participated in Army and Navy Pain ECHO programs had greater declines in opioid-related prescriptions than patients whose providers opted not to participate.6 In this same study, 96.8% of primary care physicians reported that they were likely to make changes to their treatment practice for patients with diabetes based on what they learned in the ECHO program.The Air Force Diabetes ECHO program found that, after training, the average self-score for (1) perceived level of knowledge and (2) personal confidence level with regard to treating complex diabetes patients increased for all respondents.10 For VA patients with chronic pain and opioid substance use disorder, SCAN-ECHO has proven beneficial in lowering opioid dose while incorporating a strong biopsychosocial model of care.11 In another large VA SCAN-ECHO pain study, the authors found that clinicians who present their patients to SCAN-ECHO for pain had an increase in referrals to rehabilitation and for non-opioid medications suggesting safer approaches to pain care. The study also concluded that primary care clinicians learned best practices pain management while presenting cases to SCAN-ECHO.12 An IHS ECHO telementoring study has shown significant benefit in clinician knowledge, confidence, and attitudes toward treating patients with chronic pain and addiction after training more than 1,300 clinicians.13Many other federal agencies have adopted the ECHO model including the Veterans Affairs (VA) and the Indian Health Service (IHS). The VA has developed more than 10 Specialty Care Access Network (SCAN) ECHO programs, beginning in 2011. A recent study at the Ann Arbor, Michigan, VA has shown improved survival of patients with liver disease if clinicians use SCAN-ECHO as compared to usual care.7 Tools have been developed specifically for military personnel to support fidelity of the program.4 Studies comparing the cost of pain management for patients using the ECHO model vs. in clinic found that, although the costs per patient were similar , following the ECHO model provided faster access to interdisciplinary and collaborative consultation while minimizing burden on patients.14 The ECHO telementoring in the military can leverage scarce specialty care resources to rapidly share emerging best practices and guidelines. Earlier recognition, treatment, and prevention of chronic pain, substance use, diabetes, TBI, sexual assault sequelae, and postamputation complications can each improve force readiness, which is a priority for all military commands. Additionally, military ECHOs provide training on subjects that are used as performance measures for healthcare facilities. For example, the Air Force, which runs the Diabetes ECHO, had A1c testing for 91.69% of members as of September, 2019. As the DHA adopts cross-service approaches to patient care, the ECHO model presents an ongoing opportunity to standardize care and share best practices. Not only is the ECHO model beneficial in leveraging specialty care access when geographic distance may prohibit a timely consultation, but it has proven to be beneficial during public health emergencies as well. When in-person patient and clinician safety are paramount, such as during the COVID-19 pandemic, the ECHO model has proven to be successful for specialty care consultation.15The ECHO model has proven to be a useful educational tool to support standardization of training and mentorship, as demonstrated by mock ECHO sessions conducted by organizations replicating the ECHO model.In addition to the successes of ECHO telementoring in the military, the services have faced several challenges, which may need to be addressed if the model is to expand. The first is that initial champions for some subject areas may undergo changes of station at regular intervals, leaving gaps in hub team coverage or interest in maintaining the teleECHO sessions. The second is that videoconferencing software approved for military use is not as user-friendly as the Zoom platform, now used for the civilian ECHO experience. Thirdly, it is important that continuing education credits be optimized for all healthcare professionals as part of each ECHO program. This allows clinicians to both account for their time and to benefit from distance learning instead of needing to travel away from their post for an educational meeting. Finally, and probably most importantly, the decision to utilize the ECHO model requires the investment and commitment of military leadership if it is to endure among competing priorities for resources. Clinicians participating in the ECHO sessions, whether hub or spoke members, need protected time in order to continue to excel in their clinical activities and benefit from the educational venue of Project ECHO.16 diabetes, sexual assault, and amputation care among active and retired personnel and their dependents. These programs ensure that small and large MTFs can deliver a uniform standard of care while expanding the capacity of military primary and specialty clinicians to make effective use of limited resources.Military ECHO participants at all levels model a concern for patient and peer well-being that is exemplary, and the pride they take in the care they deliver is well deserved. Each program has done a comprehensive job of making its content available across services, of collaborating in design and delivery of needed content, and of engaging a broad variety of clinical learners in acquiring new knowledge and skills. Military ECHO programs are integral parts of reducing the burdens of opioid overdose,


Evidence-based medicine relies on appropriately designed, conducted and reported clinical trials (CTs) to provide the best proofs of efficacy and safety for pharmacological and non-pharmacological treatments. Modern clinical research features high complexity and requires a high workload for the management of trials-related activities, often hampering physicians’ participation to clinical trials. Dealing with children in clinical research adds complexity: rare diseases, parents or legal guardian reluctance to engage and recruitment difficulties are major reasons of pediatric trials failure.However, because in pediatrics many treatments are prescribed off-label or are lacking, well-designed clinical trials are particularly needed. Clinical Trial Units (CTUs) are indeed an important asset in the implementation of clinical trials, but their support to investigators is limited to administrative and non-clinical tasks. In this paper we present the model of the Investigational Clinical Center (ICC) of the Bambino Gesù Children’s Hospital in Rome. The ICC includes clinicians supporting the Principal Investigators for clinical management of enrolled patients in compliance of Good Clinical Practice, the legal framework of Clinical Trials. Furthermore, we present 10 years’ experience in pediatric clinical trials and how it has been affected in 2020 by the COVID-19 pandemic. The activity of the ICC has been evaluated according to specific metrics of performance. The ICC model offers a complete support, helping investigators, patients and their families to overcome majority of barriers linked to clinical research, even in time of pandemic. We propose this organization as an innovative model for total-supportive and patient-centered clinical trial implementation.The online version contains supplementary material available at 10.1186/s13052-021-01099-0. Clinical trials (CTs), when appropriately designed, conducted and reported, provide the best evidence of efficacy and safety for pharmacological and non-pharmacological treatments, but are increasingly expensive, complex, and need highly specialized competencies.Trials are particularly lacking in children: in this population evidence of safety and efficacy of drugs is scarce, and off-label prescription is a common practice, with potential issues for patients’ safety . Trials Principal Investigators (PIs) and their close collaborators dealing with pediatric trials are often chosen by Sponsors mainly for their specialized clinical competence and for their access to the population for which the clinical trial is designed.Performing pediatric trials involves important burdens in terms of time for non-clinical and clinical activities. It requires specific knowledge and multidisciplinary competences in terms of regulatory, ethic and scientific expertise, which may represent a barrier for clinicians busy with routine clinical duties. Indeed, clinicians are often not willing to participate in clinical trials due to many related complexities and burdens. In a survey conducted among American pediatricians, the training of the site staff in clinical research procedures resulted as one of the main barriers for participating in a trial . MoreoveClinical Trial Units (CTUs) have been established in the last decades, including study coordinators, data managers, statisticians and personnel who help clinicians in performing administrative activities. CTU assistance does surely improve trials implementation and management, but it does not address all the barriers to participation of clinicians in trials, leaving all the clinical tasks to PIs and their close collaborators .Standard Clinical Trial Site: trials conducted in hospital wards, PI and Sub-Investigators not supported in any of the trial activities;Supported Clinical Trial Site: a CTU supports PI and Sub-Investigators in administrative and non-clinical activities;Investigational Clinical Centre: PI and Sub-Investigators are totally supported not only in administrative activities, but also in clinical duties by physicians, nurses fully dedicated to trials.Indeed, different models have been adopted for the implementation of clinical trials:We present in Fig. Investigators and study nurses are key figures for all clinical trials, as they perform clinical tasks required by the protocol and assess the safety and the efficacy of the investigational treatment, the permanence of a patient in a trial, the severity and causality of adverse events (AEs). These duties, together with many other barriers associated to the growing complexity of trials procedures, and the lack of time due to standard clinical obligations, hamper the participation of clinicians to clinical trials, ultimately slowing down clinical research .The “Centro Trials” of Bambino Gesù Children’s Hospital, is an archetype of Investigational Clinical Center (ICC). The ICC is led by a core team of clinicians, who support PIs and specialist sub investigators in conducting the trials and helping in the clinical management of the trial subjects. In this model, the investigators are supported by ICC’s physicians and nurses, who are confident with the clinical duties common to all trials, such as timely AEs and SAEs report with causality assessment, randomization, blinded drug administration and compliance.Study coordinators support the start-up process, organize visits procedures, complete case report forms (CRFs) with trial patients’ data and help sponsor delegated clinical research assistants (CRA) in their monitoring visits at the IC. Furthermore, they work closely with clinical staff, providing timely information about trial’s required data.All ICC’s personnel are trained in Good Clinical Practices (GCP) and take part in feasibility visits and site initiation visits (SIVs). They act also as trainers to PIs and their entourage for GCP and trial procedures, organize annual GCP courses for Hospital and external investigators and ultimately promote a Hospital-wide clinical research culture. Moreover, the ICC personnel offers their methodological services to various stakeholders (including independent researchers and drug companies), supporting not only the conduction but also the design and development of profit and no-profit clinical trials.The ICC features its own dedicated spaces, providing investigators with an efficient infrastructure to recruit, perform and manage clinical trials, far from the crowd of clinical wards and ambulatories. Those spaces include dedicated rooms for medical visits, drug infusions, monitoring visits, archive, and a samples’ processing room with regularly monitored equipment (centrifuge and freezers) according to the requirements of Good Clinical Practices.In scientific literature, there is a paucity of relevant publications regarding performance indicators of trial implementation . We prodnumber of new opened trials per year;number of active trials per year.Overall activity:ratio of final number of enrolled patients over expected as stated in the contract agreement per year;dropout ratio (% of patients that prematurely ended their study participation after enrolment for own decision or protocol deviation) on the total of randomized patients per year.Recruitment and retention:percentage of randomized patients with at least one protocol violation.Protocol Compliance:We divided the metrics of performance in three main distinct areas, considering:The ICC of the Bambino Gesù Children’s Hospital showed a positive trend in the overall activity metrics: in the period from January 2010 and December 2020 the number of opened studies amounted to 211. The number of active studies per year from 2010 and 2020 increased almost constantly, moving from 18 active studies in 2010 to 104 active studies at the end of 2020. The number of new opened trials per year ranged from a minimum of 10 to a maximum of 31. The recruitment and retention metrics are of relevant importance, as the enrolment phase is particularly critical in pediatric trials. The average ratio of final number of enrolled patients over the expected per year was 74% , and many different therapeutic areas , the sponsor had to request prior authorization to the Institutional Data Protection Officer.Despite these additional hurdles, the structure of ICC succeeded to maintain its activities, and assured continuity in clinical assistance to his patients. Only one dropout and no protocol violations occurred in 2020. Moreover, 23 new studies were started in this year.ICC of the Bambino Gesù Children’s Hospital contributed to provide evidence for the approval of some of the most relevant drugs recently approved in the pediatric population, such for the treatment of rare diseases, including Cystic Fibrosis, Duchenne Muscular Dystrophy, Batten disease and Spinal Muscular Atrophy Table .Table 1To address the difficulties in the design and conduct of paediatric clinical trials, there is the need of trained and specialized centres, and collaborative international network, like the Collaborative Network for European Clinical Trials for Children  in Europe, or national networks .. The ICIn conclusion it has been highlighted how CTUs provide an important assistance but are not exhaustive and cannot address many barriers and issues related to clinical research . The impAdditional file 1: Supplementary Material S1. Number of trials per therapeutic supported by the Bambino Gesù Children’s Hospital Investigational Clinical Center.


To evaluate the clinical efficacy of nonpharmacological interventions in improving balance function of patients with osteoporosis or osteopenia using network meta-analysis (NMA).  We searched seven databases  for relevant randomized controlled trials (RCTs) up to August 31, 2020. Berg Balance Scale (BBS) and Time Up and Go Test (TUGT) were used as outcome measures. Two researchers independently screened studies, collected data from the studies, and estimated risk of study bias. Divergence in the evaluation process was settled by consulting a third researcher. We used Stata 15.1 software for network meta-analysis (NMA).  We identified 10 eligible RCTs, with a total of 737 patients and four intervention methods, including traditional Chinese medicine (TCM) exercises, pulsed electromagnetic fields (PEMFs), strength training, and balance and strength training. The results revealed that all nonpharmacological interventions could improve balance function, but the effect of balance and strength training was better than other interventions.  Exercise can significantly improve the balance function of patients with osteoporosis and osteopenia, and balance combined with strength training has the best effect, followed by TCM exercises. Osteoporosis is characterized by deterioration of bone microstructure and loss of bone mass and easily leads to fracture. Osteopenia is the precursor of osteoporosis. The fracture rate of women with osteoporosis is 2.7 times higher than that of average women, and the fracture rate of women with osteopenia is 1.7 times higher than that of average women –5. The pFall risk is an independent factor affecting fracture frequency, and decline in balance ability is highly correlated with the incidence of falls . Currenthttp://www.prismatatement.org/) and is registered in the PROSPERO database as CRD42020206838.Research was conducted in accordance with the Preferred Reporting Items for Systematic Review and Meta Analyses (PRISMA) statement  up to August 31, 2020, based on MESH terms and free words, and pertinent details of results saved.(1) Types of Patients. The 1994 diagnostic criteria recommended by the WHO defines those with a BMD value lower than the peak bone value of normal adults of the same gender and race by 1–2.5 standard deviations as having osteopenia, and those with a reduction ≥2.5 standard deviations as having osteoporosis; individuals meeting either criteria were included.(2) Interventions. Nonpharmacological interventions included in the experimental group were acupoint stimulation therapy, TCM exercises, kinesitherapy, occupational therapy, and physiotherapy. The control group included the basic treatment recommended by national guidelines .