TBs have been shown to improve patient outcomes and enhance high-quality patient care, resulting in altered treatment plans in up to 50% of reviewed cases and improved survival for solid organ tumors.5 Weekly TB preparation compiles clinically relevant data from a variety of sources and has been shown to be both time- and labor-intensive.6 When TB data are subsequently used for institutional quality improvement efforts or clinical research such as traditional retrospective analysis, repeated manual chart review is typically required, resulting in additional, time-consuming effort prone to human error. Here, we present a simplified, expert-driven TB documentation workflow that automates oncologic data capture and reporting. This provides a framework for regular sharing and analysis of treatment-related outcomes and quality metrics and a method for efficient clinical trial screening and recruitment.Multidisciplinary tumor boards (TBs) have increasingly become the standard of care for developing optimal treatment plans for complex cancer patients.Multidisciplinary head and neck TB conferences occur weekly at our institution in a virtual format led by a head and neck surgeon and last approximately 2 hours. Quorum requires, at a minimum, at least one faculty member from head and neck surgery, medical oncology, radiation oncology, head and neck neuroradiology, and nuclear medicine. Before each weekly conference, cases are submitted throughout the prior week, and internal and external pathologic slides and diagnostic imaging are reviewed by attending-level specialists. Structured TB notes that collate relevant clinical data are prepared in advance by resident physicians for presentation at the conference. Treatment recommendations, disease status, and clinical trial candidacy are then recorded during the meeting by the head and neck oncology fellow on the basis of group consensus achieved through discussion and recommendations consistent with National Comprehensive Cancer Network (NCCN) guidelines. In the occasional setting of differing opinions or differing treatment recommendations, these are recorded as well; typically, free text is used to elaborate on the differing recommendations.In 2018, our electronic health record Head and Neck Tumor Board template was transitioned to a simplified, expert-designed template with relevant clinical data populated into structured fields with a combination of automatic population from the electronic health record and manual selection of drop-down menus. Before this intervention, all patients discussed at the conference had free text TB documentation recorded either by faculty or resident, inclusive of any relevant information presented at the multidisciplinary meeting. The impetus of this change was to provide a more streamlined documentation and presentation workflow and to facilitate department-specific, user-friendly decision support tools for retrospective review of performance and quality improvement metrics. Four different templates are currently used, titled HNTBPRETX for primary treatment discussion, HNTBPOSTOP for adjuvant treatment decision making, HNTBRECURRENCE for evaluation and discussion of possible recurrent or persistent disease, and HNTBTEMPLATE for case considerations that do not fit into the previously listed categories.Required, structured fields within each documentation template with associated branching logic ensure uniform, complete data entry regarding disease stage, status, and prior primary and adjuvant therapy Table . TreatmeImplementation of this efficient, high-quality documentation workflow in more than 1,700 head and neck TB cases has ensured standardization of processes for case discussion and reduced errors during retrospective analyses performed for clinical research and quality improvement efforts. This simple digital tumor board system built directly into the clinical note has supported the complex workflows of clinical practice, including identification of an additional 162 (9.3%) patients as potential candidates for clinical trials on the basis of categorical inclusion criteria and generating reports of surgical pathology outcomes such as lymph node yield from neck dissection and margin status (Fig


Narrow band imaging (NBI) is a powerful tool that allows visualizing the mucosal and submucosal vasculature. Among the available diagnostic techniques, NBI is one of the most valid for early detection of oral squamous cell carcinoma (OSCC).We carried out a bibliographic search in PubMed, Scopus and Web of Science databases using relevant keywords. Articles selected were screened by two independent authors based on inclusion and exclusion criteria. Nine papers were singled out according to the eligibility criteria and included in this review. We investigated the articles for pooled sensitivity, specificity, accuracy, positive predictive value and negative predictive value of pre-operative NBI.The use of NBI examination in the oral cavity revealed higher specificity, sensitivity, positive and negative predictive values and accuracy compared to white light examination for the diagnosis of oral squamous cell cancer (OSCC). In addition, NBI has proved great utility in detecting malignancy features in oral pre-malignant lesions.This review shows that NBI is a powerful tool for examining oral suspicious lesions. Most of the articles examined revealed high values of sensitivity, specificity, positive predictive value, negative predictive value and accuracy in detection of oral malignant and pre-malignant lesions. Therefore, the use of NBI is highly recommended for the early detection of oral cancer and potentially malignant disorders. Future studies should seek to affirm the validity of NBI and in particular to standardize NBI classification. Oral cancers are a vast group of cancer type that manifests in different areas of the oral cavity. Oral squamous cell carcinoma (OSCC) is one of the most common histological types of head and neck cancer and its incidence typically increases with age. Tobacco and alcohol are well-known risk factors for the development of this cancer, other risk factors included in the etiology comprehend human papillomavirus (HPV) infection, chewing betel quid or radiation exposure. The early detection of the tumor or precancerous lesions and their radical resection is key for oral cancer prognosis. Indeed, several studies show that failure to obtain both superficial and deep clear margins in OSCC lead to an increase of local recurrence and mortality rate of OSCC patients. In recent years several new non-invasive technologies have been introduced for the diagnosis of oral pathology. Among these, the Narrow–Band Imaging (NBI) has shown excellent results in the identification of superficial mucosal lesions of oral mucosa .NBI was first conceived for the detection of gastrointestinal tumors, subsequently, used in identifying early stage mucosal SCC of head and neck -4. This device uses two narrow bands of light (blue and green), the former at 400-430 nm and the latter at 525-555 nm, to penetrate superficial mucosa and to enhance the visualization of the mucosa surface and the capillary pattern of the superficial layer, in particular the intraepithelial papillary capillary loops (IPCLs) ; Takano Normal mucosa has IPCL type I and is characterized by regular vascular arborization; IPCL type II and III show dilated and elongated vasculature pattern respectively; lastly, irregular morphology or ripped blood vessels are classified as type IV. IPCL types I and II are typically present in non-neoplastic lesions while patterns III and particularly pattern IV can be predictive of High-Grade Dysplasia (HGD), carcinoma in situ (CIS) and OSCC . Since tThus, this systematic review aims to evaluate the validity of NBI in terms of sensibility, specificity, positive and negative predictive values (PPV and NPV) and accuracy compared to white light endoscopy for detection and follow up of oral cavity lesions.Study CriteriaOur literature is based on the review of studies that investigated and evaluated the effectiveness of NBI for aiding the detection of malignant and pre-malignant lesions of the oral cavity. The studies considered had to meet inclusion and exclusion criteria. Inclusion criteria were: original articles reporting the use of NBI in patients affected by oral cavity lesions; studies evaluating at least two of these values between sensitivity (SE), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), accuracy (Acc) for NBI or both WL and NBI; studies published in English in peer-reviewed journals. Studies were excluded if: they were review articles or editorials, opinions, conference abstract or case report; studies that investigated the parameters  for oral together with oropharyngeal lesions.Search strategy Using PRISMA guidelines, research through online databases included PubMed, Scopus and Web of Science (up to 1st April 2020).A detailed search strategy was performed for each database using relevant keywords, such as “Narrow band imaging” or “NBI”, “oral” or “suspicious oral lesion”, “oral cancer”, “oral cavity”, “oral erythroplakia” or “oral leukoplakia” .Selection of studiesAfter removing duplicates, two authors (GG and EC) independently screened titles and abstracts of all studies retrieved from the initial database research. Potentially relevant articles were identified, and their full text obtained for detailed assessment of eligibility according to our inclusion criteria . A thirdData extractionDetails of the selected studies were extracted including authors name, publication year, study type, inclusion and exclusion criteria, number of patients enrolled, number and type of lesions analyzed and in particular the values of sensitivity, specificity, PPV, NPV and accuracy of NBI and eventually WL too (using tissue histopathology as the gold standard comparison)  and 3.We searched from databases including PubMed, Scopus and Web of Science selecting a total of 1657 articles. After eliminating duplicates (1406), we screened 251 articles for title, abstract and language criteria, and 222 of these were eliminated because not relevant. Out of these, 29 full papers were obtained for screening against the inclusion criteria. A total of 9 studies was eligible to be included in the review.Preoperative use of NBI and suspicious malignant lesionsYang et al (9) retrospectively reviewed 63 patients with chronic oral non-healing ulcers. All patients underwent NBI examination and each lesion was classified using IPCL classification. The clinical sensitivity, specificity, PPV, NPV and accuracy for the detection of OSCC by NBI were 93.75%, 91.49%, 78.95%, 97.73%, 92.06% respectively. Most of the IPCL type patterns III and IV were associated with pre-malignant and malignant lesions, with only 4 cases out of 19, among those classified as pattern IPCL III and IV, diagnosed as benign ulcers. Shibahara et al  performeComparison between WL and NBIThree of the selected articles compared the use of NBI as opposed to WL for the assessment of oral mucosal lesion in the analysis of their results -15. In pNBI and erythro- leukoplakiaIn addition to the study concerning NBI and leukoplakia mentioned above , Yang etAs widely known, early detection of malignant and potentially malignant oral mucosal lesions plays a key role in improving patient prognosis. WL endoscopy may not be enough to detect specific neo angiogenic patterns suggestive of pre-malignant and malignant transformation. This review examined the diagnostic accuracy of NBI in comparison with traditional WL technique or as an independent test for oral malignancy. The papers included in this review tested NBI validity in different clinical contexts. The study about chronic oral non-healing ulcers - lesions poorly responding to medical treatment for at least 3 weeks - was performed by Yang et al  in 2014.We have encountered some limitations in this study. First of all, a large percentage of the selected studies has a retrospective design. Secondly, NBI examinations often were performed consecutively after conventional oral examinations (COE) and WL examinations so that NBI images were not blinded by previous WL evaluations. Thirdly, the results could have been affected by the endoscopist's lack of experience which, as seen, could increase the rate of false positive. Lastly, the study can present a chance of sampling error considering that not all lesions were subjected to a total excision but only to simple biopsy.The early diagnosis of malignant and pre-malignant lesions in the oral cavity is crucial to provide a better prognosis for patients. The purpose of this review was to evaluate the efficacy of NBI in the detection of oral squamous cell cancer. Based on current evidence, the study found that the use of NBI is strongly recommended for the early detection of oral cancer and potentially malignant disorders. In particular, this technique allows better visualization of malignancy features compared to only white light examination, especially in cases of precancerous lesions like erythroplakia and leukoplakia. However, further studies, especially about IPCL classification criteria, are necessary to make sure that future NBI examination could have a direct impact on patient pre-operative management. Indeed, we noticed that different authors often describe the same morphological features of IPCL using different nomenclatures. Therefore, it would be essential to refer to a single and common IPCL classification among the medical community. Although several studies show the great value of NBI in the diagnostic phase, there is still little evidence about its intraoperative role. That is the reason why the future challenge will be to consolidate this technique as a valid instrument able to guarantee a complete resection of oral malignant and pre-malignant lesions.


Scholars reported statistically significant gains in knowledge across three broad domains of biomedical research and seven distinct areas within TIE. Training in translational research incorporating immersions in clinical trials operation, biobanking, drug development, and community engagement adds value to career development of UREG researchers.North Carolina Central University (NCCU) and Duke Cancer Institute implemented an NCI-funded Translational Cancer Disparities Research Partnership to enhance translational cancer research, increase the pool of underrepresented racial and ethnic group (UREG) researchers in the translational and clinical research workforce, and equip UREG trainees with skills to increase diversity in clinical trials. The Cancer Research Education Program (C-REP) provided training for UREG graduate students and postdoctoral fellows at Duke and NCCU. An innovative component of C-REP is the Translational Immersion Experience (TIE), which enabled Scholars to gain knowledge across eight domains of clinical and translational research . Program-specific evaluative metrics were created for three broad domains  and eight TIE domains. Two cohorts ( Two key and interrelated challenges to cancer disparities research include the lack of investigators from underrepresented racial and ethnic groups (UREGs) contributing to cancer research and the underrepresentation of UREG participants in clinical trials . DespiteInadequate participation of diverse UREGs in clinical research and trials compromises generalizability of research findings and fosters biased reporting of both therapeutic and adverse effects that may differ by race, resulting in insufficient data to assess the efficacy or safety of new treatments and drugs . This diSimilarly, a significant disparity exists across the academic pipeline where UREG students are less likely to complete undergraduate or graduate degrees in the biological sciences. When they do matriculate, underrepresented students are less likely to pursue a path towards research independence . For exaThese factors highlight two urgent and compelling needs: (a) to provide vital translational cancer disparities research training opportunities for UREG trainees to diversify the research workforce and (b) to arm scholars with comprehensive training emphasizing clinical trials operational infrastructure, evidence-based strategies to increase diverse participation in biomedical research – particularly among UREGs, and link trainees to community engagement and outreach.The Cancer Research Education Program (C-REP), a core and required component of an NCI-funded P20 grant , was developed as an innovative educational collaboration between a Comprehensive Cancer Center, the Duke Cancer Institute (DCI), and a historically black university, North Carolina Central University (NCCU). C-REP addressed three critical needs in translational cancer disparities research and training: (a) increasing diversity in the cancer disparities research training pipeline and workforce, (b) training the next generation of translational scientists in clinical trials operations, processes, and strategies to increase UREG accrual, and (c) increasing researchers’ awareness, knowledge and skills in community outreach and engagement as a key component of the translational research spectrum.This 2-year program included traditional training and educational activities that exposed graduate and postdoctoral scholars to cancer disparities translational research, professional development opportunities , and cross-institutional mentoring. Trainees also attended two mentorship workshops that focused on improving communication, aligning expectations, and diversity and inclusion. Providing mentorship training around bias, inequities, and strategies to mitigate their effects was an innovative component of the program designed to increase retention among UREG trainees in the academic pipeline. Discussions of case studies provided opportunities for trainees to improve their skills in recognizing biases and prejudices, working across differences , and addressing issues of equity and inclusion within professional settings. Although C-REP was a required component of the P20 grant, the Translational Immersion Experience (TIE), a program within C-REP, was designed by the grant team to provide a series of interactive, experiential, and educational opportunities that enabled scholars to expand their skills and knowledge around the structure, processes, and roles within clinical trial operations, high-throughput drug screening, strategies to address lack of UREG participation in clinical research, and the importance and practice of community engagement and outreach within the translational research spectrum Fig. .Trainees who participated in C-REP included UREG doctoral students and postdoctoral fellows from NCCU and Duke. The program was structured around two cohorts of scholars. We conducted outreach campaigns using diverse platforms, including listservs, programs, workshops, and departments geared towards UREG graduate students and postdoctoral fellows across both institutions. Prospective candidates submitted applications outlining career and research goals, letters of recommendation, and a curriculum vitae. A review committee assessed applications based on each candidate’s career and research goals and their alignment with the program.The TIE was comprised of two summer sessions. Each cohort participated in the introductory TIE during the summer of their first program year. During the first summer, eight TIE sessions occurred every 2 weeks lasting 2 hours each. Facilitators included staff and investigators from both NCCU and Duke who were part of the larger P20 partnership. Scholars experienced a deeper dive into particular domains during the second summer session. In this communication, we describe the introductory TIE program  and evaluative findings for both cohorts.Introductory Seminar that consisted of an in-depth analysis of an active clinical trial through a disparities lens, which provided the framework and context of translational cancer disparities research across the spectrum. Scholars attended a Translational Research Lab Meeting where protocols were developed and discussed and all members of the research team attended and contributed, including the Principal Investigator(s). The Regulatory and Data Monitoring sessions allowed scholars to gain a deeper understanding of protocol development, regulatory requirements, and key operations around data collection, storage, and analysis.Scholars engaged in a series of educational sessions along the translational spectrum, including clinical trials operations, UREG accrual in clinical trials, and community engagement. Sessions were led by experts at NCCU, Duke, and industry leaders, such as representatives from contract research organizations and pharmaceutical companies, who were invited to share diverse perspectives from outside of the two institutions. The TIE sessions began with an Duke BioRepository and Precision Pathology Center and the Duke Early Phase Clinical Research Unit. This full experiential learning session started in the biobank where scholars learned what the biobank does and the methodology for storing tissue. Scholars also had an opportunity to experience preparing tissue for the biorepository. Scholars then toured the Duke Early Phase Research Clinic where they learned more about early phase studies and the resources required to operate the clinic and accelerate the availability of therapies, diagnostics, and medical devices to the broader population. Scholars attended a session on Clinical Trial Operations, which provided orientation to the clinical aspects of oncology research and included shadowing a variety of clinical research team members, including the physician, research nurse, and/or the clinical research coordinator, to gain an understanding of how clinical trials are operationalized. The High-Throughput Screening experience was a 2-day training at the Biomanufacturing Research Institute and Technology Enterprise (BRITE) Institute at NCCU where scholars learned about automation and liquid handling, detection devices, and data processing and software that has the ability to perform thousands of pharmacological experiments. Scholars received an overview and learned assay design for the purpose of drug discovery, which included assay development, optimization and validation, screening, and data analysis.Scholars also toured the Community Engagement and Diversity in Clinical Trial Session was both an immersion and educational session in which scholars learned the principles of community and patient engagement, strategies to address these issues along the translational spectrum, and gained a deeper understanding of disparities in cancer and the social determinants of health. Scholars learned barriers to diversity in clinical trials as well as novel and proactive strategies to address these barriers in their own work. Scholars also participated in two community outreach programs each year: the Men’s Health Initiative  and Women’s Health Awareness Day . Both events provided no-cost screenings and patient navigation services to attendees. Lastly, a Career Panel and Networking session was held where scholars were introduced to diverse careers in clinical research settings, including academic institutions, pharmaceutical companies, and contract research organizations. Collectively, this novel training program was designed to ensure scholars were immersed in key areas of clinical research operations and community engagement, and have a more comprehensive view of translational cancer disparities research and the impact they can have as research scientists in their field and the community.The A total of 14 C-REP scholars were selected: 7 were Black/African-American, 4 were Latinx/Hispanic, and 3 were Mixed Race/Other. Twelve were doctoral students and two were postdoctoral fellows (one at each institution). All C-REP Scholars (seven from NCCU and seven from Duke) were basic scientists.n = 13). In the results that follow, continuous data are presented as mean ± standard deviation (SD) in conjunction with results from paired sample t tests. Paired sample t tests indicate whether pre- and post-assessment scores were statistically significant, thus reflecting meaningful changes in self-reported knowledge and confidence. Significance was established at the .05 level.To evaluate the effectiveness of the TIE, program-specific evaluative metrics were created for three broad domains of biomedical science and all eight immersion topics. Two cohorts of C-REP Scholars completed an online pre-post survey during Summer 2018 and Summer 2019, respectively , knowledge of basic science and lab research (P < .05), and knowledge of population-based studies (P < .001). Two of three confidence metrics were statistically significant, including confidence in abilities to perform clinical operations  and basic science/lab research .As Table P < .05), operating clinical trials in a clinical environment (P < .001), data monitoring (P < .05), regulatory operations (P < .001), UREG accrual (P < .001), bio-banking (P < .001), and community engagement (P < .05). At the end of their summer immersion experience, trainees ranked their preferences for a deeper immersion in Year 2: approximately 38% of trainees ranked clinical immersion as their first choice for deeper immersion with both regulatory affairs and high-throughput screening each selected by 15% of trainees.Table During my graduate studies, I didn’t do community outreach but it’s something I’m leaning towards now. [TIE] is a really good exposure to community outreach.” Other trainees noted feelings of personal fulfillment in the wake of their participation in community outreach efforts, As one trainee elaborated, “The Men’s Health Day Initiative was great to see, especially how many people we were able to improve health outcomes for…it meant a lot to give back to the community in that way.” Another C-REP Scholar shared, “As a graduate student from an underrepresented group, I have developed a deep appreciation for participating in events where I can share my time and training to serve my community. This particular experience has shaped my perspective on how much more impactful these types of initiatives can be for patients/individuals when they are able to talk and ask questions to someone who looks and talks like them.”Community outreach was an important facet of training, noting the importance of understanding barriers and facilitators to increasing diverse participation and the role of community engagement around research participation – an area most trainees do not receive training in as they go through the traditional curriculum. Interestingly, gains in knowledge of community engagement were statistically significant while reported knowledge gains in community outreach were not. Yet qualitative data collected through interviews and observations suggest outreach events provided a meaningful opportunity to engage the public. One participant noted the program’s impact on his/her future career aspirations: “Trainee placements after graduation provide further support for positive program impacts on career development. C-REP trainees to date have successfully transitioned to positions within clinical research: one trainee is in a postdoctoral fellowship with the federal government, two accepted positions with contract research organizations, one is continuing educational training in the Duke Medical Scientist Training Program, and one NCCU Scholar was hired as a postdoctoral research associate at the Duke Cancer Institute. The postdoctoral fellow at Duke recently won the 2020 AACR-Genentech Cancer Disparities Research Fellowship. The remaining scholars will be completing their doctoral programs in the next 1–2 years. Taken collectively, C-REP scholars gave 17 conference presentations, 2 grants were awarded, 3 articles were published or accepted for publication, and 3 manuscripts were under review.Increased knowledge and exposure to contract research organizations, translational research, increasing diversity in clinical trials and community engagement were the principal goals in the program for Year 1. Through participation in the TIE, trainees built their knowledge of health disparities in translational cancer research, clinical operations, and community engagement, the latter being an area in which basic/bench scientists often have limited exposure or experience. Hands-on involvement in outreach events, such as the DCI’s Men’s Health Initiative, provided important opportunities for Scholars to experience firsthand the impact of their involvement in outreach efforts to vulnerable communities. The combination of training, hands-on experience in community outreach, and networking among established professionals has better prepared trainees to positively impact UREG accrual, diversity in the cancer disparities research workforce and pipeline, and engagement in community outreach as part of the translational research spectrum.With current patterns of uneven representation of UREG scholars across the academic pipeline, there is a pressing need for training programs that center around the experiences of UREG scholars. C-REP provides an innovative model for preparing underrepresented researchers to enter the translational research workforce by (1) providing mentorship training with an explicit focus on navigating issues of diversity and inclusion; and (2) providing exposure to the different phases of clinical operations. The C-REP TIE provides trainees with novel and tangible experiences and skills that set them apart from other peer researchers in biological sciences and expand their professional opportunities after matriculation.Preliminary results suggest that scholars were impacted in seven of the eight sessions within TIE. In the future, both cohorts will participate in a second rotation of TIE and gain more focused experience in one of eight areas. From this exposure, we predict that trainees will learn more about various aspects of clinical trial operations, gain more perspective about career options within clinical research, and ultimately determine viable career trajectories for themselves. As UREGs, C-REP Scholars will have a broader set of skills and knowledge in both clinical and translational cancer research and the importance of diversity in research participation and community engagement. These domains are not explored in the traditional training curriculum for basic science graduate students and postdocs and yet provide added value to the strengths each C-REP Scholar brings to the translational research workforce, further contributing to the elimination of cancer health disparities, and improving population and community health.While our program is focused on increasing UREG representation in the clinical and translational research workforce, we also see that key elements of this training could be beneficial for non-UREG trainees in providing skills for community engagement, clinical trial operations, and strategies for improving diversity in clinical trials. Future program iterations may incorporate project-based assignments, such as team science in health disparities and/or integrate more hands-on experience around increasing diverse participation in clinical trials. Future evaluations of similar initiatives could employ additional objective measures of professional development and career performance as well as quasi-experimental methods that compare measures of diverse study recruitment for early career investigators who participated in C-REP with those who did not.Overall, this program leverages significant strengths across both institutions, which includes (a) cutting-edge research that covers the translational spectrum, including strong community engagement programs and activities and a clinical trials platform at Duke and (b) NCCU’s Biomanufacturing Research Institute and Technology Institute (BRITE), offering hands-on education and training in key areas of drug development and biomanufacturing. Other resources that are important in adapting C-REP to other institutions are a strong network of mentors, an established network of biotechnology/contract research organizations, and dedicated staff time for implementation and evaluation. Together, these strengths are critical components that have led to a successful program and, to date, allowed Scholars to matriculate to the next level of their career in translational research.


About one in five Americans is engaged in providing care to a family member. Caregivers (unpaid family members or friends) support older adults and persons with disability with a variety of conditions, including challenges in physical, cognitive, and mental health. In California, 4.5 million family caregivers are assisting individuals over the age of 18. The CA Department of Health Care Services funds 11 Caregiver Resource Centers (CRC) to support caregivers and, in 2019, provided support to expand information technology services through adoption of a statewide online assessment platform and client portal, CareNav™, to serve as a client record and referral tool. CareNav™ facilitates collection of consistent state-wide data that can inform program improvement and policy. This study evaluated the implementation process from the perspective of 35 CRC team members in guided focus group discussions. CRC staff identified several potential benefits to adopting CareNav™, including ease of client access and convenience, the ability to aggregate data to inform planning and policy, and a streamlined process for resource sharing. Challenges included customizing site-specific data needs, as well as concerns about equitable access to internet services. Ongoing evaluation will focus on validation and visualization of data, and translation of data into actionable information to improve quality and reach.


These key developmental processes include neural progenitor cell proliferation, neuronal and oligodendrocyte differentiation, neural cell migration, neurite outgrowth, synaptogenesis and neuronal network formation, as well key events identified in the existing Adverse Outcome Pathways (AOPs). AOPs deliver mechanistic information on the causal links between molecular initiating event, intermediate key events and an adverse outcome of regulatory concern, providing the biological context to facilitate development of Integrated Approaches to Testing and Assessment (IATA) for various regulatory purposes. Developing IATA case studies, using mechanistic information derived from AOPs, is expected to increase scientific confidence for the use of in vitro methods within an IATA, thereby facilitating regulatory uptake. This manuscript summarizes the current state of international efforts to enhance DNT testing by using an in vitro battery of assays focusing on the role of AOPs in informing the development of IATA for different regulatory purposes, aiming to deliver an OECD guidance document on use of in vitro DNT battery of assays that include in vitro data interpretation.A major challenge in regulatory developmental neurotoxicity (DNT) assessment is lack of toxicological information for many compounds. Therefore, the Test Guidelines programme of the Organisation for Economic Cooperation and Development (OECD) took the initiative to coordinate an international collaboration between diverse stakeholders to consider integration of alternative approaches towards improving the current chemical DNT testing. During the past few years, a series of workshops was organized during which a consensus was reached that incorporation of a DNT testing battery that relies on  These ein vivo tests, the difficulty in interpreting results and the need of significant resources, contributing to the environmental release of chemicals that haven’t been ever tested for their DNT potential [in vitro DNT assays [in vitro assays within integrated approaches for testing and assessment (IATA) to address DNT and to assist with data interpretation of these assays and IATA.Other factors that impeded the broader use of DNT Test Guidelines are the complexity of the otential . In a nuotential . NotablyT assays  could beT assays ,21. In rT assays ,21, in 2in vitro assays to mechanistic knowledge, and b) the integration of multiple information sources not restricted to data only derived from the DNT in vitro battery (DNT-IVB). For this reason, the OECD chemical safety programme has been working over the last ten years towards the development of IATA that rely on combination of multiple layers of information  and that can be supported by mechanistic knowledge organised according the adverse outcome pathway (AOP) framework.Two critical aspects are important when developing guidance for internationally recognised non-animal-based approaches to address challenging endpoints such as DNT: a) the anchoring of https://aopkb.oecd.org/ and https://aopwiki.org/) that stores mechanistic information relevant to adverse health effects that would support development of mechanistically-informed and human-relevant alternative assays to be used for toxicity testing within the OECD Test Guidelines programme [in vitro assays and even beyond using the IATA framework [The AOP framework was introduced and adopted by the OECD to allow structured collection, organisation and evaluation of existing scientific knowledge and that could ultimately facilitate regulatory decision-making . This knrogramme . Anotherramework . In thisThis paper briefly summarises the current status of the OECD regulatory and scientific effort to develop guidance on the development and implementation of a DNT-IVB, leading to improved and faster chemical testing for this specific endpoint. The paper also discusses the considerations for regulatory uptake of the DNT-IVB and the overall role of the mechanistic information captured in AOPs, which should be incorporated into the IATA framework.2DNT is one of several toxicological  areas of high concern whose biological underpinnings are complex due to multiple neurodevelopmental processes which take place during brain development . Key neuin vitro DNT testing battery that permits quantitative evaluation of these key neurodevelopmental processes following exposure to chemical(s) was proposed in the OECD/EFSA workshop [in vivo. Disturbances of key neurodevelopmental processes are considered as important biomarkers of potential DNT-induced chemical effects at the cellular level. The readiness of the existing in vitro DNT assays for various regulatory application has been recently evaluated and, depending on the problem formulation, these assays will be incorporated into an IATA accordingly [Therefore, an workshop ,30 and cordingly .in vitro assays and data that permit quantitative evaluation (via concentration-response relationships) of the impact of a compound on various stages of brain development. Recently there has been an emphasis on using human in vitro neuronal cultures derived from neural progenitor cells (NPCs) as they are self-renewable and can be differentiated into several neuronal and glial cell types [in vitro critical brain developmental processes, including proliferation, migration and neuronal, astrocyte and oligodendrocyte differentiation as well as neuronal network formation and maturation measured by evaluation of synaptogenesis, myelination and neuronal network activity [in vitro test methods exists for studying these critical neurodevelopmental processes, whereas in vitro assays permitting evaluation of glia-specific processes, such as oligodendrocytes differentiation, myelin formation, radial and astroglia development, differentiation, microglia interaction/activation, and functional responses to chemical exposure are still not well developed or missing. Taking into consideration these gaps in the current DNT-IVB, CEFIC has recently published a call titled “Expansion of a regulatory accepted in vitro testing battery for developmental neurotoxicity (DNT) evaluation” , aiming to deliver the missing in vitro assays that could be incorporated into the existing DNT-IVB.Currently, rodent and human neuronal and glial cell models are available which can deliver a range of reliable ll types ,31. Humaactivity ,28,32. Ain vitro testing battery for the assessment of DNT [in vitro testing battery, consisting of 5 test methods measuring 10 DNT-specific endpoints and 9 viability/cytotoxicity-related parameters, was challenged with 119 chemicals for which rich toxicological information was available. The applied DNT testing strategy is promising since its performance demonstrated a sensitivity of 82.7 % and a specificity of 88.2 %, even with the known battery gaps.Another step forward towards the use of alternative approaches for DNT for regulatory purposes is the recently published EFSA report on the implementation and interpretation of an t of DNT . In thishttps://comptox.epa.gov/dashboard/chemical_lists/TOXCAST) and is expected to be finalised in 2021.All these research efforts aim to contribute towards the development of an OECD guidance on the interpretation and use of data derived from the DNT-IVB that can be applied to address relevant regulatory questions. Additional experimental work that will feed in this international effort is expected from the US Environmental Protection Agency (US EPA) that will be made available through the US EPA’s CompTox Chemicals Dashboard . To achieve this goal, it is useful to capture neurodevelopmental processes as AOPs, since the AOP framework allows compilation, structured organisation and evaluation of biological information demonstrating the causal links between steps in each process or pathway. In the AOP framework, these pathways are structured as beginning with the molecular initiating event (MIE), the initial perturbation triggered by a stressor , followed by subsequent perturbations of downstream key events (KEs) at cellular, tissue, organ and organism levels, representing a cascade of toxicity pathways potentially resulting in adverse outcomes (AO) of regulatory relevance [Design of testing strategies that would sufficiently cover the breadth of biological processes involved in neurodevelopmental disorders requires inclusion of a battery of elevance ,34. Indiin vivo, epidemiological information and any data coming from in silico, in vitro and other non-animal tests [Development of AOP-informed IATA starts with problem formulation , followed by gathering all available data relevant to the specific question, including available al tests ,28,35. Tin vitro assays should be anchored to selected set of CKEs [in vitro assays that allow an evaluation of the key neurodevelopmental processes specific for brain development but not yet defined as KEs in AOPs  if needed, should also be incorporated into such IATA.If new data are needed to inform the decision with the necessary certainty, the AOP network can guide experimental work to deliver required information. As a first tier, it is recommended that  of CKEs ,19 ident of CKEs , deliverin vitro DNT assays and other non-testing methods will vary. For instance, our current level of understanding supports the application of an IATA based entirely on a DNT-IVB of assays for chemical screening and prioritization, since these decisions tolerate a higher level of uncertainty. IATA for hazard identification and characterization should potentially include highly standardized in vitro assays and potentially be combined with read across and QSARs, driven always by the problem formulation. In the case of a decision bearing on risk assessment, information on exposure and ADME will need to be incorporated into the IATA. Zebrafish is the only relatively high-throughput non-mammalian animal model to study DNT that could potentially complement the DNT-IVB and provide additional data to be considered under IATA. A subgroup of the OECD DNT expert group is currently working towards standardization of zebrafish protocols and testing of significant number of compounds in 4–6 different laboratories.Based on the fit-for-purpose principle, the combination of different in vitro DNT assays were evaluated based on the (semi)-quantitative analysis of their readiness with respect to various use since none of these assays had previously been validated following classical EURL-ECVAM validation process. The scoring results suggest that several assays are currently at high readiness levels. Therefore, suggestions are made on how DNT in vitro assays may be assembled into an IATA, depending on the problem formulation [Taking into consideration these various regulatory applications , 17 mulation .learning and memory impairment in children/decrease of cognitive function” is identified as the most common AO among the currently available DNT AOPs . These AOPs are triggered by various MIEs that lead to different early KEs, but converge on CKEs close to AOs including neuronal differentiation, synaptogenesis or neuronal network formation and function [in vitro test methods based on human mixed neuronal glial cultures [in vitro evaluation of these CKEs are the first choice candidates for inclusion in IATA DNT-IVB. The current low number of DNT-AOPs concerning neurodevelopmental KEs should not prohibit the use of the battery, but might foster further development of DNT AOPs.The DNT-IVB  proposedfunction . Interescultures ,33. Ther4in vitro data derived from the testing battery together with other information sources by applying the IATA framework. These case studies will support the OECD DNT guidance by providing valuable context-specific examples. All case studies begin with problem formulations designed to address a number of regulatory needs by applying targeted DNT in vitro testing. In addition, the IATA case studies aim to elucidate how mechanistic understanding of chemical-induced DNT that is guided in most cases by AOPs can support the required decision-making. The case studies will cover the following scenarios: a) evaluation of DNT alerts that derive from computational models, b) screening and prioritization of chemical libraries with no DNT data, c) design of screening strategies to test small chemical classes, some of which have in vivo DNT data and others not, and d) individual chemical hazard assessment .Case studies are in development to illustrate the integration of in vitro assays. Finally, the broader use of the DNT-IVB will generate data that would allow the building of new AOPs and the expansion of the existing AOP network.It is expected that with the release of the guidance and the systematic use of the battery, additional experience will be gained and knowledge will be gathered to effectively address not only the above-mentioned but also additional regulatory scenarios. New IATA case studies that might be developed after completion of the OECD guidance document can further enrich the regulatory application of the DNT-IVB by shedding more light on DNT mechanistic understanding and by incorporating additional 5AOP networks relevant to neurotoxicity have recently been published , and sevimpairment of learning and memory in children/decrease of cognitive function” and can be triggered by diverse MIEs, including binding of antagonists to NMDA receptor (AOP ID 12 and 13), binding of agonists to NMDA receptor (AOP ID 48), inhibition of the thyroid peroxidase (TPO) (AOP ID 42) or inhibition of the sodium/iodide symporter (NIS) (AOP ID 54). These MIEs trigger different early KEs but, closer to AO, they converge on common KEs (CKEs), such as alteration of synaptogenesis and neuronal network function. The AOP network described by Li et al. identified additional CKEs including mitochondrial dysfunction, cell injury/neurodegeneration, altered differentiation of neural stem/precursor cells into neurons and glia cells, and decreased T4 levels in neuronal tissue [The AOP network published by Spinu et al.  covers Al tissue .In vitro batteries of assays anchored to CKEs which are triggered by multiple MIEs may serve as simplification for the activation of multiple AOPs. By choosing those CKEs as testing priorities, the number of assays can be reduced while still covering multiple AOPs covering a variety of MIEs triggered by different classes of chemicals. This rational was applied for evaluation of DNT effects induced by mixtures of chemicals using a set of in vitro assays, anchored to CKEs of several AOPs with the common AO of “learning and memory impairment in children” [hildren” ,40.e.g., AOP 54 in the AOP Wiki) contain KEs that are relevant to DNT and should be included for consideration when designing a battery of in vitro assays.Although endocrine disruption-based DNT AOPs are outside of the scope of the OECD guidance document under development, the thyroid disruption-triggered AOPs ; attention-deficit/hyperactivity disorder (ADHD); intellectual disability ; conduct disorders; cerebral palsy; impairments in vision and hearing; motor disorders including developmental coordination disorder and stereotypic movement disorder; communication, speech and language disorders; foetal alcohol spectrum disorder, open neural tube defects, in vitro test batteries, which processes are still not being covered and for which processes assays should be developed.Having a comprehensive network of DNT AOPs that covers most major neurodevelopmental diseases will allow the identification of a more comprehensive set of CKEs. It would also clarify to what extent these diseases are covered by the existing AOPs, CKEs and associated in vivo test guidelines do not predict neurodevelopmental disorders [Neurosome (https://www.neurosome.eu/) and the Japan Environment and Children’s Study (https://www.env.go.jp/chemi/ceh/en/index.html), when organised using the AOP framework, will further improve our ability to both understand the effects of and to better regulate chemical exposure through the use of alternative approaches for DNT testing.The current isorders , nor do 6Neurodevelopmental disorders are multifaceted conditions characterized by impairments in cognition, learning disabilities, speech and language disorders, poor social development, behaviour and/or motor skills resulting from abnormal brain development. Intellectual disability, communication disorders, ASD, ADHD, epilepsy and schizophrenia fall also under the umbrella of neurodegenerative diseases (NDDs) where a gene-environmental interplay is involved.Moreover, many symptoms are not unique to a single NDD, and several NDDs have clusters of symptoms in common. For example, impaired social cognition is common to ASD and schizophrenia. Furthermore, NDDs may have diverse pathophysiology that underlies similar clinical phenotypes, or conversely, diverse clinical outcomes may result from similar pathophysiology. For example, ASD is a neurodevelopmental disease with an increasing incidence and is more prevalent in males . Howeverhttps://aopkb.oecd.org/ and https://aopwiki.org) . Others have been submitted to the AOP Wiki  or published in peer review journals [https://www.oecd.org/chemicalsafety/testing/projects-adverse-outcome-pathways.htm) and are expected to be submitted to the AOP-Wiki and reviewed in the near future [Currently, there are only a few AOPs relevant to DNT or adult neurotoxicity available that are already endorsed by Working Party of National Co-ordinators of the Test Guidelines Programme and the Working Party on Hazard Assessment  triggered by diverse MIEs including the interaction of chemicals with specific receptors and enzymes. There is also an AOP resulting in neurological dysfunction and hearing loss (AOP 8) triggered by upregulation of thyroid hormone catabolism (https://aopwiki.org/wiki/index.php/Aop:8), and a few AOPs published describing neurological and cognitive impairment triggered by interferences with thyroid hormone synthesis, metabolism and transport [Although there are only 7 fully OECD endorsed AOPs, the existing network of AOPs covers already biological space critical to several adverse outcomes and identifies CKEs that can contribute to both hypothesis generation and the design of IATA for DNT assessment. A few DNT AOPs have been developed referring to impairment of learning and memory/cognitive damage in children  [The analyses of modes-of-action of DNT compounds as well as of the pathways contributing to impairment of neurodevelopmental processes showed that multiple pathways involved in neurodevelopmental disorders merge into most of these processes. These are summarized in ndromes) ,21.Tablein vitro and in vivo data from multiple sources as well as a set of computational workflows to assist with chemical activity characterization [Further insight into chemical modes of action could be facilitated by tools being developed by the US National Toxicology Program Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM), including the Integrated Chemical Environment (ICE) and the Developmental NeuroToxicity Data Integration and Visualization Enabling Resource (DNT-DIVER). ICE provides an interactive dashboard for accessing curated in silico, rization . DNT-DIVrization .The most studied DNT compound is lead and indeed, there is already an AOP endorsed by OECD working parties of the chemical safety programme where lead is described as a trigger of the MIE defined as “binding of antagonists to NMDA receptor . One of Synaptogenesis is also affected by exposure to chlorpyrifos  and orgaSeparate studies are required to link potential DNT compounds with perturbed signalling pathways described in in vitro assays.Ideally, all biological pathways leading to neurological and developmental neurological pathologies would be known and described in the form of AOPs. Without knowing the full extent of coverage, it is possible that the existing AOPs cover only limited DNT-relevant biological space and mechanisms of concern. Therefore, it is desirable to gather further mechanistic understanding of neurological diseases captured as AOPs, to explicitly describe and link to major neurodevelopmental adverse outcomes not yet covered. Further development of AOPs will inform about the mechanistic space that is potentially not assessed by the current proposed DNT-IVB and identify KEs as anchors for development of additional As mentioned earlier, development of AOPs for complex neurodevelopmental disorders such as ASD or ADHD is challenging because of (1) a general lack of understanding of the contribution of biology, including genetic and environmental factors to these disorders, (2) a lack of identification of specific pathways that might be triggered, including potential MIEs and causally linked KEs that lead to these outcomes. The molecular and cellular mechanisms of many neurodevelopmental disorders are largely unknown and in some cases, features/symptoms of these diseases are overlapping since they are not specific for one particular disease. Therefore, the complex aetiology and limited mechanistic understanding of neurodevelopmental disorders make identification of KEs  extremely difficult. Additional difficulty arises from the dynamic processes of brain development, which can be affected differently by the same chemical, depending on the window of exposure and the susceptibility of specific brain regions and various neuronal or glial subtypes at different times during development.e.g., e.g., mitochondria impairment, oxidative stress, dysfunction of ion channels, receptors, changes in neurotransmitter synthesis/release, increased proinflammatory cytokines production, etc.) could be used as a precautionary screen, irrespective of identification of a particular pathway or AO. A battery of more specific CKEs could be used to identify more specific pathways or developmental stages , or network or pathway specific KEs could then be used to identify affected pathways with increasing specificity . The current DNT-IVB included in the OECD guidance document is based on assays that mostly cover this biological space but is not considered exhaustive.One advantage of focussing on AOPs describing basic neurodevelopmental processes is to make sure that the testing battery covers these fundamental processes, increasing the likelihood that any neurodevelopmental effect will be captured, even if it cannot yet be explicitly described in an AOP and linked to critical KEs or AOs. Another advantage of the AOP framework is that, as biological information is added and AOP networks evolve, testing strategies could be designed for different purposes or levels of specification e.g., . A testietc., it is important to increase the mechanistic understanding to make sure the underlying mechanisms can be covered by available testing approaches. Therefore, when developing future DNT-related AOPs, the systematic organization of available information and the identification of critical knowledge gaps are equally important. In vitro assays anchored to KEs can deliver relevant molecular and cellular information which would contribute to better mechanistic understanding of pathways involved, especially when using human-cell based test methods, contributing to further AOP development relevant to neurodevelopmental disorders.For diseases not covered by current AOPs, such as ASD and ADHD 7e.g., cerebrospinal fluid) or information gathered from functional MRI could potentially, in the near future, reveal reliable biomarkers of effects that could further confirm and provide stronger evidence supporting the DNT-relevant AOPs. Most importantly, studies that provide quantitative human data would identify factors that modify potential outcomes; these factors could then be described quantitatively in KER for eventual modelling, for example to predict susceptible individuals.The AOP framework provides the opportunity for explicitly describing both the common and unique molecular features of the various developmental stages of the nervous system. Developing AOP networks reflecting the complexity of the key neurodevelopmental processes that are anchored to phenotypic adverse outcomes seems a challenging task. However, emerging experimental data can provide significant empirical evidence and support to AOP building and reveal better understanding of chemicals’ involvement in manifestation of DNT relevant adverse outcomes. In parallel, epidemiological studies complemented by ‘omics’ generated data from patient samples  AOP network has been created as part of the IATA case study ,60, and AOP-supported IATA can address various regulatory and non-regulatory problem formulations that range from chemical prioritisation to risk assessment. Developing IATA case studies using focused problem formulations supported by mechanistic information from AOPs, with clear recognition of uncertainties involved, can ultimately increase the uptake and gain support from the regulatory community.The opinions expressed and arguments employed herein are those of the authors and do not necessarily reflect the official views of the OECD or of the governments of its member countries.The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.


Research networks offer multidisciplinary expertise and promote information exchange between researchers across Europe. They are essential for the European Union’s (EU) health information system as providers of health information and data. The aim of this mapping exercise was to identify and analyze EU research networks in terms of health data collection methods, quality assessment, availability and accessibility procedures.A web-based search was performed to identify EU research networks that are not part of international organizations  and are involved in collection of data for health monitoring or health system performance assessment. General characteristics of the research networks , quality assessment procedures, availability and accessibility of health data were collected through an ad hoc extraction form.Fifty-seven research networks, representative at national, international or regional level, were identified. In these networks, data are mainly collected through administrative sources, health surveys and cohort studies. Over 70% of networks provide information on quality assessment of their data collection procedures. Most networks share macrodata through articles and reports, while microdata are available from ten networks. A request for data access is required by 14 networks, of which three apply a financial charge. Few networks share data with other research networks (8/49) or specify the metadata-reporting standards used for data description (9/49).Improving health information and availability of high quality data is a priority in Europe. Research networks could play a major role in tackling health data and information inequalities by enhancing quality, availability, and accessibility of health data and data sharing across European networks.The online version contains supplementary material available at 10.1186/s13690-021-00766-2. Research networks (RNs) offer multidisciplinary expertise and promote information exchange between researchers across Europe and extra-European countries. They are essential for health information systems as providers of health information and data, collaborating with various sectors, such as government, industry, academia and independent research groups. Many RNs include in their activities the collection of data and information from multiple verified sources into compiled databases, following standardized procedures. Therefore, their data and information could be considered more reliable compared to those from individual sources. Among the advantages of RNs are data collection and sharing, collaborative research across different geographical areas, strengthening research capacities and quality by conducting research according to standardized methods and practices, and provision of training courses for network members and the wider community. Ultimately, collaboration in RNs enhances the productivity of individual researchers , 2.There are various definitions of a network and types of RNs. A network is generally a relationship between three or more individuals or groups characterized by shared objectives to achieve common goals . NetworkThe present study is part of the Joint Action (JA) on Health Information InfAct (Information for Action), that was launched in 2018 and will be active until 2021. The JA involves 40 partners from 28 EU and 4 associated countries working together towards a sustainable infrastructure for the European Union’s (EU) health information to support evidence-based policy and research activities. The purpose of this mapping exercise is to identify and evaluate EU RNs that are not part of international organizations  and are involved in collection of data for health monitoring or health system performance assessment.European research network(s) AND health information, and the first 10 pages were analysed. Additional RNs were identified through a cross-sectional study addressing InfAct project partners on health data collection methods and procedures across EU Member States (EU MS). The study led to the identification of EU research projects, which were also part of EU RNs. EU RNs that are not part of international organizations  and are involved in data collection for health monitoring or health system performance assessment are included in the present study. The websites of the identified networks were then evaluated according to the following sections of an ad hoc extraction form:i)General characteristics ;ii)Quality assurance ;iii)Data availability ;iv)Data accessibility .For the purpose of this study, a RN is defined as a project involving at least two institutions or stakeholders in a country  or institutions/stakeholders in at least two countries . RNs were retrieved through a web-based desk research between April and June 2019. The desk search was conducted using publicly available information on the European Commission’s Community Research and Development Information Service (CORDIS) database and on the websites of international organizations . A Google search was also performed using the terms Regarding quality assurance, the RNs were assessed by four researchers independently through 10 quality dimensions or criteria , population-based surveys or interviews, longitudinal or cohort studies, population-based disease registries , and medical records . The principal area of research for most networks was health monitoring (32/57) and, to a lesser extent, health system performance assessment and monitoring (8/57).The health topics or diseases considered by the RNs included, but were not limited to, non-communicable diseases , unhealthy lifestyles, non-fatal injuries, environmental hazards and urban health, health system performance, healthcare utilization, health inequalities, and health promotion and interventions. According to the main areas of research, various risk factors, high-risk conditions or health behaviors were assessed by the RNs; namely, behavioral, environmental, socio-economic, and disease-specific risk factors. The indicators elaborated from the collected health data included prevalence, incidence, outcome and performance measures, attack rates, injury disability indicators, and more.Data sharing with other projects or RNs was not in place for most networks (40/49). On the contrary, data sharing was in place for eight RNs Table  and in pMore than 40 RNs are funded or co-funded by the EU Commission; funds are also provided by Ministries of Health, research councils and various institutes from the participating countries. Five RNs were supported by private foundations .Most RNs 41/56) performed and reported the quality assessment procedures of the collected health data on their website. Quality assessment was not applicable to the Commonwealth Fund Multinational Comparisons of Health Systems Data (MultiCom) which uses data collected and processed by OECD. Out of 49 RNs providing health data on their website, 39 RNs provided only macrodata through reports and/or scientific articles, while only microdata was provided by two networks . Both microdata and macrodata were available from eight RNs . Moreover, health data was not reported on the website of eight RNs . Further data analysis, such as aggregation or stratification, was possible for 16/47 RNs providing macrodata. The metadata reporting standards used for health data description were specified by 9 out of 49 RNs. These standards were: International Classification of Diseases  performe; EurostaHealth data provided by 40 RNs was accessible as macrodata through guidelines, reports and scientific articles; microdata was not provided in open access. A formal request for microdata and/or macrodata access was required by 14 networks, of which three applied a financial charge . Data access was usually granted by scientific, ethics, steering or management committees. Overall, data provided by 34 RNs, out of 49, was reusable based on data usage licences .The mapping exercise allowed the identification of EU RNs that were evaluated in terms of collection methods, quality assessment, availability and accessibility of health data. Most RNs used various population health data sources to compile databases covering different health topics. In this light, the data produced by RNs could be considered more reliable and relevant for the development of evidence-based interventions and policy measures compared to single datasets of individual researchers. Accurate and reliable information is the keystone to policy planning and scientific research.Health information systems collect public health data, analyse and convert data into information for policy-making, ensuring data quality, relevance and timeliness . The staAccess to health data, mostly microdata, is granted by various types of committees. This finding is an indicator of the compliance to the General Data Protection Regulation (GDPR) on data protection and privacy in the EU and the European Economic Area (EEA) . CompliaAnother critical aspect of health information is the fragmentation of health research, which is related to the lifespan of RNs. Indeed, barely 8 of 57 identified RNs are still active. The reasons behind the disruption of research activities are mainly financial resources, relevance and discipline of the projects [Sipido 2020]. In times of financial constraints, resources have to be reallocated at the expense of some research activities. Research based on collaborations across a wide range of disciplines, through an extensive network of researchers, have higher probability to be funded compared to independent scientists or small research groups .The list of RNs included in the mapping exercise was not exhaustive. However, the aim was not to list all existing RNs in EU but to perform a qualitative analysis of identified RNs that could highlight the barriers and facilitating factors related to research networking in EU MS. Moreover, only RNs that are not part of international organizations  and involved in health data collection for health monitoring and health system performance were considered. This surely limited the number of possible networks that could be included in the study. However, the web-based desk search was enhanced with additional information provided by the partners of the InfAct JA.The critical issues related to data quality, availability, accessibility and data sharing underlined by the current findings pose a serious challenge to the scientific advancement and sustainability of the EU information system. Adherence to guidelines and protocols on standardized procedures in data collection and analysis may ensure the comparability and reusability of research results. Moreover, the development of extensive and multidisciplinary RNs could facilitate the optimal allocation of research funds and prevent the fragmentation of research activities.RNs are essential for the health information system in EU as providers of accurate and reliable health data and have an important role in information exchange between researchers across and outside the EU, and in providing reliable health information for evidence-based health policy decisions. Therefore, RNs could tackle health data and information inequalities by enhancing quality, availability, and accessibility of health data and data sharing across geographical regions.Additional file 1. Brief description of the identified research networks. Description of the research networks.


This paper assesses variation in rates of emergency surgery (ES) amongst emergency admissions to hospital in patients with acute appendicitis, cholelithiasis, diverticular disease, abdominal wall hernia, and intestinal obstruction.Records of emergency admissions between 1 April 2010 and 31 December 2019 for the five conditions were extracted from Hospital Episode Statistics for 136 acute National Health Service (NHS) trusts in England. Patients who had ES were identified using Office of Population Censuses and Surveys (OPCS) procedure codes, selected by consensus of a clinical panel. The differences in ES rates according to patient characteristics, and unexplained variations across NHS trusts were estimated by multilevel logistic regression, adjusting for year of emergency admission, age, sex, ethnicity, diagnostic subcategories, index of multiple deprivation, number of co-morbidities, and frailty.versus those aged 45–49 years: 0.34 (appendicitis), 0.49 (cholelithiasis), 0.87 (hernia), and 0.91 . Patients with diverticular disease aged 75–79 were more likely to receive ES than those aged 45–49 (OR 1.40). Variation in ES rates across NHS trusts remained after case mix adjustment and was greatest for cholelithiasis .The cohort sizes ranged from 107 325 (hernia) to 268 253 (appendicitis) patients, and the proportion of patients who received ES from 11.0 per cent (diverticular disease) to 92.3 per cent (appendicitis). Older patients were generally less likely to receive ES, with adjusted odds ratios (ORs) of ES for those aged 75–79 For patients presenting as emergency hospital admissions with common acute conditions, variation in ES rates between NHS trusts remained after adjustment for demographic and clinical characteristics. Age was strongly associated with the likelihood of ES receipt for some procedures. This study reports variation in rates of emergency surgery (ES) across National Health Service trusts amongst patients presenting as emergency admissions to hospital with acute appendicitis, cholelithiasis, diverticular disease, abdominal wall hernia, or intestinal obstruction, after adjustment for differences in patient-level characteristics. Older patients were less likely to have ES, after allowing for differences in other patient characteristics, including number of Charlson co-morbidities, level of frailty, and diagnostic subcategory. The decline in the rate of ES with increasing age was greatest for patients presenting with acute appendicitis and cholelithiasis. The study involved secondary analyses of existing pseudo-anonymized data and did not require UK National Ethics Committee approval. The study drew from the findings of two workshops with patients and the public, held in July 2020, that reported it was of potential benefit to patients and the public to examine why access to ES might vary according to different patient groupsS2) that was deemed relevant according to the consensus of a clinical panel; was within an emergency admission through the emergency department or was from a primary care referral; was under a consultant general surgeon, subspecialty general surgeon, or surgeon working in the general surgery specialty; and was the first or second episode within the admission. For the intestinal obstruction cohort, a relevant diagnosis could appear in the second diagnosis field if the main diagnosis was colorectal cancer. An admission was excluded if there had been a prior emergency admission with a relevant diagnosis in the previous 12 months, or further diagnostic exclusion criteria were met according to the consensus of a clinical panel (see Table S2).An admission can comprise several finished consultant episodes, and patients aged 18 or over were eligible for inclusion in the study cohort if a finished consultant episode met the following criteria: occurred between 1 April 2010 and 31 December 2019; included a main diagnosis with an ICD-10 diagnosis code  codes, and the maximum number of days within which the surgery had to occur to constitute ES were defined by consensus of the study’s clinical panel , sex, ethnicity, Index of Multiple Deprivation (IMD), diagnostic subcategories, number of co-morbidities, and frailty. The Charlson co-morbidity indexversus the volume of emergency admissions within NHS trusts.Summary statistics were used to describe patients’ demographic and clinical characteristics. Age was categorized into 5-year age bands. The proportions of eligible emergency admissions were calculated for each cohort overall, and according to prespecified subgroups of interest. For each condition, multilevel logistic models were developed which included year of emergency admission, age, sex, ethnicity, diagnostic subcategories, IMD (quintiles), number of Charlson co-morbidities, and SCARF frailty index as independent variables, and whether the patient received ES or not as the dependent variable. The unit of analysis was the emergency admission. The multilevel model included random intercepts for each NHS trust to allow for clustering and to report the level of unexplained trust-level variation in ES, after allowing for patient factors and time period. The model was used to predict the case mix adjusted odds ratios  of ES associated with patient factors, and the levels of unexplained variation attributable to NHS trusts. Funnel plots were used to display the variation in case mix adjusted proportions receiving ES Table S4). Table 1 presents the patient characteristics for each cohort of emergency admissions that met the study’s inclusion criteria. The numbers (proportions) of patients in each diagnostic subcategory are listed in Table S5.The number of patients who were eligible as emergency admissions were: 107 325 (hernia), 137 744 , 139 090 (diverticular disease), 241 626 (cholelithiasis), and 268 253 (appendicitis) (Table S4). The most common ES procedures are listed in Table S6. The proportion of patients who received ES was generally lower for older age groups (Table 2). Women were more likely than men to receive ES for cholelithiasis, hernia, and intestinal obstruction. For all five conditions, patients with co-morbidities were less likely to receive ES than those without. The proportion of patients who had ES increased with frailty for patients with diverticular disease and intestinal obstruction but decreased for those with appendicitis and hernia.The proportion of patients who had ES was highest for acute appendicitis (92.3per cent), and 57.9 per cent for hernia, 29.9 per cent for intestinal obstruction, 21.5 per cent for cholelithiasis, and 11.0 per cent for diverticular disease  to 0.68 (hernia). The decline in ES rates with increasing age was sharpest for appendicitis and cholelithiasis. For patients with diverticular disease, ES rates were higher for patients aged 60–80 years . After adjusting for frailty and other factors, patients with any of the five conditions were less likely to have ES if they had co-morbidities. The association of frailty with ES rate differed by condition. There was a consistent decline in the rate of ES as the number of Charlson co-morbidities increased, but the association between frailty and ES rate was less consistent. The relationship was strongest for patients with acute appendicitis or hernia; those patients with more severe frailty were less likely to receive ES. Patients at any level of frailty who presented with cholelithiasis, diverticular disease, or intestinal obstruction were more likely to receive ES (Table 3). Investigation of interactions between co-morbidity and frailty showed that in diverticular disease and intestinal obstruction, the effect of frailty was strongest in patients with no Charlson co-morbidities (Table S7).Fig. 2 shows that variation in ES rates across NHS trusts remained after case mix adjustment. Rates of ES between trusts were positively correlated for all conditions and highest between appendicitis and cholelithiasis (r = 0.33). The level of unexplained variation did not appear related to the volume of emergency admissions for the respective condition within each trust. The estimated proportion of unexplained variation at the level of the NHS trust, rather than of the patient, was highest for cholelithiasis, with intraclass correlation  of 0.169 (0.137 to 0.205), followed by appendicitis , hernia , diverticular disease , and intestinal obstruction .Before case mix adjustment, the overall variation in ES rates across NHS trusts was greatest for cholelithiasis  and hernia , followed by intestinal obstruction , appendicitis , and diverticular disease . This report describes variation in rates of ES across NHS trusts for patients presenting as emergency admissions to hospital with acute appendicitis, cholelithiasis, diverticular disease, abdominal wall hernia, or intestinal obstruction. This variation remained after adjustment for differences in patient-level characteristics and the time period of the emergency admission, and was greatest for patients admitted as an emergency with a diagnosis of acute cholelithiasis. The study also reported wide differences in ES rates according to age group. Older patients were less likely to receive ES, after allowing for differences in other patient characteristics, including number of Charlson co-morbidities, level of frailty, and diagnostic subcategory. The decline in the rate of ES with increasing age was highest for patients admitted as an emergency with appendicitis and cholelithiasis.versus delayed cholecystectomy for patients with biliary colic, acute cholecystitis, and gallstone pancreatitisThe study found differences amongst the five conditions in the levels of unexplained variation across NHS trusts in ES rates. The highest level of unexplained variation in ES rates was for patients presenting with acute cholelithiasis, which suggests that some trusts did not follow the National Institute for Health and Care Excellence (NICE) guidelines recommending laparoscopic cholecystectomy within 7 days of diagnosisversus delayed cholecystectomy, or the enthusiasm to deliver an emergency cholecystectomy serviceThe unexplained variation in ES rates for patients with acute cholelithiasis was obtained after adjusting for the annual volume of ES procedures performed in each trust and may reflect differences in the level of surgical expertise and resource availability across NHS trusts. Previous research on trust-level variation for patients with benign gallbladder diseases reported higher rate of ES in those centres with a specialist hepatobiliary centre available, which may reflect better availability of operating theatre space, clearer understanding of the evidence comparing emergency versus NES for these conditions, that there are fewer well defined care pathways, and a lack of clinical guidelines in the UK to inform the choice of whether or not a patient has ES,,,For acute appendicitis and abdominal wall hernia, the level of unexplained variation in ES rates across trusts was moderately high and may reflect the lack of evidence on which patients benefit from ES For patients with intestinal obstruction and diverticular disease, the variation in ES rates across trusts was relatively low, which, for diverticular disease, may reflect increased standardization in clinical management of the condition over the study period, that RCTs were undertaken, and that these required clinical pathways to be developedversus those aged 65–74, notably for patients with acute cholelithiasis,,versus those who did not, after adjustment for some case-measuresversus patients aged 65–74,This research extends previous studies which have generally found lower rates of ES for specific subgroups of older patients presenting with acute conditions in the UK,Previous reports comparing the rate of ES across age groupsThis study has several strengths. First, the study considered all eligible emergency admissions from 136 acute hospital trusts across a 10-year period. By adopting these broad inclusion criteria, the study had a representative sample of emergency admissions that was sufficiently large to draw inferences about the association of a multitude of routinely measured patient factors with receipt of ES. Second, unlike previous comparisons of ES rates across areas and patient demographicsversus NES assumes accurate coding of OPCS procedures and episode dates. It is conceivable that there were coding differences across NHS trusts; for example, some trusts could code patients with umbilical hernia as ventral hernia, and vice versa. However, this would seem unlikely to explain differences in ES rates of the magnitude reported. Third, this paper does not contrast outcomes between patients who received ES versus those who had NES, or according to NHS hospital trusts with different rates of ES. The paper does not therefore attempt to define the optimum level of ES versus NES for each condition.The limitations of this paper include the following. Detailed information on patients’ acute condition, for example information on their physiology, and for some conditions  the lack of information from imaging could mean that differences in the true diagnosis or severity of the condition may explain some of the variations in ES across NHS trusts or patient subgroups. Other unmeasured variables which could be important in helping understand these variations include patient preferences and lack of emergency theatre capacity for these conditions within the local healthcare systems. A further challenge is that the categorization of ES versus NES strategies for subgroups of patients presenting as emergency admissions with acute conditions. The ESORT study will use the variation in ES rates across NHS trusts and hospitals to assess the effectiveness and cost-effectiveness of ES for each of these five conditions,This paper therefore highlights important areas for further research. In particular, given the wide variations in ES rates reported, there is a clear requirement for evidence on the effectiveness of ES The ESORT study is funded by the National Institute for Health Research Health Services and Delivery Research (project number 18/02/25). This report is independent research supported by the National Institute for Health Research ARC North Thames. The views expressed in this publication are those of the author(s) and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care.zrab094_Supplementary_DataClick here for additional data file.