Falls have become a major public health issue in China with population aging. Although falls prevention for older community-dwelling people has been included in the National Essential Public Health Service Package since 2009, there is limited understanding of the implementation of this program.To identify the associated factors and provide recommendations to inform the better implementation of falls prevention in the Chinese primary health care system.This qualitative study was conducted in 3 purposively selected cities in China from March 1 to June 7, 2021. Health administrators from the local health commission or bureau, staff members from local Centers for Disease Control and Prevention and primary health care facilities and community-dwelling older people were recruited, using a combination of purposive sampling and snowball sampling.In-depth interviews were conducted with health administrators and focus groups with other participants. Data analysis followed the guidance of the Consolidated Framework for Implementation Research. Study outcomes included facilitators and barriers of implementing falls prevention for older people in the Chinese primary health care settings. A framework with recommendations was developed to inform the future intervention implementation.Among a total of 130 participants interviewed, 77 (59.2%) were female and the mean (SD) age was 47.4 (16.7) years. Clear recognition of the challenges and benefits of falls prevention, adaptive regionally tailored guidance plans, and continuous governmental policy and financial support were the major facilitators, whereas the major barriers consisted of insufficient confidence in delivering interventions and poor understanding of the falls burden, low recognition of the importance of falls prevention, limited multisectoral collaboration, and weak financial incentives. A 7-strategy embedded framework—including data-driven surveillance, audit and feedback, implementation strategy, workforce strengthening, community empowerment, internal services integration, and external enabling environment—was developed to foster successful implementation.This qualitative study identified major facilitators and barriers to the implementation of falls prevention for older people at the primary care level, which have the potential to contribute to better implementation of falls prevention for older people in the Chinese primary health care system. This qualitative study identifies facilitators and barriers to implementation of falls prevention programs in the Chinese primary health care system and provides recommendations. What are the facilitators and barriers of implementing falls prevention for older people in Chinese primary health care (PHC) settings?This qualitative study of 130 participants identified 14 facilitators and 20 barriers in implementing falls prevention for older people in PHC settings. A recommendation framework was then developed to respond to those barriers.This study’s findings suggest the challenges of implementing falls prevention for older people in Chinese PHC settings and also provide a high-level framework to inform future better implementation. This study, therefore, aims to identify the associated factors, and further provide strategies for future better implementation of falls prevention.In 2009, the Chinese Central Government launched the National Essential Public Health Service Package (NEPHSP), with the primary aim to strengthen the primary health care (PHC) system.12: Chang’an District of Shijiazhuang City from Hebei Province, Beilun District of Ningbo City from Zhejiang Province, and Longhua District of Shenzhen City from Guangdong Province (COREQ) reporting guideline was used to guide the reporting of this study  and PHC institutions. They were eligible if they had worked more than 3 months in their organizations, were responsible for or providing technical support for falls prevention in the NEPHSP and willing to sign informed consent. Older residents aged at least 65 years as consumers were eligible if they lived in the community under the jurisdiction of selected PHC institutions and were willing to provide informed consent, whereas those with serious medical conditions not suitable for interviews were excluded. Written informed consent, including permission to be audio recorded, was obtained before commencing the interviews and discussions. A combination of purposive sampling and snowball sampling was adopted to recruit participants  is consistent with previous recommendations.18In-depth interviews were conducted with health administrators, mainly because of scheduling difficulty and ensuring privacy so that participants were able to share views freely, by 1 interviewer (P.Y.) and 2 notetakers (Y.E. and Y.J.). Focus group discussions were used for the other 3 stakeholder groups by 1 moderator (Y.E.) and 2 notetakers (P.Y. and Y.J.). The interview and discussion guides were developed based on the Consolidated Framework for Implementation Research (CFIR)  independently reviewed the transcripts and inductively generated preliminary codes about influencing factors around emergent concepts. Second, similar preliminary codes were grouped into themes and then deductively mapped to the CFIR. A coding framework was generated using constant comparison to establish a hierarchy of conceptual codes until no new themes were identified. The coding discrepancies were discussed and resolved by all research team members to optimize intercoder reliability. For the strategy design, suggestions from interviews were inductively summarized, linked to the implementation strategies compiled by the Expert Recommendations for Implementation Change (ERIC) project using the CFIR-ERIC matching tool,A total of 130 participants  were invited for either in-depth interviews (6 interviews with 50 to 70 minutes for each) or focus group discussions (19 discussions with about 6 participants in each session lasting for 90 minutes). About half of the service professionals have worked more than 10 years in their positions, with 12 service professionals working in age-specific areas for more than 10 years . FacilitAs a government-funded service package, NEPHSP has a well-established region-tailored guidance plan refined on an annual basis. Service professionals reported 3 major challenges to fall-prevention implementation: (1) the large number of older residents to be served, (2) insufficient PHC clinicians, and (3) the low acceptability of interventions among older people. “Six PHC providers needed to serve more than 3000 older residents. Some older people complained that fall-prevention services cost their grandchild-care time without benefit” (PHC clinician 0104).Specific barriers are listed in In addition, there were no performance assessment indicators and dedicated budget for falls prevention in the NEPHSP. The integration within the same service item or with other service items  was limited.Despite that fall-related awareness, experience, and policies could facilitate the implementation , serviceThe service provider organizations had limited collaboration with other government departments, civil societies, private sectors, and academic institutions. Service professionals highlighted the urgent need for a national action plan or guideline to provide specific evidence-based fall-prevention guidance in PHC settings. There were no outside financial incentives for service professionals.Despite service professionals recognizing the value of falls prevention to improve older people’s health , they diMost PHC clinicians cited the low financial incentives within their organizations as a barrier. PHC clinicians with less experience emphasized that they needed more capacity-building resources to acquire professional fall-prevention concepts and skills.Most service clinicians’ positive attitudes toward falls prevention were consistent with their organization’s mission and values . HoweverThree elements of the process were identified as facilitators. For example, the region-tailored guidance plan was well-developed 1 to 3 months prior to the new round of implementation . HoweverDespite personal and team debriefing being regularly organized among PHC clinicians to share the implementation progress, very limited feedback was provided to them to understand the quality of current interventions and the older people’s attitudes toward these interventions.A framework consisting of 7 interrelated strategies was developed to systematically respond to identified barriers . SpecifiThis qualitative study was carried out by a multidisciplinary team with diverse research experience, to constantly reflect upon our attitudes, assumptions and positions in the interview process and analysis, to ensure that our understanding truly reflects the participants’ perspectives, roles, and concepts of the NEPHSP. This study identified the influencing factors to implementation of falls prevention programs for older people in the Chinese PHC settings. Clear recognition of the challenges and benefits of falls prevention, an adaptive regionally tailored guidance plan, and continuous governmental policy and financial support were the major facilitators, whereas the major barriers consisted of insufficient confidence in delivering interventions and poor understanding of the magnitude of falls, low recognition of the importance of falls prevention, limited multisectoral collaboration, and weak financial incentives. This study also developed 7 strategies to address these challenges.20 and Chinese National Injury Surveillance System,21 while fatal outcomes of falls are captured by the Chinese National Mortality Surveillance System.22 An integrated surveillance platform could routinely provide consistent analysis and interpretation of multiple data sources about the burden, cost, and risk factors of health issues over time. For example, the national data linkage infrastructure in Australia provides policy makers and researchers with comprehensive information of the health and well-being of the community.23 In Chinese PHC settings, armed with integrated data, service professionals could better prioritize and mobilize more resources for fall-prevention implementation.The data-driven surveillance strategy could provide fundamental support to prioritize falls prevention under local jurisdiction. In China, the fragmentation of information systems in the design, interoperability, and data quality compromises the understanding of the epidemiology of falls. For example, the nonfatal outcomes of falls are separately recorded in the Chinese Chronic Disease and Risk Factor Surveillance program24 However, the absence of performance indicators in the current audit and feedback loop could affect the quality of the implementation and health gains. Some key performance indicators, including the number of consultations per year, the number of older people in each education lecture, and the proportion of older people receiving the fall-risk screening, are therefore suggested additions. These process-oriented indicators could gauge the individual effort of PHC clinicians. To avoid the concern about the excessive pursuit of the number of service users over the service quality,25 some quality-oriented indicators could be included to encourage quality assurance fall-prevention practices . It could also deepen service professionals’ understanding of older people’s views and perceptions, and the extent of their involvement in improving the quality of falls prevention programs.A well-established audit and feedback strategy could improve professional practice and health care outcomes through an ongoing review of agreed performance indicators.4 which were again identified as a barrier in this study. The implementation strategy aims to conduct robustly designed studies to evaluate the effectiveness of a regionally tailored fall-prevention program. In addition, a national task force of experts should be established to reach a consensus on the current and emerging evidence of implementation strategies in fall-prevention research. Based on this support, an evidence-informed national action plan or guideline can therefore be developed to articulate specific interventions with recommendation levels and minimum resources required. A similar technical package has been developed by the World Health Organization to support all concerned individuals and institutions to prevent and manage fall-related injuries in their work.1 This package provides implementation guidance on interventions for which implementation caveats feature strongly in the global evidence base.1A recent review reflected the low quality of evidence for the effectiveness of fall-prevention interventions in China,26 The lack of professional knowledge and skills of service professionals identified in this study could compromise effective fall-prevention implementation. To address this constraint, the workforce strengthening strategy recommends that various on-site and online training opportunities, for example, in-service training, seminars, and workshops, could be offered to junior and less highly trained PHC clinicians. These training resources should encompass essential parts of falls prevention . The ultimate goal of training is to ensure that PHC clinicians have a clear understanding of key issues in falls prevention for older people and competency to perform their roles in implementation.Health service coverage and outcomes are highly dependent on the quality of the health workforce.28 Of note, it would take a long time to show the benefits of community empowerment,27 but the time cost could be affordable in NEPHSP as long-term nationwide work.5The current fall-prevention interventions often told older residents how to prevent falls, rather than listening and responding to their needs, which could be characterized as “a program was done for older people” not “a program was done with older people.” Consequently, it was not surprising that these interventions were viewed with apprehension and underused by many older people. The community empowerment strategy attempts to enable the transformation of older people being only end users into being engaged in self-determination of their communities. This change could boost their confidence and motivation to actively acquire more knowledge, skills, and resilience against fall-related injuries. Positive outcomes through promoting community empowerment, such as high retention in following services and a better sense of health gains, have been reported in previous studies.29 The information generated from these services could be shared to identify important fall-related risk factors for older people, such as fall history, poor vision, and the intake of multiple medications,30 and can be applied to provide personalized services in fall risk screening, prediction, and management. In addition, 2 service items related to health education29 , could facilitate the development of easy-to-understand fall-prevention information and the delivery of these materials to older people. Based on the experience shared in previous studies, the internal services integration strategy could have the potential to improve the cost-effectiveness, access, and uptake of fall-prevention intervention in older people.31Presently, there are 5 other service items closely related to older people .30 For example, a safe outdoor environment with accessible transport infrastructure could encourage older people to maintain healthy levels of physical activity as they age, which would in turn reduce the risk of falls.32 The design, construction, and maintenance of these physical environments need intensive human resources and financial support from the department of transportation and urban construction rather than the health sectors. In addition, the strength of any group with an interest or responsibility for falls prevention could be leveraged to support the implementation. For example, civil society groups could be actively engaged in policy dialogue to address potential conflicts across different government departments in the implementation of falls prevention.33 Public-private partnerships could be introduced through the involvement of business and the private sector to raise additional funds to support resource-consuming interventions.34 All these collaborations could benefit the achievement of desired outcomes of the complex fall-prevention interventions.35The external enabling environment strategy encourages identifying potential collaborators and forging partnerships with other like-minded agencies. As a cross-cutting health issue, the prevention of falls usually requires extensive resources beyond the capabilities of health sectors.This study has some limitations. First, the findings should be transferred to other regions with caution due to the purposive sample method. However, readers could judge the transferability based on the details of the method. Second, all factors were qualitatively identified without causal inference. Further quantitative surveys should be conducted to triangulate these findings.This study identifies the facilitators and barriers to the implementation of falls prevention for older people in Chinese PHC settings, at 3 study sites, through a synthesis of qualitative interviews and discussions. Seven strategies were developed as a systematic response to foster the fall-prevention implementation for older people in Chinese PHC settings. These findings also have to the potential to provide valuable insights for other regions or countries experiencing similar challenges, within a rapidly aging population, to maximize the benefits of falls prevention for older people.


Collaborative research between institutions may not yield results to transform communities. Many research collaborations come to the end of their life time without achieving their originally set goals and with a dearth of community transformation to show for it.To delineate and highlight the achievements of the Uganda (Makerere University)-Case Western Reserve University Research CollaborationWe retrospectively compiled and reviewed the data on research, training and policy impact achievements of the Uganda (Makerere University)-Case Western Reserve University Research Collaboration over a period of 30 years of its existence.Over the last 35 years, the Uganda (Makerere University)-Case Western Reserve University Research Collaboration trained a total of 104 Ugandans with Masters, PhDs and other varied graduate training programs. More than 70 large tuberculosis/TB+HIV studies were conducted with more than 360 manuscripts published including landmark local and global TB/HIV policy impact publications.The Uganda (Makerere University)-Case Western Reserve University Research Collaboration has in the past 35 years built the capacity of Ugandan and international students through conducting landmark research, training and mentoring and contributed to TB HIV management policy changes in Uganda. The Uganda (Makerere University)-Case Western Reserve University Research Collaboration began in 1986 after a presidential invitation to the late Prof. Fred C. Robbins, Case Western Reserve University (CWRU) Professor and Nobel Laureate, to visit Uganda and assist with the then Human Immune Deficiency Virus/Acquired Immune Deficiency Syndrome (HIV/AIDS) epidemic. Prof Robbins and his Uganda colleague's  vision of a multi-disciplinary research collaboration on HIV/AIDS and its complications officially began with grant funding from the United States (US) National Institute of Health (NIH) in 1988 to better understand the new HIV epidemic health issue in Uganda, undertake capacity building in Uganda through research, training and mentoring.In 1986, a memorandum of understanding was initially signed between Makerere University, Case Western Reserve University, Cleveland Ohio and Uganda's Ministry of Health. Over the years, the collaboration has evolved into a multi-project, multi-disciplinary research, training and capacity building organization funded by the US and European grants to collaboration investigators at Makerere University, Joint Clinical Research Center (JCRC) and Case Western Reserve University. The collaboration's focus has expanded into more disciplines including epidemiology, biomedical sciences, clinical trials, nursing, anthropology, bioethics, biomedical engineering, cancer, cardiovascular disease and other related areas. Training has been key for the collaboration with former trainees currently in leadership positions at Uganda ministries, Universities and the private Sector.Makerere University has been the key host Institution for both local and international students who have been trained and mentored through this collaboration. In addition, senior research investigators under Makerere University have significantly contributed to landmark research that has contained the HIV epidemic over the past 35 yearsThe collaboration has received ongoing support from the Fogarty International Center for HIV and TB-related training grants, which have been funded continuously since 1989.Over the last 35 years, Uganda (Makerere University)-Case Western Reserve University Research Collaboration has trained a total of 104 Ugandan trainees, graduating with Masters of Public Health, Masters of Epidemiology and Biostatistics, Masters of Science in biomedical science, doctorate of philosophy degrees as well as other varied graduate training programs. The DocThe collaboration has conducted more than 70 large tuberculosis, HIV and joint HIV /Tuberculosis(TB) studies in the last 35 years and has published more than 360 manuscripts, with its scientists having made numerous contributions as advisors, reviewers, and committee and protocol team members. Important contributions include trials done early in the HIV epidemic confirming severe cutaneous side effects of thiacetazone-containing TB treatment regimens in HIV-infected patients1, the inferiority of ethambutol-based continuation regimens for TB treatment in HIV-infected personsOther areas where the Uganda (Makerere University)-Case Western Reserve University Research Collaboration has been key included matched Uganda-USA fellows research training, as well as clinical disciplines including interventional cardiology, improved management of stroke and other neurological conditions as well as cancer management. In 2015, Drs Emmy Okello and James Kayima (of the Uganda Heart Institute) were trained as interventional cardiologists by Uganda-CWRU collaboration.Uganda (Makerere University)-Case Western Reserve University Research Collaboration and training has had significant impact on the management of TB/HIV in adults and children globally, such as preventive treatment of tuberculosis 4. Other Over the last 35 years of research, training and mentoring young researchers; the collaboration has the following lessons to share: (i) Training is a key element, as it leads to newer generation of researchers and clinicians to continue with activities. (ii) It is important to conduct research responsive to local health conditions as this has a positive impact on local health outcomes. (iii) Team work is important with equal contributions and benefits to both Ugandan and international collaborators (iv) Mentorship is a key element of all successful research collaborations. (v) High quality research generates knowledge that transforms policy and reflects into a nation's GDP to the benefit of the local communities, (v) Academic institutions present sustainable collaborations for training and research for societal transformation.


The advancement of natural language processing (NLP) has promoted the use of detailed textual data in electronic health records (EHRs) to support cancer research and to facilitate patient care. In this review, we aim to assess EHR for cancer research and patient care by using the Minimal Common Oncology Data Elements (mCODE), which is a community-driven effort to define a minimal set of data elements for cancer research and practice. Specifically, we aim to assess the alignment of NLP-extracted data elements with mCODE and review existing NLP methodologies for extracting said data elements.Published literature studies were searched to retrieve cancer-related NLP articles that were written in English and published between January 2010 and September 2020 from main literature databases. After the retrieval, articles with EHRs as the data source were manually identified. A charting form was developed for relevant study analysis and used to categorize data including four main topics: metadata, EHR data and targeted cancer types, NLP methodology, and oncology data elements and standards.A total of 123 publications were selected finally and included in our analysis. We found that cancer research and patient care require some data elements beyond mCODE as expected. Transparency and reproductivity are not sufficient in NLP methods, and inconsistency in NLP evaluation exists.We conducted a comprehensive review of cancer NLP for research and patient care using EHRs data. Issues and barriers for wide adoption of cancer NLP were identified and discussed. For many current usages in so far as cancer research and patient care, the structured data provisioned as part of many popular EHR systems are considered to be incomplete2 in that it is limited to specific subsets of clinical data, such as billing codes, and laboratory tests. Some data critical for cancer research and patient care may be recorded only in unstructured text, for example, whether and when a cancer improves or worsens after a given therapy.3 Advancement in natural language processing (NLP) techniques has promoted the usage of clinical information extraction (IE) from unstructured texts to help supplement this information gap,5 and consequently, the application of NLP in cancer domain has also been increasing.As a real-world data source, electronic health records (EHRs) have the potential to provide the comprehensive and relatively timely clinical information necessary to facilitate cancer research and patient care. One of the major challenges associated with the use of EHR data for cancer research and patient care is data quality.Key ObjectiveTo assess electronic health record (EHR) for cancer research and patient care through assessing the coverage of natural language processing (NLP)–derived data elements by the Minimal Common Oncology Data Elements and reviewing existing NLP methodologies for data extraction.Knowledge GeneratedA comprehensive review of cancer NLP for research and patient care using EHRs data extraction was conducted. Issues and barriers for wide adoption of cancer NLP were identified and discussed.RelevanceOvercoming the identified issues and barriers will improve the readiness of EHRs for cancer research and patient care, thus propelling translational clinical research and care.6 The mCODE data standard was initiated from 2018 by ASCO, other founding collaborators, and a group of collaborators, including oncologists, informaticians, researchers, and experts in terminologies and standards, to develop and maintain standard computable oncology data formats in EHR for cancer research and practice. The final mCODE data standard (version 1.0) included six primary groups (domains): patient, disease, laboratory/vital, genomics, treatment, and outcome. Each domain is organized into several concepts, which then have associated data elements. These concepts are referred to as profiles. In total, 23 profiles exist across mCODE's six primary domains , and Clinical Modification.9To gain an understanding of the gaps and opportunities of NLP in EHR for cancer research and patient care, we conducted a scoping review of literature relevant to cancer NLP in EHR. We hypothesize that the need of NLP solutions to extract data elements reflects critical information not captured by structured EHR, and the readiness of NLP for extracting those data elements highly depends on the performance of NLP methodology involving many aspects including NLP tools, methods, evaluation, and reproducibility. Data elements defined as part of the Minimal Common Oncology Data Elements (mCODE) standard are used as a proxy for data elements that would be important for cancer research and patient care.10 providing an introduction to NLP and its potential applications in oncology, describing specific tools available, and summarizing on the state of the current technology with respect to cancer case identification, staging, and outcomes quantification. Similarly, in 2019, Datta et al conducted a scoping review of clinical NLP literature extracting information from cancer-related EHR notes according to frame semantic principles.11 They created frames from the reviewed articles pertaining to cancer information such as cancer diagnosis, tumor description, cancer procedure, breast cancer diagnosis, prostate cancer diagnosis, and pain in patients with prostate cancer. This review paper emphasized data model construction. Another relevant work reviewed the major NLP algorithmic advances and cancer NLP application developments over 3 years since 2016, summarizing the main trends of clinical cancer phenotype extraction from EHRs.12Some prior work exists. In 2016, Yim et al conducted a similar literature review,In our scoping review, we focus on (1) presenting all the efforts using NLP in extracting cancer information as an end point or intermedium step and aligning them to mCODE, that is, the new data standard for oncology domain and (2) categorizing them based on NLP methodology.13This scoping review was performed based on the following five stages of the framework from Arksey and O'Malley.In this scoping review, we aim to assess the alignment of NLP-extracted data elements with mCODE and review existing NLP methodologies for extracting said data elements.We included articles to a 10-year period from January 1, 2010, to September 4, 2020. Only studies written in English were considered. Literature databases surveyed included Ovid MEDLINE(R) and Epub Ahead of Print, In-Process & Other Non-Indexed Citations, and Daily; Ovid Embase; Ovid Cochrane Central Register of Controlled Trials; Ovid Cochrane Database of Systematic Reviews; Scopus; and Web of Science. The search strategy for articles using NLP in cancer domain was designed and conducted by an experienced librarian (Larry J. Prokop). A detailed description of the search strategies used is provided in The NLP involved could be used either as an end product or an inter-medium step for other downstream analytics.The NLP was cancer related.NLP was conducted for cancer, as defined below.EHR-sourced textual data in English were used as data source.All the titles and abstracts after deduplication were screened, and the publications were included ifNot written in English.Retrieved by irrelevant term matching.Using non-EHR data sources such as literature, web resources, knowledge bases, clinical trials, and clinical guidelines.Not using English EHR data.Review papers/letters.We excluded publications if they wereA standardized charting form was established to synthetize relevant publications. The information of interest can be categorized into four main sections: metadata, EHR data and targeted cancer types, oncology data elements and standards, and NLP methodology.The Metadata section consisted of publication year; publication domain; major country of authors (first/senior author); types of author's organizations; the main study aim, defined as one of research and patient care, of the article; and the NLP study aim. The NLP study aim was classified into two groups: IE as an end point and IE as the input for machine learning. The research aim and patient care aim were further subcategorized, and more details are presented in Figure The EHR Data and Targeted Cancer Types section aimed to summarize information including the targeted cancer types of the article, data time frame, and document types . We defined the targeted cancer types using the ultimate cancer type around which the study was focused, for example, if the study focused on lung nodules but the aim of this study was to screen lung cancer, then we charted the targeted cancer type as lung cancer.In summarizing oncology data elements and standards, we aggregated NLP-extracted data elements based on the 23 profiles presented in Appendix Table AIn the NLP methodology section, we described the most frequently used NLP tools, frameworks, or toolkits and cancer-specific NLP tools. In addition, we also analyzed NLP methods, the evaluation environment, performance metrics used, NLP methods evaluation granularity, and the study's reproducibility and rigor of evaluation.NLP methods were categorized into one of six groups: rule-based, machine learning, deep learning, hybrid , more than one (multiple models), and unspecified. Evaluation granularities included mention level, sentence level, document level, patient level, multiple levels, previously reported, and unknown.v external), we categorized the evaluation environment into one of the following groups: single center, multiple centers, Veterans Affairs, benchmark data set, single center and benchmark, no evaluation, and unspecified.To understand the study's evaluation scope  studies from clinical notes,103 and 41 (33%) studies from radiology reports.131 As various document types could be used for one study, document type numbers in each article were further analyzed  adopted standard terminologies to normalize NLP output. Appendix Figure A3 adoptedFigure 5Table 132 for NLP in the general domain, a delay of such application in the cancer domain can be observed.Appendix Figure ATo delineate the association among the charted items, we conducted a synthetic analysis for NLP methods, study aim of the article, and evaluation level Fig C.Among the total 123 articles, 86 (70%) articles conducted the NLP evaluation in a single-site environment, 14 (11%) articles conducted a multisite evaluation, two (2%) studies used an external benchmark data set , and four (3%) studies were based on VA data. In addition, 10 (8%) studies did not specify an evaluation environment, and seven (6%) studies did not report evaluation details.A variety of metrics were used to evaluate NLP methods Fig F. The laFor the ability to replicate NLP methods, there were 78 articles (63%) in level 1. Level 2 contained 35 articles (28%), and level 3 had only 10 articles (8%). For the evaluation rigor of NLP methods in each study  focus in 2020 (12 articles). This growth reflects the value of NLP for clinical practice as NLP becomes more accessible. In the meanwhile, issues and barriers for wide adoption of cancer NLP were identified and discussed as follows.133 Cancer is highly complex and diverse; consequently, cancer research and patient care require diverse types of data, which can be reflected in our document type summary  and the laboratory/vital group (three studies), primarily come from structured data, thus less involved. Nevertheless, from the perspective of precision oncology and given that genomic data created through molecular diagnostics and treatment have been increasingly accumulated in EHR, tackling genomic data extraction from text could greatly advance the efficient secondary use of EHRs. In the meanwhile, aligning the standardized terminologies for data elements to mCODEs, for example, International Classification of Diseases for Oncology, American Joint Committee on Cancer, and ClinVar, is also important.104 proposed a scalable NLP pipeline that was capable of inferring Brain Tumor Reporting and Data System report scores. In the study by Sevenster et al, measurements used to synthesize treatment response status were extracted and paired across consecutive free-text computed tomography reports.125 Clinically relevant outcomes can be extracted by NLP methods based on rules and machine learning130 as well as deep NLP models113 from radiologic reports. The codependent effects of NLP and machine learning in categorizing cancer disease status were investigated in computed tomography and magnetic resonance imaging reports.117 Those studies focused on radiology reports which are ubiquitous and central to ascertainment of cancer disease status. However, additional information relevant to cancer status cannot be captured in radiology reports, such as laboratory results. The only study that did not use radiology reports applied deep NLP models to extract meaningful outcomes from clinical progress notes.3NLP for the outcome group (six studies) was under explored probably because of the intrinsic challenge of cancer disease status assessment as it mostly relies on clinicians' qualitative judgment on the current trend of the cancer. The judgment can be based on a single type or multiple kinds of evidence, such as imaging data, assessment of symptoms, tumor markers, and laboratory data, at a given time. Among those six studies, a study by Lee et al136 Moreover, oncologists sometimes have irreducible uncertainty about whether the cancer is responding or progressing when a clinical note is filed.3 In addition, recorded cancer disease statuses may be time varying resulting in multiple instances of outcomes, which poses additional challenges for outcome extraction. For such mCODE data element as cancer disease status without a discrete data field in EHR, we believe NLP could play the most important role in extracting and structuring it.Owing to the complexity of cancer disease status assessment, current EHR recording practice does not favor a seamless outcome assessment. For example, there is generally not a mechanism to input the staging information into the radiation oncology EHR or link metastatic sites to the original diagnosis, which are usually of interest for outcome analyses.Our review does indicate that some critical information elements covered by mCODE and needed for cancer research and patient care are not currently captured as part of structured EHR data. Since mCODE is used only for minimal critical oncology specialty information, it is as expected that those data elements extracted by NLP but not covered by mCODE are also important for cancer research and patient care, ie, cancer screening such as lung nodule screening and social determinants of health such as family history and smoking. This implies a great opportunity for improving structured data capture in EHRs to improve the readiness of EHRs for cancer research and patient care.Recently, US Food and Drug Administration published a guidance for assessing EHRs and medical claims data to support regulatory decision making for drug and biological products, which also provides insightful and sharable recommendations for real-world data applications in other domains. For AI methods extracting data elements from unstructured data, it recommended to specify methods, tools, data sources, and the metrics associated with validation of the methods. Although the ability to replicate NLP methods was not mentioned in the US Food and Drug Administration guidance, we consider it a crucial factor that affects the adoption of NLP methods in the cancer domain. Unfortunately, our review revealed that more than half of the surveyed articles failed to provide either the source code or sufficient detail in the methodology to fully replicate the developed NLP systems, indicating a poor reporting practice in the domain.Transparency and sharing contribute to assessment of research reproducibility, robustness, and replicability. Guidance on code sharing aligned with a specific study aim would be helpful to support transparency and reproducibility that would then strengthen the credibility of NLP results and promote downstream patient care leveraging NLP results.For studies reporting no evaluation metrics for NLP methods, reasons that justify no assessment include that NLP evaluation has been reported in a previous publication or that NLP was simply used as an intermediate feature extraction step that feeds into a downstream machine learning algorithm that was itself evaluated. Although most studies reported metrics for NLP methods, reported metrics were not consistent across all studies, but rather study dependent, even for those with the same study aim.137 thus propelling the translation to clinical application.Most studies inherited traditional and typical NLP study paradigms, focusing on mention-level, sentence-level, or document-level evaluations using a benchmark data set. Patient-level evaluation was relatively sparsely used (20 articles), and such an alignment to real-world clinical settings was very seldom seen. A considerable number of studies assigned with not clear evaluation level were extracting data elements from pathology reports. Without pondering that potential conflicting information could be reported for a single patient through multiple text reports, the evaluation level was failed to be clarified in these studies. There is certainly a need to evaluate NLP methods to an extent where the true level of complexity of clinical EHR data could be reflected following a scientific and rigorous evaluation process,Although NLP solutions can be potentially leveraged for large-scale IE, single-site studies are still predominant among current cancer NLP research. We acknowledge the potential barriers of multisite cancer NLP research such as complex concept definitions requiring extensive effort for participating sites to reach consensus and variations in clinical documentation patterns and data infrastructures  hampering cross-institutional experimentation. Regardless, we observed that 11% of studies surveyed involved more than one EHR in the study, demonstrating the feasibility of multi-institutional NLP efforts in cancer research and care.There exist some limitations in our study. First, this review may be biased due to the potential of missing relevant articles caused by search strings and databases selected. Second, we only included articles written in English with the focus on using NLP in cancer EHR. Articles written in other languages would also provide valuable information. Third, our review did not include methods based on non-English EHRs. Finally, our study may also suffer the inherent ambiguity associated with data element collection, normalization, and analysis due to subjectivity introduced in the review process.


The COVID-19 pandemic created a series of challenges for children's health, including several challenges related to nutrition and physical activity in child care settings. Thus, this study explored: 1) how COVID-19 impacted nutrition and physical activity in child care settings and how to address these challenges moving forward; 2) potential best practices in child care that emerged during the COVID-19 pandemic worth continuing; and 3) future directions for accessing, implementing, and evaluating COVID-19 federal investments in child care settings.n = 4), practitioners (n = 7), and researchers (n = 6). Recruitment continued until saturation was achieved. Virtual interviews lasted approximately 45 to 60 minutes and were recorded, transcribed, and coded for themes and subthemes using thematic content analysis.The study utilized a qualitative content analysis approach. In June 2021, the investigators conducted 17 qualitative interviews with federal representatives (Primary findings included: 1) COVID-19 likely increased stress and exacerbated prevalence of food insecurity for child care staff and participating families, and decreased diet quality among children both while in and outside of child care; 2) flexibilities to federal child care requirements, outdoor learning opportunities, and meal provision strategies implemented during the pandemic were perceived as positive and could continue post-pandemic; and 3) future efforts could utilize the recovery funds to make changes that are equitable and sustainable, such as conducting equity assessments and collaborating with community organizations, along with evaluating impacts of these efforts on food insecurity and health of child care staff and participating children.Overall, recommendations focused on several social determinants of health, including addressing food insecurity among both children and staff, and infrastructure for nutrition and physical activity. Continued programmatic and public health recovery efforts aimed to mitigate the negative impacts of COVID-19 are critical to fostering health and wellbeing in child care settings. The child care system is a critical component to aid in child development in the United States (U.S.). In 2019, an estimated 59 percent of children five and younger were in non-parental care at least 1 day a week, and of those, 82 percent were enrolled in center-based care or family day care homes . The CorCOVID-19 response and recovery efforts in child care are likely to have short-term and longer-term impacts on behavioral, household, societal, and economic influences on population health and well-being, including weight status . In 2020CARES, enacted on March 27, 2020, included $3.5 billion for the Child Care and Development Block Grant and $750 million for Head Start programs  CARES alThe rapid infusion of large investments in child care infrastructure via the CARES, ARPA, and other recovery bills has the potential to meaningfully influence SDOH and child nutrition and physical activity outcomes. Thus, research on the impacts of COVID-19 and the utilization of the CARES and ARPA funds in child care is critical. A qualitative study design was employed to gather insights from federal agency representatives, child care practitioners, and public health researchers on: 1) how COVID-19 affected nutrition and physical activity in child care settings and how to address challenges resulting from the pandemic; 2) adjustments in child care settings that emerged throughout the pandemic as potential best practices worth continuing; and 3) future directions for accessing, implementing, and evaluating the CARES and ARPA child care investments.n = 4), child care practitioners (n = 7), and public health researchers (n = 6). Most interviews were conducted with one key informant; however, there were two instances of two participants joining together in one interview . Purposeful sampling was used to identify potential interviewees with experience and expertise in child care policies, practices, and research , Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), and Supplemental Nutrition Assistance Program (SNAP)). Their work primarily centered around young children and families, specifically low-income families. Researchers provided specific feedback on future directions for evaluating child care utilization of CARES and ARPA funding. Practitioners contributed perspectives from both the non-profit and for-profit sectors, commercial centers and family day care homes, and at the national and state levels. Their involvement in child care spanned programmatic, policy, and advocacy duties, within the topic areas of nutrition, physical activity, wellness, food systems, and federal nutrition programs. Federal representatives worked across different agencies, such as the Administration of Children and Families' Office of Child Care and Early Child Development and Office of Minority Health, and described their work in child care settings related to nutrition and physical activity priorities. They reported that responsibilities included but were not limited to overseeing health services for Head Start, crafting policy, training, and technical assistance for child care staff, and building systems within child care.A semi-structured interview guide included questions asked of all interviewees along with tailored questions and probes specific to each of the three interviewee groups to align with their specific area of expertise. The core interview questions focused on identifying the impacts of COVID-19 on childhood nutrition and physical activity, the nutrition and physical activity opportunities that emerged due to COVID-19, and how recent CARES and ARPA investments may affect child care settings, as well as how these investments could be used to strengthen nutrition and physical activity in these settings. Three investigators trained in qualitative interviewing conducted the interviews. Two of the investigators held master's in public health (MPH) degrees and the other investigator held a doctorate degree in nutritional sciences. All three investigators had recent experience collecting and analyzing qualitative data along with the expertise in child care and child health and wellness. The interviews were conducted by telephone or Zoom video conference. The investigators had no conflict of interest or other relationship with the interviewees that could influence the data collected. Each interview took approximately 45–60 min to complete. Participants were informed that the interview was voluntary, and they had the option to end the call at any time or choose not to answer a question for whatever reason. However, none of the interviewees refused or ended the call early. Interviews were audio recorded with participants' permission and transcribed verbatim by a professional transcription service. Participants were provided a $30 incentive in the form of a gift card for those who could accept it.n = 10) and subthemes (n = 7) that are presented in the following findings section. As part the analysis and to ensure feedback from the various interviewee subgroups were represented, the investigators tracked which themes were supported by feedback from the different interviewee groups. Example quotes also are included below to support the themes.The qualitative thematic analysis  started Interviewees across all groups reported that child care professionals experienced high levels of both mental and physical stress due to the COVID-19 pandemic and that “stress” was one of the priority issues to address. They described that state and local COVID-19 restrictions caused most child care programs to temporarily close or operate with limited capacity. Practitioners and researchers cited limited staffing as a major challenge to reopening due to perceived health concerns and low wages of child care staff. Researchers noted that child care staff were overwhelmed with implementing new health and safety procedures and needed additional support. One solution mentioned by multiple researchers was to provide child care staff health and wellness support, including mental health support.The COVID-19 pandemic also caused a high-level of stress among children and families. Practitioners explained that families were hesitant to put their child (ren) back into child care, especially when health and safety plans were in flux. In addition, practitioners indicated that child care closures resulted in disruption to children's daily routine and structure, resulting in child mental health and behavioral issues that were evident once children returned to child care. Interviewees across all groups described that child care staff will need to allocate additional time and resources to helping families transition back into child care. Federal representatives described an increased need for attention and funding at the state level, such as a train-the-trainer model to support child care staff in addressing child mental health issues.Interviewees across all groups overwhelmingly indicated that food insecurity was prevalent among families with young children and that food insecurity increased in this group due to COVID-19. Interviewees expressed concern that children did not receive adequate meals and snacks while not attending child care programs. Practitioners and researchers described child care staff as oftentimes being food insecure themselves and reliant on federal food assistance programs and food banks/pantries. Practitioners and researchers described this as problematic, indicating the need to better support child care providers financially. Despite this, interviewees across all groups mentioned that COVID-19 has resulted in an increased national awareness of food insecurity issues in the U.S. This presents an opportunity to create sustainable changes to help mitigate food insecurity among child care staff and the children and families they serve.Interviewees across all three groups expressed concern regarding the nutritional quality of meals among children not attending in-person child care. One federal representative mentioned that fruits and vegetables may need to be reintroduced to children in the child care setting since some families may not have had access to fruits and vegetables while not attending child care programs during the pandemic. The reintroduction of fruits and vegetables could help increase the likelihood that children will try and consume these healthy foods once returning to child care.For child care programs that were open, practitioners noted that the nutritional quality of foods served may have deteriorated. Practitioners and researchers described that COVID-19 halted family style dining practices due to health and safety guidelines. Without family style dining, children missed out on critical developmental opportunities, such as gross motor development. However, several practitioners and researchers noted that child care programs were hesitant to reinstate family style dining due to health concerns and that programs may need guidance in doing so. Aside from family style dining, interviewees reported that other nutrition promoting efforts, like taste testing and new recipe development, were reduced due to additional staff time spent on health and safety procedures. In addition, a practitioner also highlighted an increased reliance on processed foods among child care programs that remained opened during the pandemic.Lastly, interviewees across all groups hypothesized a general reduction of physical activity when children were not in child care. Several practitioners posited that physical activity levels are influenced by the communities' children live in. Lower-income and/or urban neighborhoods may allow for fewer physical activity opportunities due to limited access to outdoor recreation spaces and safety concerns. Interviewees across all groups felt that targeted outreach efforts and strategic planning are needed to bring low-income and ethnic minority community members into conversations with local and state officials within their communities and neighborhoods.A summary of the themes with supplemental quotes on the impact of COVID-19 child care settings is shown in During the COVID-19 pandemic, the USDA issued CACFP waivers that allowed programs flexibility in targeted meal patterns, nutritional requirements, and increased reimbursement rates . CACFP wDue to health and safety concerns, interviewees reported that child care settings increased outdoor activities during the COVID-19 pandemic. Practitioners and researchers highlighted the positive impact that outdoor play and learning had on physical activity and mental health of both children and staff. They also recommended shifting additional learning activities to the outdoors post pandemic. Practitioners cited “farm to early care and education (ECE)”  as an opportunity for outdoor learning. They further described that “farm to ECE” provides nutrition education, experiential learning, and connections with produce growing processes and practices, while also allowing for social distancing. A few practitioners and federal representatives suggested maximizing excitement among children once they return to child care settings and reunite with friends in person by supporting outdoor group activities, such as learning trails, center/school gardens, and other activities that can positively impact nutrition and physical activity.Interviewees discussed various strategies to serve meals to children and families outside of the child care setting to address food insecurity. Examples included grab and go meals, partnerships with local entities , meal kits, meals in the mail, backpack programs, on-site food pantries, and meal transportation for families. Practitioners noted that these efforts increased access to healthful meals, but gaps in providing meals to eligible children may still exist, such as child care programs that were not able to make contact with certain children and child care programs that do not have the capacity to distribute take home meals. Programs could continue to work with community partners to help with reach and sustainability.A summary of the themes with supplemental quotes on the opportunities that emerged during COVID-19 is shown in One federal representative also suggested that CARES and ARPA funds could be used to hire designated nutrition and wellness staff and/or funds be used to incorporate nutrition and wellness training as part of the onboarding training curriculum. This investment could help further empower staff to be “healthy lifestyle” role models for the children participating in their programs.Several interviewees across all groups indicated that a key component to recovering from the negative impacts of COVID-19 is recruiting and retaining sufficient child care staff, who are mostly negatively impacted by low wages. Interviewees stressed that child care staff salaries need to be increased and sufficient breaks and paid time off need to be available for staff to help with staff satisfaction and retention. A few practitioners and federal representatives indicated the potential benefit of an advisory board composed of key partners from the local to federal levels who could support child care programs in utilizing CARES and ARPA funds. Suggested partners included nutrition and physical activity experts and researchers who could advise on child care programs and how to invest money in nutrition and physical activity related items. Community-based organizations also could help ensure funds are being allocated equitably, including social service organizations. Advisory boards could include representatives from federal food assistance programs to help address food insecurity issues. State, regional, and/or federal partners could provide technical assistance to ensure funds are spent in accordance with state and federal laws and regulations. It was also recommended that child care staff and families be involved in various processes to ensure changes made are appropriate and effective for the intended end-users.Multiple interviewees across the three groups indicated that child care programs should use COVID-19 recovery funds to make long-term changes rather than using funds as a short-term “band aid” solution. Sustainable investments related to nutrition and physical activity cited by the interviewees included developing and maintaining: school vegetable gardens, nature trails, nutrition education curriculum, food service equipment, cooking classes, nutrition and physical activity training for staff, technology software, and coordinated, cross-program data sharing systems.Multiple interviewees across the three groups also suggested that sustained funding is critical for sustainability of child care programs, which could entail embedding child care program funding with other existing federal or state funding and/or taxation earmarked for child care. Researchers and federal representatives indicated that state legislators need to be responsible for identifying ways to ensure child care programs have sufficient and sustainable funding.Interviewees were asked about suggestions for recovery options specific to programs and families disproportionately impacted by the COVID-19 pandemic. One practitioner recommended ensuring technical assistance and guidance are made available to programs and families impacted the most. This would help maximize awareness of the resources available. One researcher and one practitioner also suggested that COVID-19 recovery funds could be used to invest in local municipalities and community organizations to support those disproportionately impacted by COVID-19. These organizations are often aware of the programs and families most impacted by disaster situations, such as COVID-19, and could connect them with resources to help recover. Noted examples of community organizations included: YMCA, parks and recreation departments, 4H clubs, and local colleges and universities.One practitioner and one federal representative also suggested that an equity assessment be conducted to help guide those decisions prior to allocating or spending any funds. The assessment could help identify needs and proposed services to address those needs. The equity assessment also could help avoid creating unintended consequences and negative impacts. Child care programs might not have the capacity to do such assessments; therefore, other partners at the state and local levels could potentially help.Interviewees, primarily researchers, expressed the importance of evaluating the impacts of COVID-19 recovery funds on nutrition and physical activity in child care programs. Specifically, researchers suggested that recovery funds could be used to support the research areas and projects shown in A summary of the themes with supplemental quotes on future directions to help recover from COVID-19 is shown in Findings from the interviews demonstrated that the COVID-19 pandemic has exacerbated the major challenges in child care settings that existed pre-pandemic, including child care staff compensation for their role in child development. Specifically, COVID-19 spotlighted the need for staff salary increases along with staff mental health support and sufficient paid time off. This need is consistent with emerging research which showed COVID-19 intensified work inequities, especially regarding wages among ECE educators in child care settings compared to ECE educators in school settings in Virginia . Child cResources, such as Nemours Technical Assistance Brief and the Center for the Study of Child Care Employment's public database of compensation strategies, provides guidance regarding how child care settings in eligible states can use child care stabilization grants to strengthen nutrition and physical activity efforts in this setting , 22. COVCOVID-19 also intensified the issue of food insecurity among child care families and staff. Findings from the interviews demonstrated that opportunities need to be in place to support and sustain food security efforts, so that the financial burden is not on the child care providers. Child care partnerships with community organizations, such as food banks and social service programs, could help alleviate food insecurity among child care staff and families. A recent analysis of ECE providers found that child care centers, primarily those participating in CACFP, increased efforts to connect families to food assistance  through community entities .Prioritizing the SDOH ensures individuals can obtain optimal health and quality-of life outcomes . SDOH “aResearchers could add to this area by conducting impact assessments and pilot studies on child care recovery focused on issues, such as food insecurity and health and wellness. Furthermore, studies could prioritize assessing diet and physical activity behaviors in children and ways to promote these behaviors during challenging times. Since there is a coexistence of food insecurity and obesity, childhood obesity research funding mechanisms could incorporate food insecurity as part of their research agenda . ResearcFuture research also could investigate ways to improve the health and wellness of child care staff. For instance, formative research could be conducted to identify and test staff health and wellness training curricula. These curricula could have a dual benefit of improving the health status of child care staff while also equipping staff with the knowledge and skills to be healthier role models for children under their care. The Head Start and Early Childhood Learning and Knowledge Center provides resources for promoting staff well-being in child care settings that can be utilized when designing and adapting this type of curricula .Formative research also could be conducted on adding standards related to health and wellness of child care staff and participants to the Quality Rating and Improvement System (QRIS) . QRIS isThis study had a couple limitations to note. First, the study was conducted prior to the surge of the Delta and Omicron SARS-CoV-2 variants, so there could be additional impacts of COVID-19 on child care programs yet to be identified. Second, the results also are based on a convenience sample, and thus, not representative of all federal representatives, practitioners, and researchers involved with child care. In addition, the study did not interview caregivers of children participating in child care. These caregivers could provide valuable insights on the end-user perspective. However, the findings from this study also are supported by several strengths, including successfully recruiting interviewees with an 80 percent response rate. This resulted in conducting multiple interviews with the three different interviewee groups - federal representatives, practitioners, and researchers - ensuring the findings represent feedback from each key audience. Also, the investigators identified several interviewees that resulted in a sufficient respondent pool for interviews to continue until saturation was achieved.Findings from this qualitative study highlighted the impacts of COVID-19 on child care settings, including direct effects on child care staff and participating children and families. Interviewees offered potential solutions for recovery related to SDOH, including staff recruitment and retention, food insecurity and improvements to nutrition and physical activity programming. Adjustments made during the pandemic, often through flexibility of federal regulations, allowed new promising practices to emerge, and if made permanent, could continue to benefit child care programs and participants, and ultimately childhood obesity. When considering the use of recovery funds, interviewees emphasized prioritizing long-term and equitable changes. Partnering with key community organizations and gathering input from community members were noted as possible strategies to facilitate the appropriate use of available funds. Future research could be conducted to support these efforts to help ensure changes are evidence-based and sustainable.anitto@centerfornutrition.org.The datasets presented in this article are not readily available because of participant privacy assurances due to the qualitative nature of this study. Requests to access the datasets should be directed to The studies involving human participants were reviewed and approved by University of Nebraska Medical Center Institutional Review Board. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.AN, DB, AB, LC, and AY conceptualized the study design and procedures. AN, LC, and AY designed the interview guide. DB and AB reviewed and provided feedback on the interview guide. AN and LC collected data, coded, and analyzed data. AN, DB, AB, SK, LC, and AY interpreted data. AN, SK, and LC drafted the manuscript. DB, AB, and AY reviewed and provided feedback on the manuscript. All authors contributed to the article and approved the submitted version.https://www.nccor.org/projects/impact-of-covid-19-on-child-care-programs-potential-solutions-emerging-opportunities/.This study and open access publication fees were funded through a contract awarded by FHI 360 to Gretchen Swanson Center for Nutrition. The contract included funds to produce a report for FHI 360. For more details, visit The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


The history of animals” by Aristotle (384–322 BC), which refers to an infestation with wax moths and the first description of foulbrood  [Varroa destructor and Tropilealaps spp., or the small hive beetle Aethina tumida, infections with fungi (Ascosphaera apis) and microsporidia (Nosema apis or Nosema ceranae), protozoans , bacteria (Paenibacillus larvae or Melissococcus plutonius), and viruses . This is why constant and state-of-the-art research on detection methods, pathology, epidemiology, and development of treatment strategies is indispensable in the field of honey bee pathogens and parasites. In this Special Issue “Honey Bee Pathogens and Parasites”, we present original research articles, research communication on new detection methods, and review articles on the current state of the art for molecular pathogen detection and alternative treatment methods.Honey bees are important pollinators of agricultural crops and despite the reports about elevated local colony losses over the last few decades ,2,3,4, t hive.”) . Until nGalleria mellonella and the small hive beetle Aethina tumida in honey. This method will be useful for the early identification and monitoring of the spread of these two honey bee pests [The examination of honey for environmental DNA (eDNA), i.e., traces from organisms left in the honey, is a useful tool to identify pathogens that are not directly visible, but have previously been in contact with the honey matrix. Ribani and colleagues developed a PCR method that targets the mitochondrial gene cytochrome oxidase I for the specific detection of environmental DNA of the greater wax moth ee pests .Malpighamoeba mellificae, which causes amoebiasis disease in honey bees, was enabled via a RT-qPCR method based on 18S rRNA sequences. The protozoan infects the Malpighian tubules responsible for excretion and osmoregulation. The new detection method enables identification and monitoring for an improvement in honey bee health [The detection of the pathogen e health .Apis mellifera, the Asian honey bee Apis cerana (Ac), and the Asian hornet Vespa velutina, being most severe in A. cerana. The detection of all present SBV variants cannot be ensured with the existing quantitative PCR methods. In a study by Blanchard et al., 2014, the TaqMan qPCR method was improved with new primers and an MGB (minor groove binder) probe. The method now enables the detection of all currently published SBV and Ac SBV genomes [Sacbrood virus (SBV) causes disease in honey bee larvae from the European honey bee  genomes . The early diagnosis of honey bee diseases is important to avoid further spread of diseases and economic losses to the beekeeper. The review article by Lannutti et al. gives an overview of the molecular methods for the detection and differentiation of arthropod, fungal, protozoan, bacterial, and viral honey bee pathogens .Nosema ceranae, the etiological agent of type C nosemosis, in honey bee colony losses. Furthermore, the article reports the efficacy of plant extracts, nutraceuticals, probiotics, propolis, and veterinary drugs as alternative treatment strategies. They might be a substitute for the commonly used antibiotic fumagillin, which is prohibited in the European Union [Marín-García and colleagues discuss the role of the microsporidium an Union . V. destructor, A. tumida, and Vespidae. The search for alternatives in pest control is especially important, due to the emergence of resistance to the existing chemical drugs [Entomopathogenic fungi may represent a further possibility for pest and predator control, as presented by Bava et al. Entomopathogenic fungi are being investigated as biocontrol agents against al drugs .


Involving research users in setting priorities for research is essential to ensure the outcomes are patient-centred and maximise its value and impact. The Musculoskeletal Disorders Research Advisory Group Versus Arthritis led a research priority setting exercise across musculoskeletal disorders. The Child Health and Nutrition Research Initiative (CHNRI) method of setting research priorities with a range of stakeholders was used, involving four stages and two surveys, to: (1) gather research uncertainties, (2) consolidate these, (3) score uncertainties against importance and impact, and (4) analyse scoring for prioritisation. 213 people responded to the first survey and 285 people to the second, representing clinicians, researchers, and people with musculoskeletal disorders. Key priorities included developing and testing new treatments, better treatment targeting, early diagnosis, prevention, and better understanding and management of pain, with an emphasis on understanding underpinning mechanisms. We present a call to action to researchers and funders to target these priorities. Rheumatic musculoskeletal diseases include two broad areas: inflammatory rheumatic musculoskeletal diseases  and other musculoskeletal disorders. These other disorders include a range of short-term and long-term conditions that affect the musculoskeletal system, including many highly prevalent disorders such as osteoarthritis, osteoporosis, and back pain.Research into musculoskeletal disorders has been shown to produce a 25% economic return on investment; considerably greater than the rate of return for cancer research 10·0%) and the UK Government minimum threshold for investment (3·5%).·0% and tIn 2019, Versus Arthritis, the largest musculoskeletal charity in the UK, convened a Musculoskeletal Disorders Research Advisory Group with the long-term aim of improving quality and impact of musculoskeletal disorders research.This group considered the current state of knowledge and research into musculoskeletal disorders and the current evidence for how research activity in this area might be prioritised by governments and funding agencies. Although more than 20 research priority publications in musculoskeletal disorders were identified by the group, these either focused on individual musculoskeletal disorders,PanelLong-term goalTo improve the quality and impact of research which seeks to develop our understanding and management of musculoskeletal disorders. Specifically:•to enhance prevention, early detection, and treatment and care of musculoskeletal disorders•to improve the quality of life and wellbeing of those with musculoskeletal disorders•to reduce personal, social, and economic burden of musculoskeletal disordersPopulation of interest•People (aged 18 years or older) with, or at risk of, musculoskeletal disorders, their families, carers, and health-care providers•Musculoskeletal disorders, defined as: osteoarthritis, crystal disease such as gout, primary and secondary causes of musculoskeletal pain including regional and widespread pain , hypermobility, metabolic bone disorders (such as osteoporosis and rare diseases), and musculoskeletal injuries caused by acute traumatic events•Participants and research questions within the UK were identified as the main focus, although the findings could well have relevance beyond the UKTimeframe for impactBoth within and beyond 3 years Research domains identified a prioriDiagnosis and impact (diagnosis)•Achieving an early and accurate diagnosis•Measuring the true impact of musculoskeletal disorders on individuals and on society•Maximising the potential of electronic health recordsLiving well with musculoskeletal disorders (living well)•Improving self-management and support in the home or community•Improving any aspect of health-care treatmentMechanisms of disease (mechanisms)•Understanding the causes and development of musculoskeletal disorders•Disease processes shared between disordersSuccessful translation (translation)•Ensuring that research-proven tests, treatments, and approaches are routinely available in clinical practice•Enabling discoveries to move from the laboratory to the clinic, towards patient benefitAudience•People with musculoskeletal disorders•Researchers of all stages•Health-care professionals•Funders and other agencies who influence the funding and research strategy of musculoskeletal disorders•Industry, such as pharmaceutical and medical technology companiesThe Musculoskeletal Disorders Research Advisory Group Versus Arthritis (henceforth referred to as ‘the group’) designed, led, and contributed to this research; the group has 26 members including public contributors with lived experience of a range of musculoskeletal disorders, researchers, and health-care professionals. The group and its membership are facilitated by the charity. Among clinician and researcher members, there is a wide representation of clinical specialties , of subspeciality interest and research disciplines . The group is committed to diversity, with representation of varying gender, ethnicity, age, geographical location, and stage of career. The group determined the purpose and remit of the priority setting exercise, including the definition of the population, audience, and timeline of interest . The lonThe group considered and compared a range of prioritisation methods and selected the Child Health and Nutrition Research Initiative (CHNRI) method of priority setting as it enabled a range of stakeholder perspectives to be incorporated; could consider discovery science, clinical, and applied health services research questions together; and would incorporate objective scoring to enable ranking.An electronic survey (e-survey 1) was developed, which asked for important uncertainties or unanswered questions (hereafter referred to as uncertainties) that could be answered by research, to be distributed to our audience  terminology. Finally, the research avenues underwent further review by an additional group of lay volunteer research partners, involved in Versus Arthritis but independent of this group, which resulted in further edits to improve readability. These were presented to, and agreed, with the group by email.The consolidated research avenues were scored with a second e-survey (e-survey 2). The original CHNRI method suggested that each research avenue should be scored against five criteria.In the survey, which was co-designed by the group and pilot tested by public contributors, the list of research avenues was presented with two questions representing the scoring criteria for importance (“Will this research lead to important new knowledge?”) and impact (“Will this research make a difference and lead to impact?”). Respondents were asked to score to what extent each research avenue met these criteria, using a numeric rating scale, where 1 equated to ‘not likely’ and 10 equated to ‘extremely likely’. An “I am unsure” option was also available. Answers could also be missed and there was an option to save and return. It was made clear that each respondent should answer based on their own knowledge and perspective. Optional additional questions on age, sex and ethnicity, employment, and interests, similar to e-survey 1, were included at the end of the survey  of the raw score data showed that most criterion score responses did not follow a normal distribution. Therefore, the analysis plan was amended to calculate the median rather than the mean of the criterion scores, range and interquartile range were adopted as measures of dispersion, and box and whisker plots were used to show data dispersion.Before any analysis, scores were separated from responder characteristics. The group supported equal weighting of the two criteria and presenting of a single ranked ordering of the avenues, irrespective of research domain. The CHNRI method suggests summing mean scores for each criterion to calculate scores for ranking. Normality testing was done by CLLM who was provided with the data with anonymised identifiers rather than avenue labels to carry this analysis out in a blinded manner and avoid bias. This normality testing  + median impact score (i)The total score was then used to present the research avenues in rank order from highest to lowest. Median scores for each avenue were calculated for responder groups to aid interpretation, but not for ranking. All available data from all respondents completing the survey, in full or in part, were analysed. Unsure or missing responses were not allocated a score and not included in the data from which the median was derived. Descriptive statistics were used to summarise the demographic data of participants, response rates for each criterion, and research avenues. To indicate dispersion of scores within the same rank, box and whisker plots were used with research avenues within rank, ordered alphabetically.The first survey received a total of 213 respondents . People Most participants entered uncertainties under each of the four domains ; additionally, 70 (33%) entered uncertainties in the ‘other’ uncategorised section.Preference for research domain was ranked by 86% (184) of respondents. Understanding the causes and development of musculoskeletal disorders (mechanisms) was ranked as the most important overall by 57 respondents (31%). The other three categories had similar proportions of responders scoring as most important: 42 for diagnosis (23%), 42 for living well (23%), and 43 for translation (23%).Following separation of discrete research uncertainties and reclassifying the ‘other’ category into one of the four domains, a total of 1300 uncertainties  were submitted by the 213 respondents (mean 6·1 per respondent). Respondents entered 0–18 uncertainties per domain . A total of 151 uncertainties were considered out-of-scope for the exercise, not answerable by research or highlighting barriers to research (to be explored separately in a future publication or charity report). The final number of research uncertainties was therefore 1149 . At reseInitial review consolidated these uncertainties into 108 draft themes , which wDuring the subgroup meetings and whole group workshop it became clear that most uncertainties and themes related to overarching issues that were relevant to many, if not all, musculoskeletal disorders. A decision was made to phrase all research avenues without reference to specific musculoskeletal disorders to be as generalisable and inclusive as possible. Similar questions were further consolidated, and questions where they occurred in different domains were separated, leading to a final list of 68 research avenues. Of these, 19 were in diagnosis, 20 in living well, 19 in mechanisms, and ten in translation .The second survey was live from Aug 16 to Oct 3, 2021. 114 respondents from e-survey 1 who opted to receive further contact about a second survey were invited to participate. A total of 285 people answered questions in e-survey 2. A further 197 people gave consent to take part but did not contribute any survey data.Of those who contributed data, 189 (66%)  completeThe largest group of respondents represented lay people , followed by researchers (n=58 [31%]) and health-care professionals (n=43 [23%]). The respondents were from a range of age groups and represented male and female sexes, although more females than males completed the survey . InformaThe responses for partial respondents were quite evenly distributed across the questions: the mean number of responses per criterion question were 215 for diagnosis and Impact, 210 for living well, 217 for mechanisms, and 207 for translation. The minimum number of responses for a criterion question was 200 and the maximum number 227. Considering the completeness of data, of 30 964 answered questions there were 1296 (4·2%) unsure responses and 670 (2·2%) skipped responses .A ranked list was produced from the complete responders plus any partial responders for each avenue, on the basis of the total scores for each research avenue . ConsideThe top three rankings (scores 16 and higher) comprised of 25 research avenues, which included a mixture of all four research domains, with a preponderance of questions from mechanisms (n=12) and from living well (n=8). Furthermore, no research avenues from mechanisms were included within ranks 6–8. When examining the median scores by responder characteristics, there was little difference between lay, clinical, and researcher responders for the top-ranking research avenues . HoweverIf research is to address and resolve important questions and lead to impact, the identification and development of research questions must involve those people affected by the research.The process identified the top priority as developing new treatments that prevent or reduce progression of these diseases. This outcome is important as, conversely, much recent research and clinical management focus has been on optimising existing treatments, self-management, or strategies to manage symptoms. The association of common musculoskeletal disorders with ageing contributes to the normalisation, tolerance, and de-prioritisation of these disorders among individuals.We also identified key themes relating to better understanding and management of pain; improving disease prevention, accurate diagnosis, and prediction; and understanding how to better implement, personalise, target, combine, and monitor treatments, with a key emphasis on understanding the underpinning mechanisms. Musculoskeletal pain has also been highlighted as important via insight gathering by Versus Arthritis, and stratified medicine by UK Research Council funding calls.We would not encourage comparison of the performance of the research domains as there was substantial overlap between them and we adopted a flexible approach to moving uncertainties between domains. As an example, the top-ranking research avenue relating to developing and testing new treatments was identified within living well but was highly relevant to translation.To our knowledge, this exercise is the first of its kind to identify and rank priorities for research across musculoskeletal disorders, from discovery science to applied clinical and health research, including translation. Further strengths of this process were that it was robust, transparent, disease and research stage agnostic, and included a wide range of stakeholders at all stages. The whole process was overseen by an expert steering group with a final ranking process that was independent of them. Public contributors played a vital role throughout, particularly to optimise accessibility and minimise the length of the second survey.There were some limitations to this exercise. We chose not to produce a top ten or top 20 priorities list, which might be more easily disseminated; we chose not to select a threshold above which to highlight top scores, as all 68 of the research avenues received high to moderate scores. This result is perhaps not surprising, as those with the lowest ranking were still derived from themes arising from multiple respondents' priorities in e-survey 1. The separation between the highest and lowest score was relatively modest . The seemingly large number of possible research avenues appears justifiable, given the wide range of disorders and research stages, and the high prevalence of unmet needs. Arguably, our predefined research domains might not represent the full remit of musculoskeletal research and could have biased responses to e-survey 1; however, we felt that these domains adequately covered the breadth of all uncertainties elicited, and areas submitted under the ‘other’ category could be assigned to these four domains. Although this priority setting exercise has addressed some limitations previously identified in a scoping review of priority setting with regards to research topics for musculoskeletal conditions,This prioritisation exercise provides a source of information and evidence but should also be a call to action for funders to support research priorities in musculoskeletal disorders research, and for researchers and health-care professionals to consider stakeholder views on what might be important and impactful. As these avenues are developed into specific research questions or projects, it is vital that stakeholders, including public contributors, are involved at all stages in this process, and that these priorities are reviewed to identify the extent to which they have informed subsequent relevant funding calls.In summary, we have presented the prioritisation process and associated scores of 68 identified research avenues in ranked order of their combined likelihood of leading to important new knowledge and impact. These research uncertainties and their rankings were generated by a mixed stakeholder user group, including public contributors, health practitioners, and researchers. Further work is now needed to translate these priorities into researchable questions for calls by funders, in the UK and elsewhere. We will seek to disseminate and develop these priorities and audit their uptake. Specifically, the Musculoskeletal Research Advisory Group and Versus Arthritis are committed to further develop the top priorities, alongside funders and stakeholders, into more specific questions within avenues, or translation of the avenues into population, intervention, comparison, and outcome questions where possible. In the meantime, we hope the findings and key themes we have summarised will empower the research community (including public contributors) to identify what is important within their own fields of work. We expect this work to catalyse greater attention and funding of high-quality research leading to improved knowledge and impact in the understanding and management of musculoskeletal disorders.JOR Spine, Discovering Innovative Solutions for Conditions of the Spine, and Society of Back Pain Research. JL reports grants from MRC, The John George William Patterson Foundation, and the Ruth & Lionel Jacobson Charitable Trust. DJM reports grants from Orphelion, Innovation for All, Welsh Government with Knoell, Versus Arthritis, KESS2 & Hospital Innovations, EPSRC, Wellcome Trust Collaborative Award, Petplan Charitable Trust, and British Veterinary Orthopaedic Association; payment or honorarium from Eli Lilly; receipt of drugs from Novartis; participation on a data safety oversight board for NIHR (paid) and Innovative Medicines Initiative (paid), unpaid leadership role within British Orthopaedic Research Society and Orthopaedics Research Society, and paid committee role in Science Foundation Ireland Early Career Panel. NLM reports grants from MRC and Chan Zuckerberg Initiative. HP reports grants from Knowledge Transfer Partnership–Innovate UK, Pacira Pharmaceuticals, Zimmer Biomet Healthcare, B Braun, and Wellcome Trust; consulting fees from Medacta International, Smith and Nephew, Depuy Synthes, JRI Orthopaedics, Janssen, Meril Life, Zimmer Biomet, and Paradigm Pharmaceuticals; payment or honoraria from Invibio; payment for expert testimony from Kennedy's Law; support for attending meetings from Medacta International; participation on data safety oversight board for GreenBric Study; unpaid committee role for NIHR and National Joint Registry Regional Clinical Coordinators; and receipt of drug from Pacira Pharmaceuticals. GP reports grants from NIHR, Nuffield Foundation, and Versus Arthritis, and consulting fees from Office Health Inequalities & Disparities; participation on data safety monitoring board and role as Lancet Osteoarthritis Commissioner. SMR reports grants from Versus Arthritis and Wellcome Trust, honoraria from Portugal grant review, and had a role as treasurer and committee member for the Tissue and Cell Engineering Society. EJS reports grants from NHSX and committee role on musculoskeletal reform. LT reports grants from Versus Arthritis, The Macular Society, MRC, Rosetrees Trust, University of East Anglia Faculty of Medicine and Health Sciences , and the Kennedy Institute of Rheumatology; support for attending meetings from Advanced Course–Matrix Pathobiology, Signalling and Molecular Targets, and OACON 2019. RKW reports grants from EPSRC, DePuy Synthes, and Versus Arthritis; unpaid committee role in Royal Academy of Engineering; and receipt of materials from DePuy synthesis. CW reports support for attending meetings from European Pain Federation; participation on advisory board for University of Swansea , Health Data Research UK, and National Institute for Health and Care Excellence; and committee role for Versus Arthritis and University of Bath . CF is employed by Versus Arthritis. All other authors declare no competing interests.FEW reports grants from UK Research and Innovation (UKRI), National Institute for Health and Care Research (NIHR), Versus Arthritis, Medical Research Council (MRC); consulting fees from Pfizer; and committee role for MRC and NIHR. ZP reports grants from NIHR, Versus Arthritis, Royal Osteoporosis Society, General Council for Nursing, Haywood Foundation, Chartered Society for Physiotherapy, and British Society for Rheumatology; honoraria from the Royal College of General Practitioners; non-paid consultancy for UCB, and unpaid committee roles with Royal Osteoporosis Society and Haywood Foundation. CB reports grants from Versus Arthritis and University of Liverpool . EMC reports grants from Royal Osteoporosis Society, Elizabeth Blackwell Institute  MRC Confidence in Concept Award, NIHR, Southmead Hospital Charity Research Fund, Healthcare Quality Improvement Partnership; payment or honoraria from Danske Fysioterapeuter and Journal of Bone and Mineral Research Plus; and an unpaid committee role with the Royal Osteoporosis Society. MG reports grants from Versus Arthritis and Clarendon Fund. RKJ reports grants from the Ministry of Defence. CLLM reports grants from H2020 ITN, Disc4All, Flexion Plc (Contract Research Funding), Engineering and Physical Sciences Research Council (EPSRC), MRC, and Versus Arthritis; consultancy fees and royalties or licensing fees from Flexion; committee roles for


Language proficiency and comprehension in the culture and health systems of the host country are imperative factors enabling appropriate health literacy (HL) for non-English speaking immigrants. Older immigrants with limited English proficiency have fewer opportunities and limited abilities to improve their English skills. Increasing individuals’ acculturation levels can be an effective strategy for older immigrants than solely educating English language skills. The purpose of this study is to investigate the mediating effect of acculturation between English proficiency and HL in older Korean immigrants. From June to October of 2020, a total of 244 older Korean immigrants aged 50 years or older residing in the Southern US were recruited. English proficiency was categorized into limited (very limited to fair) and fluent (fluent to very fluent). HL and acculturation were measured using the Health Literacy Survey-12 Questionnaires (HLS-Q12), and the East Asian Acculturation Measure (EAAM), respectively. Covariates were gender, age, monthly income, education, and length of residency in the U.S. The statistically significant differences were shown in age, gender, education, perceived health, and length of residency in the US by the level of English proficiency The limited English proficiency was negatively associated with HL  and acculturation had a mediating effect between English proficiency and HL . To alleviate the language barrier that causes low HL in older Korean immigrants, enhancing their understanding of the host country’s culture as well as improving organizational HL should be considered.