Little is known about the electronic collection and clinical feedback of patient-reported outcomes (ePROs) following surgical discharge. This systematic review summarized the evidence on the collection and uses of electronic systems to collect PROs after discharge from hospital after surgery.Systematic searches of MEDLINE, Embase, PsycINFO, CINAHL and Cochrane Central were undertaken from database inception to July 2019 using terms for ‘patient reported outcomes’, ‘electronic’, ‘surgery’ and ‘at home’. Primary research of all study designs was included if they used electronic systems to collect PRO data in adults after hospital discharge following surgery. Data were collected on the settings, patient groups and specialties, ePRO systems (including features and functions), PRO data collected, and integration with health records.Fourteen studies were included from 9474 records, including two RCTs and six orthopaedic surgery studies. Most studies (9 of 14) used commercial ePRO systems. Six reported types of electronic device were used: tablets or other portable devices (3 studies), smartphones (2), combination of smartphones, tablets, portable devices and computers (1). Systems had limited features and functions such as real-time clinical feedback (6 studies) and messaging service for patients with care teams (3). No study described ePRO system integration with electronic health records to support clinical feedback.There is limited reporting of ePRO systems in the surgical literature, and ePRO systems lack integration with hospital clinical systems. Future research should describe the ePRO system and ePRO questionnaires used, and challenges encountered during the study, to support efficient upscaling of ePRO systems using tried and tested approaches. This systematic review examined the use of electronic systems to collect patient-reported outcomes (ePROs) post-discharge after surgery.Fourteen studies were included - most used commercial ePRO systems, few reported the types of electronic devices used, systems had limited features and functions and none described integration of the system with electronic health records.There is limited reporting of ePRO systems in surgical literature. Routine electronic collection of patient-reported outcomes (ePROs) and the electronic communication (feedback) of these data to patients and clinicians through the ePRO system are associated with improved clinical outcomes and enhanced quality of life for patientsMost ePRO research has focused on the collection and application of data during outpatient appointments or hospital admissions. Typically, such studies have focused on patients receiving specific treatments, most notably those receiving chemotherapy,Evidence for the use of ePRO systems in other patient groups, such as those undergoing surgery, is sparse. Patients who require a hospital stay for surgery may experience a wider range of, and more serious, complications after surgery than those who undergo surgery as a day case. With increasing use of enhanced recovery after surgery (ERAS) programmes to support quicker and better recovery,Appendix S1. Reference lists of publications included in the full review were searched to identify further studies for inclusion.The review was conducted according to the PRISMA guidelinesPrimary research studies on ePRO data collection were eligible for inclusion: randomized, quasi-experimental, and observational studies, including case series and case reports.The review included studies on adult patients aged 18 years or older who met all the following criteria: admitted to hospital; an inpatient stay of at least one night (not planned day cases); received surgery; and subsequent discharge. Studies on patients receiving more than one treatment were included if at least one treatment was surgical and was received during an inpatient stay.Studies on the use of electronic systems to collect PRO data (ePRO data) were included when at least some data were collected after discharge, outside the inpatient or outpatient healthcare settings (for instance in patients’ homes). Studies with ePRO data as primary or secondary outcome measures were eligible for study inclusion.Studies were excluded if they reported the development or set-up of ePRO systems  but did not describe and report results for collected ePRO data. The review also excluded narrative and systematic reviews, editorials, commentaries, opinion papers, letters, education papers, conference abstracts, protocols, reports, theses or book chapters, as they were unlikely to contain sufficient detail about the features and functionality of individual ePRO systems. Articles in languages other than English were also excluded.Titles and abstracts of articles were screened independently by two reviewers using the predefined eligibility criteria. Disagreements between the two reviewers were resolved by a third reviewer. Where abstracts were unavailable or the content inadequate to decide on eligibility, full articles were obtained and reviewed. Screening of articles, including removal of duplicates, was managed using EndNote™ X9 software .® . Three reviewers independently extracted data, with additional review to confirm the accuracy and completeness of data extraction. Discrepancies or disagreements about extracted material were resolved discussion with another reviewer.Data on the following topics were extracted from full-text articles after screening: publication details; study design; patient, disease and treatment characteristics; study methods; ePRO system and its features; ethics and governance; and results. A data extraction template form was developed for the review to ensure standardization and consistency in this process. Extracted data were collated in Microsoft ExcelA narrative synthesis of data, with descriptive analyses where appropriate, was undertaken. Risk of bias in individual studies and across studies was not assessed as only descriptive analyses were undertaken in this study and quantitative synthesis techniques  were not applied. A meta-analysis was not performed as the aim of this review was to identify and describe, rather than to establish, the effectiveness of, ePRO systems.Fig. 1). Of these, 145 (91.2 per cent) were excluded after full-text screening. The remaining 14 articles,Table S1). These included studies were published between 2003 and 2019. Approximately half of studies (8 of 14) were conducted in the USA, and the others were done in Europe. There were two RCTs. Two studies were conducted at more than one site (10 sites in each study).A total of 9474 articles were retrieved. After excluding ineligible articles during initial screening, 159 publications were obtained for further eligibility assessment  participated and 2424 (ranging from 15 to 1076) completed follow-up. Almost half of the studies (6 of 14) were in orthopaedic populations, such as patients who received knee, hip or shoulder arthroplasty, and those who had an anterior cruciate ligament repair. Other patient groups included those who had colorectal cancer surgery (1 study), vascular surgery (1) or gynaecological surgery (1). The two RCTs studied the use of ePRO systems in patients undergoing surgery for vascular disease and colorectal cancer.Where stated (13 studies), the length of patient follow-up ranged from 5 days to approximately 2 years. All but one study reported the method(s) used for patient follow-up. Almost half of the studies (6 of 14) used multiple methods that included at least two of the following: the ePRO system; e-mail; telephone; and face-to-face contact. The other studies reported the sole use of the ePRO system (6 studies) or only e-mail (1 study) for follow-up. In 10 studies the primary outcome measure was patient-reported, for example the European Organization for Research and Treatment of Cancer (EORTC) Core Quality of Life Questionnaire (QLQ-C30) version 3.0, Memorial Sloan Kettering Cancer Center Bowel Function Instrument (MSKCC BFI) and visual analogue scale (VAS) pain score. The remaining four studies used physical activity sensors for continuous or automated data collection as the primary outcome measure.In most studies (9 of 12) that described the ePRO system software, a commercial product was used. Two other studies reported the use of software developed in-house. The types of electronic device used to collect ePRO data were reported in six studies: tablet/portable device (3 studies); smartphone (2), and combinations of smartphone, tablet/portable device and computer (1).Table 1). Fewer studies reported the use of other ePRO features and functions, such as a messaging service for patients to communicate directly with their care team (3 studies), automated patient reminders to complete questionnaires or submit data (3), or graphical displays for patients to view reported symptoms (1).None of the 14 studies reported that the ePRO system was integrated with EHRs. Feedback of ePRO data to clinicians was rare: six systems provided real-time feedback to clinicians and four provided delayed feedback  collected more than one type of PRO data through the ePRO system. Postoperative symptoms and problems were most commonly collected (13 studies), such as pain and sleep disturbance. Few studies collected data on quality of life (4). Most studies (11) used at least one validated questionnaire to measure PROs. These included the MSKCC BFI, the Western Ontario Shoulder Instability Index (WOSI), and modified instruments such as a single item adapted from the Subjective Significance Questionnaire (SSQ). The three other studies reported the use of diaries (1 study) and questionnaires (2 studies) developed specifically for the study, with no information on whether these instruments were validated before use in the respective studies. Where formats in which PRO data were collected were described, these ranged from VAS ratings (3 of 12); ordinal scoring including Likert scales (11 of 12) and simple count data (7 of 12). The frequency of data collection using the ePRO systems varied, ranging from a single time point to annually. Six studies collected data once or twice a day. Studies that included the collection of physical activity data reported that data were collected and uploaded passively, either continuously and/or automatically (5 studies).Approximately half of studies (7 of 14) collected data on patients’ satisfaction with the electronic reporting system. One study assessed the economic impact of digital rehabilitation (including ePRO) compared with home visits from healthcare professionals in addition to digital rehabilitation after total hip arthroplasty. It found that the use of only electronic applications to support rehabilitation, including ePROs, was clinically non-inferior to digital rehabilitation in addition to home visits, but was significantly cheaper. No other study explicitly considered whether or how ePRO systems affected clinical or patient-reported outcomes.This review identified 14 studies published over a 16-year period examining how electronic systems are used to collected PRO data after surgery. They were conducted in the USA and Europe, and mostly used commercial ePRO systems. These systems were often used in conjunction with other instruments, and therefore their use in future research and clinical applications should consider the challenges of implementing and interpreting results from multiple tools. None of the studies reported that the ePRO system was integrated with EHRs, despite the importance of this process to facilitate clinical acceptability and assimilation of ePRO data into routine clinical practice,,,Less than one-third of studies reported that ePRO results were used directly to inform the clinical care of individual patients. There is interest in innovative uses of ePRO systems in routine careThis is the first systematic review specifically to investigate the use of electronic systems to collect PRO data after discharge from hospital following surgery. The findings were derived from a thorough search of five electronic databases. By including results from all study designs (randomized and non-randomized studies), a comprehensive review of research evidence was achieved. However, only articles in the English language were included. It is therefore possible that some relevant studies were omitted from the review, although the number of, and reasons for, excluded articles have been presented. This review reported on patient groups and specialties, such as orthopaedic surgery, but did not include detailed information about patients’ demographic and clinical characteristics. Future research should ensure that data on patient characteristics are collected and reported, to support identification of specific patient groups that engage (and conversely groups that engage less) with the use of ePRO systems.Additionally, the results may be not be generalizable to healthcare systems across the world, as 8 of the 14 reviewed studies were conducted in the USA. No studies from Asia, Australia or Canada were included in the review, and so the findings may not accurately reflect current ePRO system use in research or clinical practice in those continents. For example, unpublished material (such as recently completed studies) was also excluded, which might have affected the conclusions drawn. The focus of this review was on ePRO systems used to collect data after patient discharge following surgery. Studies on day-case surgeries were excluded as the process and duration of recovery from surgery, and common postoperative symptoms and problems, are typically distinct to operations requiring a hospital stay. Consequently, studies reporting on relevant ePRO systems may have been omitted. None of the 14 reviewed studies reported technical or methodological challenges encountered in setting up or using ePRO systems. This information on lessons learned is important to guide future research. However, articles describing this information may be missing from this review as feasibility studies that did not describe and report on collected ePRO data were excluded.Few studies have examined the use of electronic systems to collect PRO data after discharge following surgery, and very few of these were RCTs. Most studies used commercial software to collect ePRO data, but there was lack of integration with hospital clinical systems. Future research may wish to include studies with day-case surgery populations where relevant ePRO systems are used. Reporting of future studies should be comprehensive and transparent, to include difficulties related to the ePRO system that are encountered during study design or implementation, and any potential solutions. Studies should also state the name of the ePRO system software, the electronic devices used to collect ePRO data and, where non-validated or modified questionnaires are used, these should be described.zraa072_Supplementary_DataClick here for additional data file.


Biologists since Darwin considered domestication a model for the study of evolution; we argue that domestication may also be a model for the study of globalization.The long-term history of wheat and sheep domestication exemplifies the intensification of relationships between humans and a small number of species native to southwest Asia, which includes long-term globalizing processes.Specific indicators are offered for tracking the long-term globalization of sheep and wheat, with reference to production intensity, geographic diffusion, and diversity.Domestication as here understood is one outcome of human–environment interactions whereby certain plants and animals undergo genetic changes resulting from their close relationship with humans, including increasing reliance on humans for survival and reproductive success. Domestication is thus an ongoing process and may be viewed as part of an even broader process of intensification in the relationships between humans and certain plants and animals, including hunting/gathering, herding/cultivating, specialized agriculture/pastoralism, and, recently, genetic engineering. It should be emphasized that these are not stages in a necessarily directional process, but these categories do represent a scale of intensification, at least in the strict agricultural sense of more plant/animal product per unit land . DomestiWhereas the tradition of studying domestication as a model for evolution goes back to Darwin, we argue that domestication research also offers a model for the study of globalization. This suggestion ensues from the insight that several components of the meta-trajectory outlined below as intensifying relationships between humans, wheat, and sheep, are manifest in many other ongoing economic, social, and ecological processes. These can be broadly summarized as “globalization” in the widely accepted sense of intensifying worldwide interconnectedness, including in economic, cultural, political, and environmental spheres : 2. Our Ovis orientalis), with little evidence for genetic input of other wild congenerics  to extant or archaeological populations  in the mid-9th millennium BCE . A natural classification system groups wheat species by chromosomal ploidy (multiples of distinct sets of chromosomes) and combinations of distinct genomes  gradually phased out of cultivation for their lesser value to commercialized and globalized economies of antiquity and modern times, particularly in tandem with 20th-century globalization of free-threshing hexaploid bread wheat cultivation . Hulled A powerful combination of southwest Asian plant and animal domesticates emerged in the Neolithic—an “agricultural package”—of which wheat and sheep are exemplary. Increasing evidence suggests that even after initial domestication, cultivation and livestock rearing developed by numerous and diverse pathways, including much trial and error . AlthougThe most basic form of such diversity is that deriving from the combination of plants and animals. This not only provides a source of dietary diversity, as does hunting and gathering, but also an added level of risk management associated with agropastoral storage. Whereas wheat grains, among other cereals and legumes, can be stored in permanent settlements for food and sowing, sheep and other livestock are a highly mobile source of food and capital. Together, the combination of stationary and mobile storage provides a wide range of adaptations to environmental anomalies mediated by diverse cultural modes. The development of specialized nomadic pastoralism is a kind of intersociety adaptation on this theme, developed to maximize landscape exploitation by focusing grazing on regions less suitable for agriculture. This perspective is supported by the high degrees of interdependence between specialized pastoralists and farmers, alongside tensions over scarce land and sociocultural differences. Much of later southwest Asian history can be written in terms of these relationships and differences, following the lead of A different type of diversity contributing to agropastoral buffering capacity involves the set of trade-offs between sheep and wheat vis-à-vis their respective counterparts, goats and barley. Both sheep and goats provide meat, milk, and hides; both wheat and barley provide kernels for food and fodder, as well as chaff and straw for fodder, kindling, building, and other crafts. However, while offering essentially the same products, each member of the pair has slightly different ecological needs and adaptive qualities, with barley and goats generally representing the hardier counterparts to the higher-valued products of sheep and wheat among most ancient and modern southwest Asian cultures. These differences may be exploited in various ways and circumstances, including risk management. For example, drought-tolerant barley often succeeds where wheat crops fail, while slightly different ripening times between wheat and barley in southwest Asia offer a buffer against subseasonal precipitation anomalies.In addition to interkingdom and intergenus diversity just discussed, intragenus and intraspecies diversity presents another gamut of possibilities for economic exploitation, utilized by breeders for millennia. For instance, changes under domestication to seasonal cyclicity in reproduction, involving flowering time adaptations for wheat and multiple lambing seasons in sheep, were key to their global diffusion. Just as genetic diversity has influenced the globalization of sheep and wheat, human socioeconomic globalization has affected their genetic diversity. The spread of these species to diverse and often remote regions catalyzed the development of numerous breeds and varieties , creating a global force for increased intraspecies diversity—a diversity which most people throughout history were unaware of. Contemporary globalization has made this agriculturally significant diversity uniquely accessible in theory, as through gene banks, while causing declining diversity of cultivated/herded stock in practice as landraces become marginalized and extinct. These two countercurrents epitomize contemporary globalization generally: increased awareness of global diversity thanks to heightened connectivity between disparate regions on the one hand, and increased uniformity in cultural, social, and economic spheres on the other hand. If globalization widely conceived is a stage in the intensification of economic, cultural, political, and environmental interconnectedness, the globalization of sheep and wheat is a stage in domestication and agropastoral intensification, the tracking of which may broaden our understanding of contemporary globalization. We propose the following indicators for sheep and wheat intensification with relevance to long-term globalization:(1) Sheep:goat and wheat:barley ratioCentralized and market-oriented production appear to favor both wheat and sheep vis-à-vis barley and goats, as well as specific varieties/breeds of each. By the Early Bronze age, wheat and sheep were involved in increasingly extractive, landscape-altering human lifeways, which was part and parcel of the rise of urbanism and empires. Whether in Ur III, the Assyrian Levant, or the Roman Mediterranean, local maxima in wheat and sheep production over time attest to heightened societal complexity, defined simply as increasing energetic inputs and problem-solving outputs .(2) Population densityIncreasing population density may occur on highly local and global levels. The former may involve, for example, intensive rearing of large herds in pens, supported by cultivated fodder. The latter includes global population levels of sheep and goats, which in a globalized world correlate with population densities in “core” areas.(3) Geographical diffusion The extreme dispersal of wheat and sheep globally  has been(4) Ratio of species population to number of extant agriculturally significant varieties and breeds.The globalization of wheat and sheep is also associated with increasing uniformity in the varieties and breeds being raised. In post-Neolithic times, this process includes gradual phasing out of einkorn and other hulled wheats, for example, and the global dominance of “bread wheat.”(5) Geographic distribution of diversity in varieties and breeds. In a complex society as defined above, higher uniformity in varieties and breeds is expected along the major trade routes. Evenness in the geographic spread of rare landraces is expected to be a function of distance from primary economic and sociopolitical conduits.is geographic diffusion; while indicators (4) and (5) are agropastoral expressions of decreased cultural and genetic diversity  and (2) relate directly to production intensity; indicator (3)


We reviewed and coded the recovery plans to identify health-related activities. While 70% and 64% of reviewed plans included activities to address short-term healthcare and behavioral health needs, respectively, one-third or less of the plans included activities to address long-term healthcare and behavioral health needs. Further, plans have limited descriptions of health-related data collection, analysis, or data-driven processes. Additional evidence-informed public health requirements and activities are needed in disaster recovery implementation plans. State disaster recovery plans would benefit from additional description of public health roles, responsibilities, and activities, as well as additional plans for collecting and analyzing public health data to drive recovery decision making and activities. Plans should include approaches for ongoing evaluation of recovery activities.As communities recover from disasters, it is crucial to understand the extent to which states are prepared to support the recovery of health systems and services. This need has been emphasized by the United States’ experience with COVID-19. This study sought to assess public health activities in state disaster recovery implementation plans. In this exploratory, descriptive study, state-wide disaster recovery implementation plans were collected from emergency management agency websites and verified ( The Federal Emergency Management Agency (FEMA) describes the National Preparedness Goal’s disaster recovery mission area as “capabilities necessary to assist communities affected by an incident to recover effectively,” and notes that recovery entails “timely restoration, strengthening and revitalization of infrastructure, housing and a sustainable economy, as well as the health, social, cultural, historic and environmental fabric of communities” . FEMA’s Recent work in the disaster recovery field has sought to further delineate health-related roles and activities in recovery. Key responsibilities of the health sector during disaster recovery include determining priorities for recovery; assessing recovery worker and community member health impacts and needs; supporting the restoration and provision of public health and healthcare services and systems; communicating with the public about available resources and services and health impacts and risks; collaborating with partners to support equitable recovery strategies; and conducting mitigation activities to improve resilience to future disasters and impacts of climate change ,4,5,6,7.Collaborative and dynamic planning processes and partnerships have also been identified as an essential aspect of public health preparedness  and commIn addition to clarifying the role of the health sector during recovery, emphasis has been placed on the development of disaster recovery strategies that preemptively consider health impacts of strategies taken by other sectors ) . The secAssessing states’ preparedness to incorporate health considerations in recovery is particularly timely and significant in light of the ongoing COVID-19 pandemic, due to its disproportionate effects on marginalized groups, widespread impacts to physical and mental health and the social determinants of health, and tremendous strain on health systems and workers ,23,24,25Yet, it remains unclear whether and how guidance about integrating health considerations into recovery has been formalized in the context of state disaster recovery implementation plans. In order to assess the readiness of states to overcome the aforementioned barriers and integrate health considerations in disaster recovery, this exploratory, descriptive study sought to characterize health-related activities proposed in state disaster recovery plans and generate hypotheses that can be tested through future research.State disaster recovery plans  were collected from state emergency management websites in the summer of 2017. State emergency management agencies were contacted in the fall to confirm that the plan was the current and primary recovery plan, or to identify current plans if none were online. We attempted to collect or confirm plans through December 2017. A total of 34 plans were collected and confirmed. A total of 33 were coded and analyzed; one was excluded because it explicitly stated that recovery was outside of its focus. The remaining states failed to respond to our requests to confirm or obtain a copy of their plan, or did not have a recovery plan. No human subject data were obtained or analyzed for this study.Two investigators (N.A.E. and S.A.G.) reviewed all 33 plans and developed a codebook and coding questions based on the recovery activity lists in the 2015 National Academy of Medicine (formerly the Institute of Medicine) Healthy, Resilient, and Sustainable Communities After Disasters report  and thisGiven the complexity of collecting health-related information or implementing health-related services and the associated need for specific planning, our analysis only captured explicit health-related activities/strategies and did not capture activities/strategies that may have an implied health component . Given the lack of accepted recovery timelines, we deferred to the plans’ description of “short term” and “long term.” We considered activities short term unless the plan explicitly stated they were to be completed or continued in the long term.Introduction, including mission/vision/purpose or equivalent;Plan organization  structure);Health-specific section of the plan (if applicable) or entire plan .Two investigators (N.A.E. and M.K.) reviewed and coded the following plan components for all 33 plans:See We discussed and adjudicated all discrepancies in applied codes. NVivo qualitative analysis software  was used to code the plans and Microsoft Excel was used to capture responses to the coding questions and calculate descriptive statistics.n = 13) of coded plans used an RSF format and 27% (n = 9) used the RSFs outlined in FEMA’s NDRF [State disaster recovery plans analyzed were formatted in a variety of ways, including as annexes to comprehensive emergency management plans, emergency support functions, or standalone emergency plans. A total of 39% (The majority of reviewed plans (70%) explicitly mention building back better in the mission/vision/purpose  of the overall plan, and approximately one-third (30%) explicitly mention health in the mission/vision/purpose  . While 7Plans have limited descriptions of health-related data collection or analysis or data-driven processes. While most plans (67%) describe collecting or using health-related data in initial post-disaster recovery assessments, few plans reference utilizing pre-disaster health-specific assessments or data (15%), conducting ongoing health-specific assessments (27%), conducting surveillance activities (33%), or evaluating health-related recovery activities (9%) .Our findings suggest that while some states have planned to implement a variety of health-related activities during recovery from disasters, including from COVID-19, there is a significant opportunity to improve the ability of recovery plans to facilitate health promotion in recovery. Templates, examples, and guidance may facilitate the development of recovery plans that more clearly articulate and facilitate the execution of public health roles in recovery. The National Association of County and City Health Officials’ (NACCHO) 2018 “Public Health Recovery Landscape Analysis”—a review of 21 local health department recovery plans/annexes and nine federal/national resources, and seven key informant interviews to better understand the state of local recovery planning—identified needs for publicly available plan examples, a unified command process for recovery, and clearer roles and responsibilities for different stakeholders in recovery , suggestCharacteristics of evidence-based public health include sound evaluation and systematic use of data and information systems . As suchSimilarly, recovery-related training may enhance public health workers’ understanding of and ability to plan for and implement health-related recovery functions. Free and publicly available online trainings, such as the National Center for Disaster Medicine’s Public Health System Training in Disaster Recovery, offer the potential for widespread access to recovery-related training . RecoverThe lack of attention to healthcare workforce issues and addressing long-term health needs in state recovery plans may also affect the success of COVID-19 recovery, given the pandemic’s widespread health effects and impacts to the healthcare workforce. Documentation of effective strategies and lessons learned related to recovering the healthcare workforce and addressing long-term healthcare and behavioral health needs during COVID-19 recovery could facilitate the inclusion of these areas in future disaster recovery planning efforts.Our study has several limitations. First, this exploratory analysis is limited to state-wide disaster recovery plans, and further limited to specific sections of plans that were most likely to describe health-related recovery activities. It is possible that states or health departments have outlined implementation of health-related recovery activities in other plans; for example, in health department-specific plans or comprehensive emergency plans without specific recovery sections. In addition, we were only able to obtain and analyze plans from 33 states, meaning that plans from one-third of states were not included in our study. It is possible that the states that were able to confirm or provide plans had different characteristics than the states not represented in our analysis, so study results should be interpreted with this in mind. Additionally, as “community recovery” remains one of the core capabilities of the CDC’s Public Health Emergency Preparedness Cooperative Agreement [Future research should examine whether and how the COVID-19 pandemic motivated recovery planning and/or integration of health-related activities or requirements in recovery plans. Future studies could also assess the extent to which state approaches to COVID-19 recovery diverged from the actions prescribed by their pre-disaster recovery plans, evaluate the implementation and/or effectiveness of recovery strategies employed, and assess differences in planned recovery actions based on state characteristics, such as region, level of resources, or political leanings. Moreover, as the NDRF calls for local leadership in recovery , additioOur findings suggest a need to improve the integration of health considerations in state disaster recovery planning. State disaster recovery plans could benefit from additional detail on health sector roles and responsibilities, data collection and analysis to inform recovery decision making and evaluate recovery strategies, strategies for addressing healthcare workforce issues, and approaches for providing long-term healthcare and behavioral health services. This study may also have implications for COVID-19 and other ongoing disaster recovery and the extent to which states are prepared to leverage recovery processes to promote population health, advance health equity, and strengthen resilience for future disasters.


Background: Neonatal infections represent one of the six main types of healthcare-associated infections and have resulted in increasing mortality rates in recent years due to preterm births or problems arising from childbirth. Although advances in obstetrics and technologies have minimized the number of deaths related to birth, different challenges have emerged in identifying the main factors affecting mortality and morbidity. Dataset characterization: We investigated healthcare-associated infections in a cohort of 1203 patients at the level III Neonatal Intensive Care Unit (ICU) of the “Federico II” University Hospital in Naples from 2016 to 2020 (60 months). Methods: The present paper used statistical analyses and logistic regression to identify an association between healthcare-associated blood stream infection (HABSIs) and the available risk factors in neonates and prevent their spread. We designed a supervised approach to predict whether a patient suffered from HABSI using seven different artificial intelligence models. Results: We analyzed a cohort of 1203 patients and found that birthweight and central line catheterization days were the most important predictors of suffering from HABSI. Conclusions: Our statistical analyses showed that birthweight and central line catheterization days were significant predictors of suffering from HABSI. Patients suffering from HABSI had lower gestational age and birthweight, which led to longer hospitalization and umbilical and central line catheterization days than non-HABSI neonates. The predictive analysis achieved the highest Area Under Curve (AUC), accuracy and F1-macro score in the prediction of HABSIs using Logistic Regression (LR) and Multi-layer Perceptron (MLP) models, which better resolved the imbalanced dataset . The European Centre for Disease Prevention and Control (ECDC) recognized neonatal infections as one of the six main types of healthcare-associated infections (HABSIs) in 2016 , whose mThe present paper performed a risk assessment using statistical analyses and logistic regression to reveal the main factors that affect neonates suffering from a healthcare-associated blood stream infection (HABSI) infection ,10. The This paper is organized as follows. The results are shown in This section discusses the characterization of the study population, which was composed of 1203 infants who stayed longer than two days in the level III NICU of the “Federico II” University Hospital in Naples from 2016 to 2020. These patients were continuously monitored by healthcare staff in all hospital ICUs and suffered from healthcare associated blood stream infection (HABSI). In particular, HABSIs were observed in the next two days after the date of admission at the NICU on the basis of the Disease Control and Prevention directive for neonatal acute care settings . The datNeisseria), Gram enteric bacilli , anaerobic bacteria (Mushrooms), chlamydia (Protozoa) , and mycoplasmas (Viruses). However, the association with neonatal disease was significant only for Group A and Group B streptococci, Escherichia coli, Neisseria gonorrhoeae, CMV, HSV type II, Candida albicans, and Chlamydia trachomatis.Viral infections may be vertically transmitted from mother to child at different times, from “in utero” transmission, which occurs during the gestational age, perinatal transmission, which occurs during delivery, and postnatal transmission, which is generally a consequence of breastfeeding. Mother-to-child transmission, which may occur after primary, recurrent or chronic maternal infection, is potentially harmful to the fetus or the newborn because it may result in miscarriage, fetal death, congenital anomalies, intrauterine growth restriction or severe neonatal disease. Some risk factors may affect the rate of mother-to-child transmission, such as the presence of other viral infections, maternal viral load, type of infection (primary versus recurrent), obstetrical procedures (prolonged rupture of membranes and mode of delivery), socioeconomic conditions and breastfeeding. For some vertically transmitted viruses, interventions are available to prevent mother-to-child transmission, such as vaccines, passive immunization, and antiviral drugs. Perinatal and postnatal infections may be prevented with elective caesarean delivery and avoidance of breastfeeding . Fetal ihttps://www.aslsalerno.it/documents/20181/147671/PianoRegionaleICA.pdf/67a62ec9-ade9-4552-b389-5b8dabe72e6a, accessed on: 5 January 2022).We consider different clinical information , gestational age (in weeks), sex, length of total hospital stay and invasive device exposure  for each patient. The patients studied were not stratified by disease severity, but the database used reported the score of American Society of Anesthesiologists (ASA). We had mortality data for both groups, but they were not included in the processed dataset. Notably, patients informed consent and local Ethical Committee authorization were not required, and all of the data came from HAI surveillance, which is regulated by the Regional Health Authority , ensures the integrated exercise of care, teaching and research and the unitary management of economic, human and instrumental resources. Twice a year (November and March), the Federico II University Hospital of Naples participates in the prevalence study of HAI organized by the ECDC, which allows the comparison of the data on the European territory and the evaluation of its trend over time to highlight any critical changes. In high-risk areas, such as NICUs, incidence studies are performed with staff who collect data in the risk records processed for each patient from admission to discharge. The data allow the preparation of reports on a quarterly basis subject to clinical audits with medical and nursing staff working in the affected areas. Training courses are organized at least three times per year for all health personnel, with compulsory attendance as part of working hours.p-value was less than the threshold value We performed our analysis using the chi-squared test, Fisher’s exact test and Kruskal-Wallis test, as appropriate, but the relationships between the dependent variable (HABSIs) and the different risk factors under study  were investigated using logistic regression. Notably, we considered associations significant when the https://colab.research.google.com/, accessed on: 5 January 2022), a Platform As A Service (PAAS) platform provided by Google with one single core hyperthreaded Xeon Processor @2.2 Ghz, 12 GB of RAM and a Tesla T4 GPU, using Python 3.6 with the scipy , statsmodels , scikit-learn , catboost.ai  and skranger  libraries.Finally, the analysis was performed on Google Colab  and the possible occurrence of HABSI disease (as dependent variable). Notably, the population under study included all of the neonates who stayed at least two days at the NICU of the Federico II from 2016 to 2020 (60 months) for a total of 1203 neonates, and 65 suffered from HABSI . NeonateThis section provides a predictive analysis on the basis of different risk factors that were first normalized and successively processed to predict whether a patient suffered from HABSI. We evaluated our results in terms of accuracy, AUC, and F1-score, whose formal definitions are provided in Equations  and Macrtrain_test_split  function of scikit-learn; Notably, samples of the two classes in the training set were balanced using the data augmentation method. The training set was further divided into two different sets. The first set was used to train our models by varying their parameters, which were evaluated on the second set to optimize each AI model. We provided a statistical validation of our results by providing a ten-cross validation on our dataset that corresponded to jointly varying the training and test sets to show our model different data for each iteration (ten times) through the cross_validate  function of scikit-learn. The best parameters for each model were reported in Different Artificial Intelligence models (Support Vector Machine (SVC), CATBOOST, XGBoost (XGB), Ranger Forest Classifier (RFC), Multi-Layer Perceptron (MLP), Random Forest (RF) and Logistic Regression (LR)) were used to predict whether a patient suffered from HABSI. We analyzed a cohort of 1203 neonates who were divided into Different surveillance studies were performed to determine how the condition of the host organism strongly connected to the onset of HAIs, which showed that patients suffering from infection rates among 6 and  surgery . Notably surgery . A Carri surgery  performeA large amount of data is involved in the healthcare process or service, which were analyzed using different approaches . We provided a statistical validation of our results by providing a ten-cross validation on our dataset, which corresponded to jointly varying the training and test sets to show our model different data for each iteration (ten times), and the best parameters for each model are shown in We performed statistical analyses on a cohort of 1203 patients (65 who suffered from HABSI) to investigate risk factors for the activity of the NICU of the University Hospital “Federico II”. Notably, we emphasized that patients suffering from HABSI had a shorter gestational age and lower birthweight, which led to longer hospitalization and umbilical and central line catheterization days than non-HABSI neonates. Statistical analyses showed that significant predictors of suffering from HABSI were birth weight and central line catheterization days. In summary, the current analysis supports health practitioners in improving standards and processes to contain HAIs.In summary, our analyses showed that birthweight and central line catheterization days were significant predictors of suffering from HABSI, and the LR and MLP achieved the highest AUC, accuracy and F1-macro score in the prediction of HABSI. Despite different efforts in the literature for the predicting the risk of HAIs and other adverse outcomes in ICU, these studies primarily focused on the use of Support Vector Machine (SVM) ,17 or diOur study had some limitations due to the lack of temporal analysis. However, it did not consider the temporal features between entry into intensive areas and the first clinical evidence of infection, as shown in previous studies  and otheFuture works will increase the number of analyzed patients and investigate different infection types to identify possible corrective actions on the basis of the Lean Six Sigma method  and possThe present paper investigated risk factors related to activities in the NICU of the University Hospital “Federico II” in a cohort of 1203 patients (65 suffering from HABSI). We performed a risk analysis using logistic regression and statistical analyses to identify the main factors  affecting neonates to contract a healthcare associated blood stream infection (HABSI). We successively designed a supervised approach using different artificial intelligence models to predict whether a patient would contracted a HABSI and found that MLP and LR better handled the imbalanced dataset , and the highest AUC and F1-macro score was observed. In summary, our analysis identified the main risk factors related to NICU activities of the University Hospital “Federico II” to jointly improve patient care of the NICU of the University Hospital “Federico II” and support practitioners in their analysis. The predictive analysis may be used in a decision support system to support practitioners in predicting and identifying hidden symptoms of healthcare-associated blood stream infections in the neonatal intensive care unit.


Megaptera novaeangliae) feeding within a fjord system in the Canadian Pacific (primarily within Gitga’at First Nation waters), we have documented long-term pair bonds (up to 12 years) as well as a complex societal structure, which corresponds closely to persistent patterns in feeding strategy, long-term site fidelity , specific geographic preferences within the fjord system, and other forms of habitat use. Randomization tests of network congruency and clustering algorithms were used to test for overlap in patterns of social structure and habitat use, which confirmed the occurrence of social niche partitioning on the feeding grounds of this baleen whale species. In addition, we document the extensive practice of group bubble net feeding in Pacific Canada. This coordinated feeding behavior was found to strongly mediate the social structure and habitat use within this humpback whale society. Additionally, during our 2004–2019 study, we observed a shift in social network structure in 2010–2012, which corresponded with environmental and demographic shifts including a sudden decline in the population’s calving rate. Our findings indicate that the social lives of humpback whales, and perhaps baleen whales generally, are more complex than previously supposed and should be a primary consideration in the assessment of potential impacts to important habitat.Animal culture and social bonds are relevant to wildlife conservation because they influence patterns of geography, behavior, and strategies of survival. Numerous examples of socially-driven habitat partitioning and ecological-niche specialization can be found among vertebrates, including toothed whales. But such social-ecological dynamics, described here as ‘social niche partitioning’, are not known among baleen whales, whose societies—particularly on foraging grounds—are largely perceived as unstructured and incidental to matters of habitat use and conservation. However, through 16 years of behavioral observations and photo-identifications of humpback whales ( Patterns of habitat use and the factors that determine it are of fundamental interest in animal ecology . As specEquus africanus in North America, ),the data . We usedthe data  to evaluthe data  to fit ethe data , Whitehethe data , Ramp et ),The shift in the humpback social network structure that we observed throughout the study is noteworthy for its coincidence with environmental changes and demographic shifts within the same time frame. Halfway through the study (2010–2012), we saw the network structure change from a fragmented community with a large number of relatively small social clusters to a more strongly interconnected community with a few disproportionately large social clusters . This coIt is possible that these environmental developments could be related to the shift we observed in social network structure within the KFS. Perhaps the increased social connectivity was in response to a lapse in resource dependability, which can force individuals to explore other habitats and strategies of habitat use, hence interacting with a larger number of individuals in the process. However, we lack the prey data necessary to parse the exact mechanics behind the changes we observed. Also note that the disruption to network modularity began prior to 2009 and did not result in a persistent phase shift . ResourcPseudorca crassidens; [Socially-mediated modes of habitat use, particularly those involving learning and culture, should be of intrinsic interest to conservation biologists working towards the preservation of biodiversity for its own sake , 80, 165ssidens; , 170).how a population uses a habitat, but also how much of its resources can be used—thus increasing the value of specific habitats even more.Furthermore, socially-mediated habitat use can improve resource utilization for a threatened population. Ecological studies have established that niche partitioning, in addition to enhancing biodiversity and ecosystem function, can increase a population’s access to resources . When coSocially-mediated resource-use efficiency becomes increasingly important as the extent and quality of a species’ habitat decline. Viable whale habitat is threatened worldwide by the expansion of coastal urban areas, which are associated with high rates of vessel traffic, increased ocean noise, and concentrated contaminants . UrbanizAs the availability and quality of habitat decline, it becomes increasingly urgent to effectively protect what remains. To do so, management plans should include measures that preserve each specific habitat use strategy practiced within the population, and should support ongoing research that is focused upon those site-specific strategies as well as the social dynamics that bring them about. Echoing the Expert Group on Culture and Social Complexity from the Convention on Migratory Species , we alsoThese management considerations are directly relevant to the KFS, a proposed critical humpback habitat area facing profound alterations in the next several years. At the head of the KFS is the port of Kitimat, the epicenter of western Canadian Liquified Natural Gas (LNG) project proposals , 183. ThIn order to understand the potential impact of increased noise on the social dynamics of this population, and its implications for habitat use and population viability, we suggest that the Government of Canada, Transport Canada, and BC First Nations who contribute to the decision-making and policy development in support of recovery planning for cetacean species at risk collaborate with biologists on the following research priorities: (1) Do shipping lanes affect the movements and social connectivity of baleen whales within a specific habitat? (2) Does increased noise impact the stability and frequency of pair bond interactions among baleen whale individuals? (3) How will proposed increases in shipping levels impact the hearing range and acoustic connectivity of baleen whale herds ? (4) DoeAlong the north coast of British Columbia, we have found that the social bonds and community structure of humpback whales mediate site-specific strategies of habitat use. The converse has also been true, particularly in the case of inherently social behaviors such as bubble net feeding. This reinforcing dynamic between sociality, habitat use, and place, which we have referred to here as social niche partitioning, highlights the inextricability of these whales’ ecological and social lives ; contribS1 File(1) Site fidelity, (2) Behavior, (3) Social associations, (4) Social preferences, (5) Network structure & stability. Supplementary tables and figures also included.(PDF)Click here for additional data file.S1 DatasetHumpback whale photo-identifications in the Kitimat Fjord System, 2004–2019.(CSV)Click here for additional data file.


Electronic medical record (EMR) systems have the potential to facilitate appropriate laboratory testing. We examined three common medical tests in primary care—hemoglobin A1c (HbA1c), lipid, and thyroid stimulating hormone (TSH)— to assess whether adoption of a laboratory EMR system in Ontario had an impact on the rate of inappropriate testing among primary care physicians.We used FY2016–17 population-level laboratory data to estimate the association between adoption of a laboratory EMR system and the rate of inappropriate testing. Inappropriate testing was assessed based on recommendations for screening, monitoring, and follow-up that take into account risk factors related to patient age and certain clinical conditions. To overcome the problem of potential endogeneity of physician choice to use the EMR, the EMR penetration rate in the physician’s geographical area of practice was used as an instrumental variable in an ordinary least squares (OLS) regression. We then simulated the change in the rate of inappropriate testing, by physician payment model, as the EMR penetration rate increased from the baseline percentage.The simulation models showed that an increase in the rate of EMR penetration from a baseline average was associated with a statistically significant decrease in inappropriate hbA1c and lipid testing, but a statistically insignificant increase in inappropriate TSH testing. The impact of EMR penetration also varied by payment model.This study demonstrated a positive association between availability of an EMR system and appropriate service utilization. Varying impacts of the EMR system availability by primary care payment model may be reflective of different incentives or attributes inherent in payment models. Policies to encourage physicians to increase their use of laboratory EMR systems could improve the quality and continuity of patient care.The online version contains supplementary material available at 10.1186/s12913-021-06296-5. Appropriate clinical testing is an important aspect of high-quality medical care , and theThis study examined the association between widespread availability of a laboratory EMR system and inappropriate testing among primary care physicians. We assessed inappropriate testing based on recommendations for screening, monitoring, and follow-up of each test for patients taking into account pertinent risk factors related to age and certain clinical conditions for three common laboratory tests used in primary care: hemoglobin A1c (HbA1c), lipid, and thyroid stimulating hormone (TSH).We conducted a cross-sectional observational study of the effect of EMR penetration on the rate of inappropriate testing in fiscal year (FY) 2016/17. Data were anonymized and acquired from the Ontario Ministry of Health and Long-Term Care (MOHLTC) under a data-sharing agreement with the Ontario Medical Association (OMA).Individual community laboratory test-level data were obtained from the Ontario Health Insurance Plan (OHIP) laboratory claims database for FY 2016/17. Data on physician’s age, sex, practice location area, and associated payment model were obtained from the OHIP Corporate Provider Database, and data on patient’s age and sex were obtained from the Registered Persons Database. Information on the percentage of EMR uptake (penetration) in FY 2016/17 by geographical area, known as local health integration network (LHIN) in Ontario, were provided by OntarioMD .Diagnostic information was extracted to summarize patient-level clinical complexity by processing physician billing data from OHIP claims, hospital discharge records from Canadian Institute for Health Information’s (CIHI) Discharge Abstract Database, and in-hospital and community-based ambulatory care records from CIHI’s National Ambulatory Care Reporting System using the CIHI Population Grouping Methodology , 13. CIHThe study population included all patients eligible and registered for OHIP coverage in FY 2016/17 who had at least one out-of-hospital hbA1c, TSH, or lipid test ordered by a primary care physician. The unit of analysis was the primary care physician. Physicians who ordered less than 100 specific labs per year may not represent the general population of primary care practitioners  and were excluded from the analysis to avoid biasing results.Individual physician use of the laboratory EMR may be endogenous since conscientious physicians may be both more likely to use OLIS and to follow guidelines for appropriate test ordering. Therefore, the percentage of EMR uptake (penetration) in FY 2016/17 within a physician’s geographical area was used as an instrumental variable in an ordinary least squares (OLS) regression. A table and map listing the EMR penetration rate by geographical area (LHIN) can be found in Additional file The laboratory tests analyzed in this study were hbA1c, lipid, and TSH. These tests were chosen because they are all commonly ordered tests for screening, chronic illness assessment or monitoring by primary care physicians. We employed two different approaches to define inappropriate testing. Firstly, we used repeat testing as a measure of inappropriateness based on accepted guidelines for screening, monitoring, and follow-up of each test that treats all patients the same, regardless of age and clinical history and is in line with prior research on inappropriate testing of hbA1c, lipid, and TSH , 15. UsiWe then introduced an enhanced approach to measure inappropriateness wherein criteria for hbA1c, TSH, and lipid testing were derived from guidelines and recommendations on testing for patients with risk factors based on age and/or clinical conditions that justified ordering a laboratory test –21. In tClinical conditions were identified in the data using patient-level diagnostic information from the CIHI Pop Grouper. The guidelines for each test based on patient age and clinical diagnosis, as well as a list of the specific clinical conditions/concerns where ordering additional laboratory tests may be warranted to guide clinical course, can be found in Additional file Patient risk scores (which reflect patient complexity as proxied by cost risk) were calculated based on the presence of one or more CIHI Health Conditions in the previous five years. These scores were then normalized to the population of patients included in this study who had laboratory tests done in Ontario in FY 2016/17 so that patients with a risk score greater than 1 were more complex than average for the study population, while those with a risk score less than 1 were less complex than average.We controlled for the physicians’ choice of payment and practice models: traditional fee-for-service (FFS); a retrospective payment model with elements of pay-for-performance (enhanced FFS); a predominantly prospective payment model mixed with FFS and elements of pay-for-performance (blended capitation); and a blended capitation model with an interdisciplinary team of family physicians, nurse practitioners, registered nurses, social workers, dietitians, and other health care professionals (interdisciplinary blended capitation model). Primary care physicians in other payment models were excluded, since these models comprise only a small proportion of the province’s physicians.First, we compared basic physician and patient descriptive statistics for the four different payment models. The descriptive statistics were presented separately for hbA1c, lipid, and TSH tests, and included the total number of physicians and patients, average patient age, percentage of male patients, average patient complexity, average normalized patient complexity, average physician age, average percentage of male physicians, average percentage of EMR uptake geographically, average percentage of inappropriate laboratory tests ordered, and average percentage of patients with at least one inappropriate test by payment model. The total number of patients in each payment model was not mutually exclusive. If the same patient received an hbA1c test order from both an FFS physician and a blended capitation physician, that patient would be counted twice in this metric. Descriptive statistics on both the standard and enhanced measures were reported.Y is the rate of inappropriate testing at the physician-level, X is the set of physician and patient control variables mentioned above, and Z is the percentage of penetration of the laboratory EMR system by geographical area. A second model was used to estimate the impact of EMR penetration on inappropriate testing by physician payment model as follows:EFFS is a dummy variable equal to 1 if the physician was in an enhanced FFS model and 0 otherwise; BC is a dummy variable equal to 1 if the physician was in a blended capitation model and 0 otherwise; IBC is a dummy variable equal to 1 if the physician was in an interdisciplinary blended capitation model and 0 otherwise. These dummy variables for physician payment model were included to estimate the association between adoption of the EMR system and the rate of inappropriate testing separately for each payment model. The reference, or omitted, category was the FFS model. The interaction variables of EMR penetration with the physician’s payment model represented the marginal association between the percentage of inappropriate testing and EMR penetration of FFS physicians compared to other physicians .Second, OLS regression was used to estimate the association between adoption of a laboratory EMR system and the rate of inappropriate testing for hbA1c, lipid, and TSH laboratory tests. Here, the enhanced measure of inappropriateness, based on patient age and clinical conditions, were employed. Physician and patient characteristics including physician age, physician sex, average patient age, percentage of male patients, and average patient complexity were included as control variables in the regression. The OLS model used for calculating the overall impact of EMR penetration on inappropriate testing was as follows:Z = EMR penetration mean at baseline). Results were reported in increments from increases of 5 percentage points up to 25 percentage points. For each physician payment model, the simulation would be calculated using model .Using the OLS regression results from the two models above, we simulated the change in the rate of inappropriate testing as the EMR penetration rate increased from the baseline percentage for the average rate of EMR penetration  tests.


Interviews revealed the need for a tool to help healthcare teams efficiently deliver tailored, evidence-based behavior change recommendations, motivate patients, and follow-up with patients within the constraints of clinic schedules and workflows. The PREVENT tool was developed to meet these needs. It facilitates prevention discussions, delivers tailored, evidence-based recommendations for physical activity and food intake, includes an interactive map of community resources to support behavior change, and automates patient follow-up. Based on Self-Determination Theory, the PREVENT tool engages the patient to encourage competence and autonomy to motivate behavior change. The use of this intentional, user-centered design process should increase the likelihood of the intended outcomes  and ultimately increase uptake, implementation success, and long-term results. After initial tool development, user-testing interviews (n = 13) were conducted using a think-aloud protocol that provided insight into users'  cognitive processes, attitudes, and challenges when using the tool. Overall, the PREVENT tool was perceived to be useful, well-organized, and visually appealing.Health information technology (HIT) has not been broadly adopted for use in outpatient healthcare settings to effectively address obesity in youth, especially among disadvantaged populations that face greater barriers to good health. A well-designed HIT tool can deliver behavior change recommendations and provide community resources to address this gap, and the Obesity-Related Behavioral Intervention Trials (ORBIT) model can guide its development and refinement. This article reports the application of the ORBIT model to (1) describe the characteristics and design of a novel HIT tool (the PREVENT tool) using behavioral theory, (2) illustrate the use of stakeholder-centered “designing for dissemination and sustainability” principles, and (3) discuss the practical implications and directions for future research. Two types of stakeholder engagement (customer discovery and user testing) were conducted with end users . Customer discovery interviews ( Obesity among youth in the United States, particularly those of low-socioeconomic status (SES), is a major public health concern that puts youth at risk for poor cardiovascular health (CVH) –3. ThereHealth information technology (HIT) has the potential to improve the quality, efficiency, consistency, and availability of healthcare across diverse populations , 18–20. Designing for Dissemination, Implementation, and Sustainability (D4DIS) refers to developing interventions that are closely aligned with the needs of end users and the intended context for use –25. The D4DIS strategies can inform intervention design features , implementation and sustainment strategies, and dissemination channels. We used D4DIS strategies (stakeholder engagement and the RE-AIM framework) within the process of the Obesity-Related Behavioral Intervention Trials (ORBIT) model  to develA major component of the ORBIT model is underpinning interventions with basic and social sciences research, which includes the use of behavioral theory, to increase effectiveness and allow developers to target theoretically derived drivers of behavior , 40. SelThe purposes of this report are to apply the ORBIT model to (1) describe the characteristics and design of the PREVENT tool using behavioral theory, (2) illustrate the use of D4DIS principles within the design Phase I of the ORBIT Model, and (3) discuss the practical implications and directions for future research (ORBIT Phase II–IV). This article will present two types of stakeholder engagement (customer discovery and user testing) used to design the PREVENT tool.The ORBIT model guided the development of a behavior change HIT tool (the PREVENT tool) designed to help pediatric healthcare teams  address overweight and obesity within routine care. This model proposes guidelines for developing behavioral treatments from the inception of an innovative approach through efficacy testing . Our goaThe first step in the ORBIT model is to identify and articulate a health need or clinical problem requiring a solution and generate a specific hypothesis. The goals of explicitly identifying the clinical problem are to (1) include primary behavioral or clinical endpoints in Phase II Preliminary Testing, (2) prepare the Phase III Efficacy trial to test the benefit of the treatment on a clinically meaningful outcome, and (3) commit to achieving a sufficient, meaningful change . The ORBCustomer discovery and value proposition design, a form of stakeholder engagement based on marketing and LeanStartup business methods , was usen = 12) and academic specialty clinics (n = 8). Both primary and specialty pediatric clinics were selected to provide a diverse sample of clinical settings in order to develop a tool that may benefit a variety healthcare teams and reach diverse adolescent populations, including those of low SES. Through customer discovery, our research team identified (1) what healthcare teams' felt were their most important “jobs to be done” in helping adolescents with overweight/obesity adopt healthy behaviors, (2) critical “pains” and “gains” related to overweight/obesity treatment, and (3) how they would measure success. The goals of these interviews were to identify a need or clinical problem requiring a solution, articulate a strong value proposition, and identify outcome measures that are meaningful to healthcare teams to demonstrate success in their treatment of overweight/obesity among adolescent patients. A value proposition statement articulates how a product helps create desired gains and relieve pains for unsatisfied jobs to be done for a given customer segment. Product features that address such needs are more likely to be adopted  defines the scientific foundation and basic elements of the behavioral treatment and milestones for judging success, and Ib (Refine) refines the treatment to optimize efficiency and feasibility while promoting the intended clinical change. The ORBIT model advocates for the explicit identification of behavioral drivers that translate into intervention targets, an identification of methods by which these targets can be altered, and the translation of targets into quantifiable measures to evaluate the intervention's success . InterveUser testing evolved in the disciplines of human–computer interaction and system design, thus focuses on the extent to which users find the tool easy to use and effective for the tasks they need to perform . A thinkCustomer discovery and user-testing interviews were analyzed separately using the same techniques. Rapid qualitative analysis methods  were appCustomer discovery insights and supportive provider quotes are summarized in In summary, there was clinical need for a tool to help healthcare teams efficiently deliver tailored, evidence-based behavior change recommendations and follow-up with patients. The majority of healthcare team members were not currently using nor had previously used any HIT tool that met these needs. To promote meaningful change, the tool should address the patient's motivation to change their health behaviors and overcome barriers  to healthy behaviors. The generation and use of data are necessary to measure and communicate progress to the patient and the provider and inform care decisions. The initial value proposition was “our PREVENT tool delivers tailored, evidence-based physical activity and nutrition prescriptions that account for SDOH to help you improve your care beyond generic recommendations within the time constraints of a patient encounter.” Based on the customer discovery, the value proposition was modified to: “the PREVENT tool helps healthcare teams pragmatically address obesity among adolescents by displaying pertinent health data; delivering personalized, evidence-based lifestyle change recommendations; providing community resources; and automating virtual patient follow-up.”The PREVENT tool was designed for healthcare team members to use on a tablet or computer during clinical care visits with adolescent (12–19 years of age) patients who are overweight or obese. The tool addresses a clinical need by helping healthcare teams with their job to discuss health and prevention (Step 1); deliver tailored, evidence-based recommendations for physical activity and healthy food intake (Step 2); recommend community resources to support behavior change (Step 3); and provide detailed recommendations and education and maintain relationships with patients electronically (Step 4). Motivating patients was identified by healthcare team members as a critical treatment target to successfully promote physical activity and healthy eating within the short patient–provider encounter. Features of the PREVENT tool were designed following SDT Theory basic tenets to help healthcare team members motivate patients by improving their competence and autonomy through choice and self-initiation. All healthcare team members will be trained prior to using the PREVENT tool on the use of neutral language during patient–provider interactions  to further support autonomy , 62.In addition to facilitating intervention, the tool generates data to inform care and monitor patient behavioral and clinical progress to facilitate ongoing communication between the patient and provider. In addition to an initial screening, patients will report whether they are meeting their physical activity and healthy food intake goals monthly. Survey responses are displayed immediately within the PREVENT tool to enable healthcare team members to view patient progress and potentially adjust patient care . Based on responses, patients will receive automated email messages with updated progressive goals and motivation. We hypothesize that the PREVENT tool will improve patient motivation to make behavior change, improve their physical activity and food intake behaviors, and stabilize weight gain or generate weight loss among adolescents .Step 1. PREVENT uses a robust informatics approach to automatically populate and visually display American Heart Association (AHA)'s Life's Simple 7 Risk Factors  using patients' EHR and survey data  using evidence-based behavioral recommendations  include refinements made based on user-testing results. Changes to the risk portion of the tool included a more descriptive risk title , addition of the date of health data collection, description of moderate and vigorous activity levels, addition of a “former” smoker option to distinguish from never and current, and description of smoking status categories. We incorporated an option to manually add a physical activity not included in the menu of youth activities portion of the behavior change prescription tab and removed options not relevant to our geographic region . Community resources were color coded to distinguish better between types of resources . The tool also was programmed to be more compatible with touchscreen-enabled computers, and several buttons/functions  were redesigned to be more intuitive to the user.Images and features of the PREVENT tool described above [This first phase of product development identified a clinical problem and developed a solution (the PREVENT tool) that delivers tailored, theory-based behavior change prescriptions and links patients to resources that are desired and accessible to the patient at the point of care. The next phase will test whether the use of informatics and HIT makes this patient- and healthcare team-centered, data-driven approach to behavior change feasible within routine care. The PREVENT tool may be integrated into diverse clinical settings  to address obesity among children and adolescents while prompting data-informed patient–provider prevention discussions. It is designed to engage and motivate the patient by increasing their autonomy and competence to achieve recommendations for physical activity and healthy food intake . The PREThe primary purpose of this paper was to describe the PREVENT tool and how the ORBIT Model, principles of D4DIS, and behavioral theory were used in its development. While additional testing is necessary to determine the ultimate impact of the PREVENT tool, the use of an intentional, user-centered design process should increase the likelihood of the intended outcomes  and increase uptake, implementation success, and long-term sustainment and effectiveness . ThroughHealthcare team members were satisfied with the PREVENT tool, felt it was easy to use, and indicated that it would be useful for helping patients with overweight/obesity lead healthier lifestyles. The PREVENT tool is a work in progress that will continue to undergo iterative improvements as we progress through the ORBIT phases of development. We anticipate changes in the provider interface, data input and output modalities, and measurement items. Furthermore, we were unable to incorporate all suggestions from user testing  into tool updates at this phase due to time and budget constraints. We will continue to track end-user recommendations, as well as gain feedback from patients and families, and incorporate these in future phases of development and testing. Based on the ORBIT model, the PREVENT tool has met the essential milestones for moving to Phase II (preliminary testing): confidence that the treatment package is complete; inclusion of essential components offered in an efficient, feasible, and sustainable way; safe and acceptable to the user of interest ; and plausible that the treatment will have a clinically significant benefit . The delPotential limitations include that the PREVENT tool is limited to children and adolescents who are overweight or obese but may be of interest to those who are sedentary and/or who do not meet dietary recommendations, yet maintain a normal weight. The tool is limited to the English language, yet should be translated to effectively scale to broad populations. Although PREVENT uses informatics and has HIT capabilities, the tool is not yet integrated into the EHR, which may limit workflow compatibility. The PREVENT tool was developed with input from the end user , yet has not engaged other key stakeholders . Subsequent testing will elicit patient and families' feedback on the acceptability of the PREVENT tool. We did not include patient/family perspectives to this point in PREVENT development but intend to further refine the tool based on patient and families' desires, as the patient perspective is critical to the success of this intervention. Future work should perform customer discovery with additional stakeholder decision makers who will care about time required and financial and opportunity costs that will ultimately impact dissemination and sustainment. While the tool includes multiple data inputs to tailor recommendations, it does not integrate objective measures of physical activity  that are more reliable indicators of physical activity levels than patient-reported surveys . FurtherTo our knowledge, the PREVENT tool is one of the only HIT tools that harnesses clinical and community data to help healthcare teams deliver precision prevention  and reshape the way we care for children and adolescents who are overweight/obese and promote population health . Data-cekepperm@wustl.edu.The datasets presented in this article are not readily available because all original data are qualitative in nature and potentially identifiable. Requests to access the datasets should be directed to Maura M. Kepper, The studies involving human participants were reviewed and approved by Washington University in St. Louis Institutional Review Board. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.MK and CW-B wrote, edited the manuscript, collected, and analyzed data. RB, BK, EM, JG, LF, RG, ML, and RF contributed to the editing of the manuscript. MK, CW-B, ML, and RF contributed to the development of the tool. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Agency-based supervision is essential to skilled practice and staff retention, directly impacting the quality of services. The Supervisory Leaders in Aging (SLA) program was designed to strengthen supervision of the aging services workforce. SLA was implemented in four states and trained 134 social work supervisors. The purpose of this study was to develop a novel, comprehensive, and practice-informed measure of supervisory best-practices in gerontological social services. The primary outcome for the SLA program was the Practice Inventory for Supervision in Aging Services (PISAS), which evaluated the frequency of recent use of best practices as identified by the project team and vetted by instructors and an interdisciplinary advisory board. Reliability and confirmatory factor analyses were conducted to examine its psychometric properties. Findings demonstrated the scale had good internal consistency reliability (ɑ = 0.88). Confirmatory factor analyses indicated a three-factor solution, 1) Gerontological Social Work Skills, 2) Program Development Skills, and 3) Supervision and Leadership Skills, accounting for 72% of the variability in the 27 items. Gerontological Social Work Skills captured best practices regarding mental health in late life, heightening awareness of elder abuse, and working with families. Program Development Skills captured assessment, measuring outcomes, and translating evidence into practice. Supervision and Leadership Skills captured individual supervision, group supervision and leadership in interdisciplinary practice. Results indicated that PISAS is a reliable and valid measure of use of best practices in gerontological social work supervision. Further implications and limitations of this measure in assessing outcomes of gerontological education and training programs are discussed.


Feedbacks are an essential feature of resilient socio-economic systems, yet the feedbacks between biodiversity, ecosystem services and human wellbeing are not fully accounted for in global policy efforts that consider future scenarios for human activities and their consequences for nature. Failure to integrate feedbacks in our knowledge frameworks exacerbates uncertainty in future projections and potentially prevents us from realizing the full benefits of actions we can take to enhance sustainability. We identify six scientific research challenges that, if addressed, could allow future policy, conservation and monitoring efforts to quantitatively account for ecosystem and societal consequences of biodiversity change. Placing feedbacks prominently in our frameworks would lead to (i) coordinated observation of biodiversity change, ecosystem functions and human actions, (ii) joint experiment and observation programmes, (iii) more effective use of emerging technologies in biodiversity science and policy, and (iv) a more inclusive and integrated global community of biodiversity observers. To meet these challenges, we outline a five-point action plan for collaboration and connection among scientists and policymakers that emphasizes diversity, inclusion and open access. Efforts to protect biodiversity require the best possible scientific understanding of human activities, biodiversity trends, ecosystem functions and—critically—the feedbacks among them. BiodiBox 1.Biodiversity: variety of life. We use the concept to include people in the living earth system; biodiversity is measured at many scales and in many ways, from genetic diversity to functional diversity to behavioural or cultural diversity.Feedback: modification or control of a process by the results or effects of the same process.Ecosystems: joint biotic/abiotic systems of life, characterized by dynamic stocks and fluxes of energy, materials and information and their feedbacks.Biodiversity–ecosystem functioning (BEF) relationships: refers to the relationship between diversity per se and the magnitude and stability of an ecosystem functions. BEF refers to the role diversity plays in ecosystem functioning that is over and above the importance of total abundance, biomass or composition of the biological assemblage [semblage .Ecosystem functioning: the processes of energy flow (e.g. primary production), material cycling (e.g. carbon cycling) and information processing (e.g. evolution) carried out by living systems. Functions are understood to reflect interaction networks involving multiple genetic and functional elements of biodiversity, and include stocks and pools of biomass, elements and energy forms.Ecosystem services: the value of ecosystem functions to people [o people , and orio people , it is io people ,13.Nature's contributions to people (NCP): a pluralistic view for the value of ecosystems and ecosystem functioning to people [et al. [o people ,13. PeteThe science underpinning these major initiatives has clearly demonstrated direct effects of biodiversity on ecosystem functioning and human wellbeing (B-E-H) , as wellThe next generation of biodiversity scholarship will more effectively understand feedbacks as essential features of any focus on biodiversity and how it changes in relation to human activities and ecosystem functioning . This kn. 2Biodiversity and its relationships to ecosystem functioning and human wellbeing depend on feedbacks within and between these system components figures  and 2 2–25. The Positive feedbacks are self-reinforcing, and can drive rapid change and even destabilize systems [a). Negative feedbacks (b) are self-dampening and stabilizing, and can buffer systems against change [behaviour over time of a system to the state of that same system in some way. It is this self-dependent relationship that distinguishes models with dynamic feedbacks from models that include direct and indirect effects but do not relate these in feedbacks . Negative (dampening) density-dependent feedbacks of predation, disease and pathogens on species' performance cause diverse systems to maintain diversity and ecosystem functions over time more than less diverse systems [Feedbacks explain change and stability in systems involving biodiversity, ecosystem functioning and human wellbeing. Among the processes that maintain biodiversity, feedbacks determine stability and future trajectories of population, community and ecosystem dynamics ,35,36, feedbacks . One of services –42 . Pollinator functional diversity can increase pollination and plant seed production [figure 1e). Further, humans benefit when the plants are of cultural or agricultural value. Some agricultural practices, land use change and pollution have dramatically reduced pollinator abundance and diversity [figure 1e).There are many examples of change that we now understand to depend on feedbacks between biodiversity, ecosystem processes and human activities  and the targets in the post-2020 Global Biodiversity Framework of the CBD . The IPBa) in the summary of models used to generate projections and scenarios for the most recent IPBES report. The few examples are in the integrated assessment models involving social and economic systems coupled with natural systems [outcome of integrated system models, rather than an architectural feature [figure 1e).The IPBES platform also channels and motivates scholarship and scientific research to fill gaps and improve methods for modelling scenarios. It relies on synthesis of scientific evidence for the causes and consequences of biodiversity change, combined with scientific models to project future scenarios . There ifigure 1 systems . The ass feature . The IPB feature ), yet a . 4 (1)We cannot robustly relate current or recent temporal trends in biodiversity to likely future trajectories of biodiversity change in most cases. As we have noted above, future biodiversity, and diversity's contribution to ecosystem services, may not be accurately projected by extrapolating a historical trend in biodiversity forward in time because of feedbacks among biodiversity, ecosystem function and human activities [tivities ,36,62,64 (2)We do not understand the B-E-H system well enough to relate observed recent trends in biodiversity to likely future trends in ecosystem function and human wellbeing. Dynamics of one part of the system  depend on other parts of the system , and vice versa. Because feedbacks characterize how biodiversity, ecosystem functioning and human activities change together over time, projected future trajectories or scenarios of diversity, ecosystem functioning or human wellbeing require consideration of all three components. One pervasive consequence of this knowledge gap is the persistent decoupling of biodiversity and functioning in assessment and monitoring programmes; most of the biodiversity observations being assembled for biodiversity change assessments  do not systematically include accompanying measures of ecosystem processes or human activities. Though GEO BON is moving in this direction with essential ecosystem variables, such an advance must be made in the context of statistical approaches that can allow detection and attribution of joint changes in biodiversity, ecosystem functioning and human wellbeing. (3)Trends in B-E-H components depend on scale, yet we still do not understand exactly how, and what feedbacks play in determining scale dependence. Trends observed at one scale do not necessarily predict trends at higher or lower spatial resolutions [olutions , and thiolutions . We requolutions  (figure  (4)Experimental tests for direct BEF effects have omitted feedbacks. The majority of experimental tests of the relationship between BEF conducted in the last two decades has employed an experimental design that intentionally disrupts potential feedbacks—for example, by weeding out species that colonize [a), in doing so these procedures prevent feedbacks (e.g. c) from playing out over time. Consequently, hundreds of experiments frequently reviewed and synthesized as strong evidence for direct effects of diversity on ecosystem functioning [a) cannot be used to demonstrate consequences of the feedbacks between diversity and functioning because each system studied was controlled to prevent them from occurring.colonize  or by recolonize  over thecks e.g. c from plctioning ,8 (figur (5)Human–biodiversity feedbacks are still not well understood, allowing to persist a perception within the western science framing that people affect biodiversity but that there is little feedback from biodiversity to people [o people ,31,61,68o people . The chao people ,69. Meet (6)Develop an operational understanding of how different dimensions of biodiversity are involved in feedbacks over time. Estimates of biodiversity change are based on observations of some dimension of biodiversity as defined in conventional scientific concepts: alleles, genes, traits, species  and models of phylogenies. Not only do we still require great investment in organized biodiversity sampling and monitoring [nitoring ,70, we lnitoring . We requ (7)Develop theory and workflows that explicitly relate information from emerging technologies to knowledge that can be used to deepen our understanding of feedbacks. Technological tools for observing biodiversity allow high throughput and remote sensing of biodiversity at the finest levels of biological organization  as well as some measures of ecosystem functions [unctions –74. As vOur survey revealed seven knowledge gaps in biodiversity science when we considered the B-E-H system as a whole system, rather than take previously prevalent perspectives that emphasize two of the three components—BEF that tends to consider human activities as outside the system, or socioecological systems (SES) in which biodiversity and functioning are lumped into one component. Filling these knowledge gaps requires targeting feedbacks as scientific research subjects, and considering how assessments and policies can best reflect this knowledge development. We suggest that these challenges might be used to prioritize major investment to expand the BEF paradigm and enhance our knowledge frameworks to support biodiversity policies and to realize sustainability goals (see §5).. 5in silico. Outside specialist research communities, B-E-H feedbacks and their consequences are not well represented in conceptual diagrams and models used by policy experts and decision makers to understand biodiversity change and its likely consequences over time. Greater emphasis on this representation can help minimize overlooking this important concept when identifying priorities for biodiversity observation or multifaceted conservation opportunities. Further, many knowledge systems beyond science—such as traditional knowledge systems—include knowledge of feedbacks [ (1)Collaborate and connect. We must convene and support collaborations and knowledge development that reflects the ways people know and interact with biodiversity. The action required is to come together to identify knowledge development priorities at local, regional and global scales that reflect the depth and diversity of how humans and biodiversity are co-dependent. We must take the time to listen and learn from each other, and build from these conversations to the observation and solutions programmes we call for. Doing so will result in an inclusion of a broader range of knowledge systems and perceptions of human–biodiversity interactions [ractions , benefitractions . We can ractions . (2)Build and sustain multi-scale models to develop and revise scenarios of biosphere change. Though models exist to produce biodiversity scenarios for the future [e future ,78, we me future ,79. Thes (3)Build and sustain national and global observatories for temporal change in biodiversity, ecosystem functioning and human activities. Integrated observations must be made at different spatial scales with worldwide coverage [ad hoc approaches to sampling of biodiversity that has produced a set of observations that is highly biased to developed countries and terrestrial habitats [et al. [coverage , going bhabitats ,40,80. Thabitats ,70. Furthabitats ,81. New  [et al. , must em [et al. ,81,82. (4)Experimentally and iteratively test the models and re-evaluate our understanding. To understand feedbacks, observational programmes (Action 3) should be guided by theory that includes feedbacks, and coupled with experimental programmes to understand feedbacks. As with observatories, the experimental and modelling programmes must be run by collaborations of scientists, modellers and end users from a broad range of biomes, countries and cultural backgrounds, specifically including indigenous and local peoples from the global north and south. This action item is to increase investment in experimental programmes that help to fill specific gaps in our understanding of biodiversity change, and to prioritize those programmes led by multi-sector and multi-disciplinary research and data user teams. (5)Identify and support a sustained organizational structure. A leadership team must assemble, must be able to draw on existing scientific knowledge and work with the research community to develop research programmes. The leadership team must facilitate diversity and comprehensive inclusion of nature and people in the research programmes and associated policy development programmes, can promote the research agenda to potential users and supporters, can lead public engagement activities, and can ensure fully open science practices and data archiving so the findings are available to everyone in the world. The structure of the leadership team should be consistent with current values, and consider collaborative networks and other social structures in its design.We have outlined seven gaps in B-E-H scientific knowledge that limit our current capacity to assess changes to the biosphere. Resolving these knowledge gaps will require investment in scientific research programmes worldwide to employ diverse, interdisciplinary and even transdisciplinary approaches in the field, in the lab and eedbacks ,69,75, aAlong the way, the research community will need to confront additional logistical challenges that currently limit rapid scientific advances. These have received attention elsewhere, and resolving these challenges is critical the success of the agenda we have outlined here. These include (i) the current lack of open science and the fact that data for BEF from many places is not curated or made available in a central database , (ii) li. 6Feedbacks between human wellbeing and BEF have been appreciated and understood for millennia. Yet, only in recent decades has scientific progress led to recognition of the importance of feedbacks among biodiversity, functioning and people across scales. Despite this recognition, and major progress with models, experiments and observations, major challenges remain to integrate this knowledge with new capabilities to meet the policy challenges of the coming decades. As major policy-guiding scientific assessments grow in importance, it is essential to keep striving for the scientific advances, and in particular theoretical advances, that will foster integration of state-of-the-art scientific understanding with international and local policy objectives. There is no substitute for knowledge of feedbacks. The effects of feedbacks over time cannot be approximated by static representations of direct effects . Many auClick here for additional data file.