For practising nurses, trust , cultural values , and working experience  were associated with their health literacy knowledge. Thus, responses targeted at gaps in health literacy knowledge of student and practising nurses must be sensitive to personal characteristics , social values , as well as factors relating to nursing education and experience.This study examined student and practising nurses’ health literacy knowledge, and its correlates in Ghana. It was underpinned by an adapted version of the Institute of Medicine’s (IOM) conceptual framework of health literacy. We used convenience and snowball sampling techniques to collect data from 876 nurses (477 student nurses and 399 practising nurses) in a cross-sectional survey from February 2019 to June 2019. The respondents were drawn from all the former ten administrative regions of Ghana. Approximately 75.4% of the respondents had heard of health literacy. However, health literacy knowledge was generally low (average score of 6.6 out of 20) among both groups, with student nurses (average score of 5.8 out of 20) having significantly lower scores than practising nurses (average score of 7.4 out of 20). Factors associated with health literacy knowledge among student nurses included gender (male, B = −0.499,  Health systems globally require the public and patients to participate in health service delivery ,2. AccorHealth literacy has become an essential asset in modern health promotion strategies ,11. It iPeople with sufficient health literacy demonstrate sound judgement about matters that affect their health, and they tend to avoid deleterious health behaviours, such as substance abuse and the drawbacks associated with leading sedentary lifestyles ,13. MoreUnfortunately, extant evidence suggests that many nurses have low health literacy, while others are unaware of the concept and its application and relevance to practice ,17. ManyFor student nurses, health literacy knowledge has been associated with the nursing education curricula ,23,24. AThis study aims to examine the health literacy knowledge of student and practising nurses in Ghana and identify the factors associated with that knowledge. These factors are identified through an adapted version of the Institute of Medicine’s (IOM) conceptual framework of health literacy  see Fig. By compIn both low-and-high-income countries, limited health literacy knowledge among health professionals is often associated with a multiplicity of factors relating to the health education systems, working conditions and equipment, personal characteristics  and the effects of prevailing socio-cultural norms and practices. These factors are aptly presented by IOM’s conceptual framework for health literacy. The framework identifies three domains that are responsible for health literacy and its subsequent influence on health outcomes: firstly, culture and society ; secondly, the health system itself; and thirdly, the education system . Specifipersonal and socio-cultural characteristics; health and health system; and education and professional practice . The domain relating to culture, society, and personal characteristics encapsulates factors associated with cultural and societal precepts, such as reliance on cultural principles, religiosity, trust, and group involvement. This domain includes personal characteristics such as age, gender, income, marital status, and residence location . We combine cultural and demographic factors because in many societies in developing countries, personal characteristics and their roles in societies are shaped by the prevailing cultural practices and norms [We adapted these domains to make them relevant to the aims of this study. The revised domains include practice . The domnd norms .The health and health system domain encompasses policies and structures designed to promote health, prevent diseases and offer curative services . The chaFinally, the education and professional practice domain emphasise the fact that academic training and professional experience are likely to affect health literacy knowledge of nurses . To consData for this study were derived from a cross-sectional survey that was conducted in healthcare facilities and nursing training institutions across Ghana. The survey took place from February 2019 to June 2019 in all the ten administrative regions of Ghana as part of a broader social epidemiological study. The Council for Scientific and Industrial Research (CSIR) of Ghana approved the study protocol (RPN 005/CSIR-IRB/2018).XM platform ) or paper-based questionnaire. Either way, trained interviewers were on hand to support respondents who had challenges in completing the survey. The goal was to ensure a representative sample by recruiting at least 384 respondents each for the student and the practising nurses [Ns = (Np)(p)(1 − p)/(Np − 1)(B / C)2 +(p)(1 − p), where Ns = total sample size needed; Np = size of population; p = proportion expected to answer a certain way; B = acceptable level of sampling error; C = Z statistic associate with confidence interval. These criteria were considered: 95% confidence level; sample error of 0.05; and assumption that 50% of respondents would adequately respond to questions [We used convenience and snowball sampling techniques to select respondents in nursing training schools  and practising nurses from all kinds of health facilities. Student nurses had to be in their second year of study, while practising nurses were included in the survey if they had at least two years’ working experience. This was to ensure that both groups had had adequate opportunity to study or obtain some health literacy experience. Many practising nurses in Ghana often return to school for advanced education either privately or with support from the Ministry of Health (for those in the public sector) within the first five years of practice . Therefog nurses . This miuestions .Health literacy knowledge: We measured this using an adapted version of Part I of the Health Literacy Knowledge and Experience Survey Instrument (HLKES) [ (HLKES) ,24. The  (HLKES) . The ins (HLKES) . To makeItems that were removed included those relating to specific health literacy screening and measurement tools/techniques, as well as those that were more relevant to Western contexts where the instrument was developed. For example, the following questions were removed: Which of the following statements best describes the Fry Method? Which statement best describes the Test of Functional Health Literacy (TOFHL)? “The Rapid Estimate of Adult Literacy in Medicine (REALM) is an instrument used to…” and “Low health literacy levels are more common among…?” These questions were less likely to be answered correctly or understood by respondents given the novelty of health literacy in public health and nursing education in African settings, and Ghana in particular . This mePersonal and socio-cultural characteristics: We measured several variables under this domain including age (in years), gender , marital status , area of residence , religiosity , cultural values , trust in others (yes/no), membership of an association (yes/no); monthly income/allowance; and self-perceived socioeconomic status  to 10 (high)).Education and practice characteristics: We measured these variables: nursing speciality; years of experience (for practising nurses); year of study (for students); and highest educational attainment as detailed in Health and health system characteristics. This domain measured respondents’ satisfaction with the health system ; self-rated health status ; engaging in physical activities ; use of curative healthcare .Covariates. We controlled for the following variables: whether the respondent had heard of health literacy (yes/no); the region of data collection; type of nurse (student/practising nurse).p = 0.001) and practising  nurses were not normally distributed. The final set of analyses involved an ordinal logistic regression. We used the total score of health literacy knowledge as the dependent variable. We included all variables showing a significant relationship with health literacy knowledge in the correlation analyses as independent variables. The analysis was conducted with SPSS version 26 . Significant associations were evaluated at p < 0.05.At the outset, our analyses comprised of descriptive statistics to provide an overview of the respondents’ characteristics and status as regards the key variables, using frequencies and percentages for categorical variables and mean  for continuous and ordinal variables. We subsequently carried out a Spearman’s correlation analyses to identify initial variables associated with health literacy knowledge as the data for both student . However, the analyses presented in this study were based on 250 paper-based and 626 online responses after removing significantly incomplete cases. p < 0.01), frequency of curative care use , and having trust in others  were negatively associated with health literacy knowledge. However, cultural values  and year of study  showed positive associations with health literacy knowledge. For practising nurses, factors associated with their health literacy knowledge included trust in others , cultural values , and working experience . In the overall sample, those married , and living together with someone as married  were less likely to have health literacy knowledge than respondents who identified themselves as single. Income/stipend  and cultural values  were positively associated with health literacy knowledge. However, SES  and trust in others  were negatively associated with health literacy knowledge. These results are presented in Student nurses were less likely to have sufficient knowledge of health literacy compared to practising nurses. Among the student nurses, sex , there were also similarities in the factors associated with the health literacy knowledge of both groups .While most of the nurses had heard of health literacy, we found that health literacy knowledge was low. This disparity shows that health literacy knowledge goes beyond mere familiarity with the concept. Health professionals must be deliberately trained to become knowledgeable in health literacy for personal and professional use. This finding is consistent with previous research ,9,17, anIn many instances, low health literacy knowledge among nursing students is associated with low exposure to the concept (through practice), and lack of relevant training and practical experience ,9. Some However, the above suggestion raises questions about why practising nurses in this study also had low health literacy knowledge. The situation compels a more in-depth examination. As argued in the existing literature, there are critical gaps in the nursing education curriculum in Ghana and many other places pertaining to health literacy, which explains the low awareness of the concept ,18,21. GAmong the student nurses, being male and use of curative health services were negatively associated with health literacy knowledge. The finding on gender is inconsistent with that of a population-based study in Ghana, which found that males have greater health literacy and related knowledge than females . That stThe inverse relationship between utilising curative services and health literacy knowledge among the student nurses supports the widely held position that having sufficient health literacy (and knowledge of it) promotes positive health outcomes ,12. WithAside from gender and the use of curative services, which were solely associated with the health literacy knowledge of nursing students, all the other significant correlates of health literacy knowledge were common to both student and practising nurses. Principally, the common correlates included trust in others, cultural values, and year of study/years of experience. These similarities give credence to the tenets of the conceptual framework of health literacy. This is because the factors identified correspond to two domains of the framework namely, culture, society and personal characteristics , and, secondly, education and professional practice (education and work/work experience) . TrustinAnother finding from this study corroborates the above arguments; nurses who were married or living together with their partners had lower health literacy knowledge than those who were single. People in intimate or close social relationships are likely to trust their partners for health information, making them vulnerable to miseducation if the people in their networks or themselves have low health literacy knowledge . BesidesWe also found a positive association between cultural and traditional values and health literacy knowledge among both student and practising nurses. According to the IOM health literacy framework, cultural precepts and practices are fundamental to health literacy and knowledge, as people often interpret health and healing matters from cultural perspectives . Hence, Moreover, the experience of nurses, in terms of years of nursing education and nursing practice, was associated with health literacy knowledge. This implies that deliberate efforts to train and expose nurses to health literacy through health education and practice can improve their health literacy knowledge. Existing studies show that nursing students’ overall health-related knowledge increases as they move along their study programmes . A similWe also found that income/stipend was positively associated with health literacy knowledge, but SES was negatively associated with health literacy knowledge. These findings partially explain the conceptual complexity of SES . On the The study did not deal with the state of specific domains of health literacy knowledge, such as basic facts of health literacy, and health literacy screening. Several other studies have addressed such research gap, and they appear to have consistent findings ,9,18. InWe found evidence that factors relating to all three aspects of the adapted health literacy framework were associated with the nurses’ health literacy knowledge. Being male and using curative health services was uniquely associated with student nurses’ health literacy knowledge. However, trusting relations with others, cultural values and education/work experience were commonly associated with student and practising nurses. These findings demonstrate the consistency, but also the variations in the underlying factors of the health literacy knowledge of student and practising nurses. The findings offer significant insights into potential areas of emphasis as part of measures aiming to incorporate health literacy in nursing education and address gaps in health literacy knowledge of practising nurses. For instance, efforts to create opportunities for student and practising nurses to acquire health literacy knowledge in their studies and professional practice respectively can incorporate personal characteristics , social values , and their nursing education or experience. In Ghana and other places in sub-Saharan Africa, this knowledge base is critical, given that health literacy knowledge was found to be very low among both nursing groups in this study. Future research can extend this study by examining how the various correlates of health literacy knowledge among student and practising nurses evolve in clinical and nursing education settings.


Alzheimer’s disease (AD) is the most common form of dementia worldwide. It is mostly known for its devastating effect on memory and learning but behavioral alterations commonly known as neuropsychiatric disturbances (NPDs) are also characteristics of the disease. These include apathy, depression-like behavior, and sleep disturbances, and they all contribute to an increased caregiver burden and earlier institutionalization. The interaction between NPDs and AD pathology is not well understood, but the consensus is that they contribute to disease progression and faster decline. Consequently, recognizing and treating NPDs might improve AD pathology and increase the quality of life for both patients and caregivers. In this review article, we examine previous and current literature on apathy, depressive symptoms, and sleep disturbances in AD patients and preclinical AD mechanistic models. We hypothesize that tau accumulation, beta-amyloid (Aβ) aggregation, neuroinflammation, mitochondrial damage, and loss of the locus coeruleus (LC)-norepinephrine (NE) system all collectively impact the development of NPDs and contribute synergistically to AD pathology. Targeting more than one of these processes might provide the most optimal strategy for treating NPDs and AD. The development of such clinical approaches would be preceded by preclinical studies, for which robust and reliable mechanistic models of NPD-like behavior are needed. Thus, developing effective preclinical research models represents an important step towards a better understanding of NPDs in AD. Alzheimer’s disease (AD) remains a major cause of morbidity and mortality worldwide and is substantially burdensome to affected persons and their caregivers. It has been estimated that 47 million people worldwide were living with AD in 2015  plaques, neurofibrillary tangles (NFT), and neuroinflammation, and as the disease progresses almost all brain regions become affected with cell death and neuronal degeneration in the terminal stages eventually leading to substantial brain shrinkages and death. In addition to cognitive impairment, 80–97% of AD patients experience at least one non-mnemonic symptom at least once during the course of the disease  based biomarker assessment, magnetic resonance imaging (MRI), and positron emission tomography (PET) imaging. A CSF biomarker profile for AD includes low levels of Aβ42, reflecting Aβ plaque deposits, and high levels of total tau (T-tau) and hyperphosphorylated tau (P-tau), reflecting increased axonal damage and tau pathology, but other biomarkers have also been investigated to aid in early diagnosis, prognosis, and progression of AD , and other catecholamines in the brain, ameliorates apathy in both mild  that has free radical scavenging properties, enhances mitochondrial membrane potential, and increase ATP production  possibly caused by changes in mitochondrial membrane integrity  is implicated in controlling wakefulness and arousal by the release of NE with high levels of neural activity during wakefulness and low activity during sleep  of the hypothalamus  mice (Zhou et al., −/− mice also had compromised mitochondria together with Aβ and tau pathology in the SCN (Zhou et al., SIRT1 is an NADAltogether, these preclinical studies are consistent with the clinical data and provide evidence for a causal link between sleep disturbances, apathy, depressive behavior, and AD pathology. However, to substantiate these studies and our knowledge of NPD pathology we need to develop more robust preclinical models.One notion put forward is that that NPDs appear because AD patients become unable to communicate or attend to their own needs and as a result feel misunderstood and become apathetic or depressive (Cohen-Mansfield et al., All authors contributed equally to the conceptualization of the manuscript. AC wrote the manuscript. AC, OW, and AA reviewed the manuscript.AA is employed by the company H. Lundbeck A/S. AC is employed at both the company H. Lundbeck A/S and the University of Aalborg. OW declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Much is known about the demands of caregiving for persons with dementia (PWD) and its effects on family caregivers, however sex and gender aspects have received less attention. We synthesized the evidence on sex and gender distinctions in: (1) the caregiving burden and (2) the impact of caregiving on the physical and mental health of family caregivers of PWD.Systematic review.Medline, Embase, PsycINFO and Cumulative Index to Nursing and Allied Health Literature between January 2007 and October 2019 were searched.Included studies met the following criteria: (1) examine experiences and/or impacts of caregiving among family caregivers of individuals with any form of dementia; (2) report sex and/or gender distribution of study population and/or report results stratified by sex and/or gender, and (3) include both male and female family caregivers.Two independent reviewers extracted the data and assessed risk of bias using the Critical Appraisal Skills Program checklist and National Institutes of Health Quality Assessment Tool for Observational Cohort and Cross-sectional Studies. Data were synthesized using a narrative synthesis approach.A total of 22 studies were included. Caregiving burden was measured using various methods. A majority of studies reported higher burden among females. All studies that did not report a sex and gender difference in caregiving burden accounted for confounders. Findings on sex and gender differences on physical and mental health conditions were inconsistent with most studies failing to account for confounders in their analyses.Current evidence on sex and gender differences in caregiving burden, mental and physical health is limited. Findings suggest presence of sex and gender differences in caregiving burden. Given the variety of mental and physical health constructs that were examined, further research is required to substantiate the evidence.PROPSERO Registration Number: CRD 42018070032. Dementia, which refers to a number of conditions that produce acquired cognitive decline , is a maWith the rising prevalence of dementia, an increasing number of aging family members are providing care for PWD , 7. In 2With the progressive decline experienced by PWD, family caregivers who face difficulties adapting or modifying their care strategies experience a significant level of caregiver burden . PreviouCaregiving burden can have devastating and long term effects on the physical, social emotional as well as financial status of family caregivers of PWD , 18. PreWhile much is known about caregiving burden and its effects on family caregivers and their care recipients, there has been little exploration of possible sex and gender differences between male and female family caregivers of PWD. At present, females are the predominant providers of informal care for family members with chronic medical conditions including dementia . Within Previous analyses of sex and gender differences among caregivers have shown a considerable distinction with respect to physical and psychosocial health status . SpecifiTo address this research gap, the objectives of this systematic review were to: (1) examine any sex and gender distinctions in the nature and level of caregiving burden experienced by family caregivers of persons with dementia, and (2) determine the sex and gender differences in the impact of caregiving on specific physical and mental health constructs among family caregivers of PWD.The systematic review was conducted based on a previously peer-reviewed protocol registered with the International Prospective Register of Systematic Reviews (PROSPERO) (registration number CRD 42018070032) and published in an open access journal . The preMEDLINE ;Cumulative Index to Nursing and Allied Health Literature (CINAHL);Embase;PsycINFO.Due to the extensive number of studies identified within the searched databases and the limited empirical evidence regarding the impact of search and including earlier works on systematic review findings , our seaPlease refer to the published protocol for details on the data searches and MeSH terms used for each database .Given the importance of disaggregating the data by sex and gender when conducting a sex and gender analysis , studiesTwo researchers (CX and MB) independently screened study titles and/or abstracts as well as reviewed full texts of manuscripts to determine fulfillment of the inclusion criteria. Discrepancies in opinion were resolved through discussion with a third researcher (AC). A standardized form was used to assess study quality and synthesize study findings from the included studies. Extracted information included the following: (1) author and publication year, (2) study setting and design, (3) study location, (4) information of the study population and demographic characteristics, (5) study results relating to caregiving experiences , (6) details on the methodology used to gather these experiences, (7) the statistical approach used and confounders, (9) information on sex and gender differences and (10) information on the risk of bias assessment. Two reviewers (CX and MB) extracted the data independently and a third reviewer (AC) reviewed the quality of data extraction and mediated a resolution in cases of disagreement through follow-up discussions with the reviewers.Quality assessment of the studies was conducted independently by two reviewers (CX and MB). Qualitative studies were assessed using the Critical Appraisal Skills Program (CASP) qualitative checklist and consisted of the following steps: (i) assessment of potential sources of bias through a series of 10 questions related to the results, their validity and impact, and (ii) responding to each question as “Yes”, “Cannot Tell” and “No” . QuantitThe included studies were analyzed using a narrative synthesis approach following the Guidance for Narrative Synthesis in Systematic Reviews . SpecifiPatients and the public were not involved in this review.The searches yielded a total of 13098 records, from which 7195 records remained after the duplicates were removed. Of the 7195 records, 196 met the criteria for a full-text screen. As part of the full-text screen, articles were excluded if they did not stratify findings by sex and/or gender, did not conduct a sex and gender analysis, did not involve family caregivers of dementia or did not examine caregiving experience. Of the remaining 42 studies that were included for the quality assessment, 20 of the studies were of ‘low’ quality and were excluded. A total of 22 studies, all of ‘moderate’ quality except for one which was classified as ‘high’ quality, were included  38–59].–59.38–59A summary of the 18 included quantitative studies is presented in The 18 quantitative studies reported data from a total of 5735 (range 32–1223) caregivers. All of the studies reported information on sex/gender, and most studies reported the age of participants with the exception of two studies, which did not provide any information on the participants’ ages. The mean age in studies ranged from 53.9  to 77.9 A range of methods were used to measure caregiving burden among family caregivers of PWD. Thirteen of the 18 included articles examined caregiving burden, with six using the Zarit Burden Scale (ZBI) , 47–50 aOverall, ten of the 13 studies directly assessing caregiver burden found higher reported burden or care-related distress among female caregivers , 51, 52.Eight of the included studies examined the impact of caregiving on family caregivers’ mental health. Specifically, five studies investigated depression and all found significantly higher levels of depression in female caregivers compared to their male counterparts as measured by the Center for Epidemiologic Studies Depression Scale (CES-D) , 57, BecWith respect to family caregivers’ physical health, one study examined the impact of caregiving on sexual intimacy among spousal caregivers and found no significant difference in the impact of caregiving on sexual intimacy between male and female caregivers . After aA summary of the four included qualitative studies is presented in The four qualitative studies identified sex and gender differences related to caregiving burden, roles, help-seeking patterns as well as perceptions of intimacy among spousal , 55, 59 Three studies identified higher levels of stress and greater challenges faced by both female spousal and family caregivers , 54, 59.This paper systematically reviewed the literature on dementia caregiving between 2007 and 2019 to examine the (1) sex and gender distinctions in caregiving burden experienced by family caregivers of persons with dementia, and (2) sex and gender differences in the impact of caregiving on the physical and mental health of family caregivers of PWD. Among the 13098 articles retrieved in the initial search, only 22 studies were included in the review, which represents a small proportion of the literature in the field. Despite sex and gender being widely collected and reported in studies, few explored the presence and extent of sex and gender differences in caregiving burden. Given that caregivers are mostly female , these sAmong the 22 studies included in the review, 16 studies examined the sex and gender differences in caregiving burden. This included both formal measures of caregiving burden and semi-structured interviews. Among the 13 studies that used a formal measure to examine the sex and gender differences in caregiving burden, six used the ZBI. Developed more than 30 years ago , the ZBIAdditionally, two studies used the CBI, which like the ZBI, is a scale developed in the late-1980s , 69. TheWith respect to caregiving burden, findings from the qualitative studies concur with most quantitative studies. Not only did these qualitative studies identify a higher level of stress and challenges faced among female caregivers , 54, 59,Overall, while most of the included studies on caregiving burden demonstrated a higher level of burden among female family caregivers of PWD, these studies often lacked methodological rigor, reflecting the infancy of sex- and gender-based analyses in this area. Specifically, there was a lack of inclusion of other factors that have been shown to influence caregiving burden such as age, time spent on caregiving and dementia severity in the statistical analyses. Given that all but one study are of ‘moderate’ quality, attention should be paid on employing more comprehensive statistical and qualitative methodologies to better tease apart the relationships between sex, gender, as well as other socio-demographic variables and their collective influence on caregiving burden.Among the five studies that examined the prevalence of depression among family caregivers of PWD, female caregivers reported higher scores on instruments such as the CES-D, BDI and GDS compared to their male counterparts. While these findings are in line with the prevalence of depression in the general population where women are almost twice as likely to be diagnosed with depression than men , they maThe studies in this review also investigated other mental health constructs including psychological distress , 56, selStudies on sex and gender differences in caregivers’ physical health focused primarily on comorbidities, sleep, inflammation, and intimacy experiences. Significant sex and gender differences were uncovered in most studies with the exception of intimacy experiences, where contrasting findings were found. A single study highlighted the sex and gender difference in comorbidity among family caregivers of PWD. Utilizing the CCI, female caregivers were found to have less comorbidity compared to males. However, there was a lack of any adjustment for confounders such as age, which had been previously identified as an important adjustment when using the CCI , 76. As With respect to the impact of caregiving on intimacy experiences among spousal caregivers and their care recipients, studies reported contrasting findings. While the quantitative study that examined intimacy using the Experiences of Intimacy with Partner Scale (EOIPS) found no difference between male and female caregivers , interviOne study examined sleep and found significant sex differences in sleep and its related biomarkers including IL-6 and D-dimer. In contrast with other dimensions examined, male caregivers had worse sleep compared to females after accounting for caregiver age, BMI and care recipient’s dementia severity. Sleep plays important functions with respect to physical and psychological restoration, memory and emotional regulation . Poor quOne of the main concerns regarding the included studies was the lack of consistency on the measures used. As such, the estimates provided by each quantitative study were unable to be pooled together and caution is recommended when making inferences. In addition, while socio-demographic variables of the caregivers were extracted, the types of variables are limited to the data collection process of the included studies. As such, certain socio-demographic variables that are known to influence caregiving experiences, such as family income, education level and geographic location may have been omitted by the studies. Moreover, caregiving relationships, which are intricately linked with sex and gender, were not explored in relation to caregiving experiences due to the lack of explicit classifications of the family caregivers in the included studies. Given the limited number of articles retrieved through the systematic search which suggests the overall lack of scholarly focus in this topic area, alternative approaches such as a scoping review may be able to expand the breath of the review to include other caregiving populations.Given that most of the included studies did not explicitly disentangle sex from gender differences, the review was limited in its ability to report sex and gender findings separately. As such, given the current trend towards disentangling the impacts of sex and gender in understanding male and female differences , there iTo the best of our knowledge, we have conducted a first-of-its-kind systematic review to investigate the sex and gender differences in caregiving burden and its impact on the physical and mental health of family caregivers of PWD. Findings of the included studies suggest the presence of sex and gender differences in caregiving burden, with female caregivers experiencing greater burden compared to their male counterparts. However, given the variety of mental and physical health constructs that were examined by single studies, further research is required to substantiate the evidence. More importantly, the development of a core set of burden scales to be used in studies exploring caregiving burden will enable better comparisons across studies and allow for a more nuanced understanding of the caregiving experience. On a similar note, future work should also take into consideration other socio-demographic and clinical factors such as age, family income, education level, caregiving relationship and dementia severity that may interact with the sex and gender influences in caregiving experiences in order to tease out the nuances in such an intriguing topic area. Specifically, quantitative studies could employ multivariable analyses and qualitative studies could engage in active recruitment of caregivers from a variety of socio-demographic backgrounds. Overall, the current review highlighted a critical gap in the current literature on sex and gender differences in caregiving burden, mental and physical health. While females remain the predominant caregivers at present, there has been a surge in males taking on the role of caregiving in recent years , 61. As S1 Protocol(PDF)Click here for additional data file.S1 Checklist(DOC)Click here for additional data file.


There has been substantial growth in eHealth over the past decade, driven by expectations of improved healthcare system performance. Despite substantial eHealth investment, little is known about the monitoring and evaluation strategies for gauging progress in eHealth availability and use. This scoping review aims to map the existing literature and depict the predominant approaches and methodological recommendations to national and regional monitoring and evaluation of eHealth availability and use, to advance national strategies for monitoring and evaluating eHealth.Peer-reviewed and grey literature on monitoring and evaluation of eHealth availability and use published between January 1, 2009, and March 11, 2019, were eligible for inclusion. A total of 2354 publications were identified and 36 publications were included after full-text review. Data on publication type , country, level , publication year, method , and domain  were charted.The majority of publications monitored availability alone or applied a combination of availability and use measures. Surveys were the most common data collection method (used in 86% of the publications). Organization for Economic Co-operation and Development (OECD), European Commission, Canada Health Infoway, and World Health Organization (WHO) have developed comprehensive eHealth monitoring and evaluation methodology recommendations.Establishing continuous national eHealth monitoring and evaluation, based on international approaches and recommendations, could improve the ability for cross-country benchmarking and learning. This scoping review provides an overview of the predominant approaches to and recommendations for national and regional monitoring and evaluation of eHealth. It thereby provides a starting point for developing national eHealth monitoring strategies. With this scoping review, we aim to provide an overview of the current literature produced by researchers, organizations, or government bodies, and to assess the foci, methodology, and scope of monitoring and evaluating eHealth. The focus of this scoping study lies not in addressing the quality of the studies and obtaining ‘best evidence’,This scoping review is based on the approach suggested by the Joanna Briggs Institute, adapted and developed from the five stages by Arksey and O’MalleyeHealth is “the application of information and communication technologies across the whole range of functions that affect the health sector and including products, systems, and services that go beyond simply Internet-based applications.”The search strategy aimed to ensure the identification of both peer-reviewed publications providing quantitative and/or qualitative evidence on monitoring or evaluating eHealth at a national or regional level; and other publications, peer-reviewed or not, opinions or reports. The search targeted a number of potential sources including journal citation databases, bibliographic databases, and output from known centers of excellence and governments. The protocol for this scoping review contains further information on the preliminary search strategies.To identify original peer-reviewed publications, we used the databases PubMed, SCOPUS, and Web of Science. Further, a structured search for grey literature, such as national or organizational reports, was performed using the Canadian Agency for Drugs and Technologies in Health (CADTH) checklist for grey literature “Grey Matters”.availability of eHealthMonitoring or evaluating use of eHealthMonitoring or evaluating Monitoring and Evaluation, eHealth, and Availability. Each key term had several synonymous sub-terms. When applicable, major terms were used . To ensure the detection of literature not yet indexed with major terms, free text search was used. We did not seek to identify the specific metrics used for monitoring  as these metrics would be specific to the organizational setup of national health systems and context.In the availability section, key search terms were A full search strategy for a PubMed search can be found in An iterative approach to selecting literature was applied, entailing continuous assessment of eligibility criteria and the screening process. A literature directory was created in Mendeley All materials were categorized into one of three broad categories based on an approach described by Wong et al.Category 1: Peer-reviewed study with empirical measures of availability/use. Clear articulation of the methodological approach to monitoring or evaluating availability or use of eHealth at a national or regional level, covering design, data collection, analyses, and relevance.Category 2: Non-peer-reviewed report with empirical measures of availability/use. Reports by government or non-government, health associations, professional bodies, and centers of excellence. We included these because national monitoring data is intended for public/broad consumption and therefore often not submitted for peer-review.Category 3: Methodology recommendations. Material presenting comprehensive recommendations of methodology of national monitoring or evaluating availability and/or use of eHealth. Often non-peer-reviewed.Based on the approach by Arksey and O’MalleyA general issue when comparing systems and services across countries is the subtle differences in terminology and understanding of what constitutes, for example, an electronic health record. Collecting and comparing data on functionalities rather than systems is a method of overcoming these cultural differences.Provider-centric electronic records cover the range of Electronic Medical Records (EMRs), Electronic Health Records (EHRs), and Electronic Patient Records (EPRs) and “include systems that are used by healthcare professionals to store and manage patient health information and data, and include functionalities that directly support the care delivery process.”The results of the search strategy provided a list of 1135 indexed publications for monitoring and evaluating eHealth availability and 1219 indexed publications for monitoring and evaluating eHealth use, see A total of 117 full-text publications were reviewed with 80 excluded due to not fulfilling eligibility criteria after all  or not having a clear description of the methodology. After removing one additional duplicate, a total of 36 publications were included in this scoping review. Of the 36 publications, 64% were empirical research studies, 28% were reports by governments or organizations, and 8% were published recommendations of methodology. ,The geographical origin of monitoring and evaluating activities  is distrMapping the publications showed that the majority of monitoring and evaluation activities were set in the Northern European countries and the United States. In 89% of the publications, the primary purpose was to monitor the availability and use of eHealth. Surveys were most commonly used (86% of the publications) and 42% of the publications referred to continuous or repeated data gathering activities.Publications: n = 28Category: 20 empirical studies,,,,,,,,,,,,Focus: The main aims were on monitoring the availability and use of provider-centric electronic records or functionalities, for example, entering and viewing clinical data, medication list, picture archiving, and clinical decision support.,,,,,,,,,,,Methods: Most publications had a national scope (n = 24). Surveys were the data collection method most used to gauge the availability and use of provider-centric electronic records. Few publications also used business data  to measure availability and use of provider-centric electronic record functionality.,,n = 18) and continuous or repeated activities (n = 10).Publications: n = 13,,,,,,,Category: Five empirical studies,,,,,,,,,,,Focus: The main aims in all publications were on monitoring the availability and use of patient-centric electronic records or functionalities, for example, online appointment scheduling, medication renewal, viewing of clinical data, and electronic communication with General Practitioners. Four publications addressed the citizens’ perceptions of eHealth.,,,Methods: All publications had a national scope and used surveys to gauge the availability and use of patient-centric electronic records. One publication also used business data  to assess the amount of use of patient-centric electronic record functionality.n = 7) and continuous or repeated activities (n = 6). Only two publications surveyed patients directly,,Publications: n = 16,,,,,,,,Category: 10 empirical studies,,,,,,,,,,,,,,Focus: The main aim was on monitoring the availability and use of HIE (n = 15), such as ePrecriptions, eReferrals, and exchange of clinical history, laboratory results, or radiology reports with external organizations. Only one publication regarded evaluating the systems’ effect on the organization.Methods: Most publications had a national scope (n = 13). Surveys were the data collection method most used to gauge the availability and use of HIE functionalities. A few publications also used business data .,,n = 6) and continuous or repeated activities (n = 10).Publications: n = 3,,Origin: Canada Health Infoway,Focus: The publications present thorough methodological recommendations and approaches to monitoring and evaluating eHealth. Methodological recommendations and a wide selection of indicators are provided within different domains and functionalities. All OECD domains presented in ,Methods: Data collection through survey methods is the main methodology described. Canada Health Infoway also emphasizes the use of business data  and describes which indicators could be monitored by methods other than surveys.,This scoping review synthesizes the current literature on national approaches to monitoring and evaluation of availability and use of eHealth.,,,ability to use a given functionality or system , whereas patient-centric electronic records were only addressed in 44% . This co,,Surveys are by far the most common data gathering method for monitoring and evaluating national availability and use of eHealth (used in 86% of the publications). Surveys are cost-efficient and can be used to obtain information on phenomena that are not easily measured otherwise. However, surveys are prone to issues of low external validity and bias. Recall and social desirability biases are also common limitations of surveys.Our results also reveal the potential challenge of lack of repeated national monitoring and evaluation efforts. Repeated or continuous data collection is needed to measure secular progress or to evaluate the impact of policy changes (or other interventions). Ongoing measures of eHealth progress, therefore, supports evidence-based approaches to eHealth policy.,,Finally, our results reveal that, while there are national or international methodological approaches to eHealth monitoring, there are multiple approaches that are not harmonized.The search strategy required iteration as the terminology within the research field of eHealth changed over the years, and it required adding new terms and definitions . In addition, many publications on eHealth monitoring and evaluation might only be disseminated through conferences or posters, which are not indexed in bibliographic databases in general. Thus, the choice of search terms and the focus on bibliographic databases may induce selection bias.Most publications evaluated eHealth at the single institution or single-system level - and were therefore excluded. To capture a broader set of eHealth monitoring efforts, we included grey literature and it is possible that our results would be different if we had limited studies to the peer-reviewed literature. However, we do not feel that the peer-review process would fundamentally alter the content or methods of the monitoring that was the focus of our review.Monitoring eHealth adoption is essential for providing an evidence base on which to formulate future national eHealth policies and for evaluating the effectiveness of the efforts.The authors contributed to the manuscript as follows:Substantial contributions to the conception and design of the work (Villumsen and Nøhr); and the acquisition, analysis, or interpretation of data for the work .Drafting the work (Villumsen and Nøhr) and revising it critically for important intellectual content (Adler-Milstein).Final approval of the version to be published .Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved .ooz071_Supplementary_DataClick here for additional data file.


Quality of care during the intrapartum and immediate postnatal period for maternal and newborn health remains a major challenge due to the multiple health system bottlenecks in low-income countries. Reports of complex interventions that have been effective in reducing maternal and newborn mortality in these settings are usually limited in description, which inhibits learning and replication. We present a detailed account of the Preterm Birth Initiative (PTBi) implementation process, experiences and lessons learnt to inform scale-up and replication.Using the TiDieR framework, we detail how the PTBi implemented an integrated package of interventions through a pair-matched cluster randomized control trial in 20 health facilities in Migori County, Kenya, and the Busoga region in east central Uganda from 2016 to 2019. The package aimed to improve quality of care during the intrapartum and immediate postnatal period with a focus on preterm birth. The package included data strengthening (DS) and introduction of a modified WHO Safe Childbirth Checklist (mSCC), simulation-based training and mentoring (PRONTO), and a Quality Improvement (QI) Collaborative.In 2016, DS and mSCC were introduced to improve existing data processes and increase the quality of data for measures needed to evaluate study impact. PRONTO and QI interventions were then rolled out sequentially. While package components were implemented with fidelity, some implementation processes required contextual adaptation to allow alignment with national priorities and guidelines, and flexibility to optimize uptake.ClinicalTrials.govNCT03112018, registered December 2016, posted April 2017.Lessons learned included the importance of synergy between interventions, the need for local leadership engagement, and the value of strengthening local systems and resources. Adaptations of individual elements of the package to suit the local context were important for effective implementation, and the TIDieR framework provides the guidance needed in detailed description to replicate such a complex intervention in other settings. Detailed documentation of the implementation process of a complex intervention with mutually synergistic components can help contextualize trial results and potential for scale-up. The trial is registered at The online version contains supplementary material available at 10.1186/s43058-021-00109-w. Contribution to literatureRandomized intervention trials often have limited descriptions of the detailed implementation process to enable replication. Results from the PTBi CRCT showed a significant reduction of fresh stillbirth and mortality among eligible infants in a low-resource setting, and the details for replicability are a critical and complementary part of the study findings.This paper provides a detailed account of a complex intervention, which is important particularly to inform decision-makers in low-resource settings about reproducibility and scale-up.These findings contribute to the growing acknowledgement of the importance of using a standardized framework such as TIDieR to ascertain key criteria needed to adopt these types of evidence-based packages for maternal and neonatal health.Over 80% of the estimated 15 million preterm babies born each year occur in sub-Saharan Africa and South Asia . ComplicSome authors have proposed approaches to QoC improvement around the time of birth that are integrated and address the complexity of the particular care ecosytem, including provider skills and knowledge, system readiness, and experience of care –11. HoweThe Template for Intervention Description and Replication (TIDieR) framework provides an approach to document detailed implementation of both intervention and control arms of a study trial . The fraThe East Africa Preterm Birth Initiative (PTBi-EA) addressed the complexity of QoC by implementing a multi-component intrapartum and immediate newborn care quality improvement package in rural facilities in Kenya and Uganda, between 2016 and 2019. The primary results of this cluster randomized control trial (CRCT) found a 0.66 odds ratio  for the combined primary outcome indicator of fresh stillbirths and 28-day mortality among eligible infants . This paPer the TIDieR checklist, the intervention name, rationale, and locations  are described in the “Methods” section, while the intervention activities, mode and frequency of delivery, adaptations, modifications, and fidelity  are presented in the “Results” section. A populated TIDieR checklist is provided as Additional file We conducted a pair-matched CRCT in 5 districts in the Busoga Region of east central Uganda  and Migori County, in south west Kenya, among 20 public sector and private-not-for profit health facilities. The Busoga region has a population of about 3 million people, 80% of whom live in rural areas and practice subsistence farming. The Busoga regional neonatal mortality and stillbirth rates are similar to the national rates of 27/1000 live births and 21/1000 total births, respectively . Migori Due to the study design, sample size targets, and workforce strikes in Kenya, the two countries had different timelines, though they began at the same time. The intervention period in Uganda was from October 1, 2016, to May 31, 2018, while in Kenya it was from October 1, 2016, to April 30, 2019  were randomly selected to receive the complete intervention package, as were 3 larger referral hospitals—the county referral hospital in Kenya, the regional referral hospital in Uganda, and one district hospital in Uganda that serves as a de-facto referral hospital for other district hospitals in the study. These hospitals were determined to not have a comparable pair and were excluded from the randomization process and primary analysis.The intervention package included maternity register data strengthening (DS), introduction of a modified WHO Safe Childbirth Checklist (mSCC), capacity building in emergency obstetric and neonatal care using PRONTO simulation training and mentorship, and creation of a Quality Improvement (QI) Collaborative aiming to improve intrapartum and newborn care, particularly among preterm births. All 20 study facilities received DS and mSCC components to foundationally improve data quality, standardize definitions of key indicators related to gestational age (GA) and newborn outcomes, and reinforce critical EBPs. Intervention sites received additional mentorship and support of these two interventions through synergies with PRONTO simulation and team training and QI Collaborative components. A summary graphic describing the package by arm can be found elsewhere .Before any interventions were introduced, we conducted health facility assessments in December 2015 and January 2016 for Kenya and Uganda, respectively. These assessments helped us select which facilities could accommodate this intervention package and what needed to be “added” in order for the site to be included in the study. The initial facility assessments uncovered critical gaps in equipment and supplies. These investments, while critical, were relatively small in scale and did not exceed US$50,000 in either country and were provided to both intervention and control facilities. Since these items were only provided at the start of the study, they were not considered part of the intervention package. A description of all equipment and supplies is provided in Additional file In Uganda, neonatal mortality has barely changed over the past 15 years. The neonatal mortality rate is 27/1000 live births despite increased facility delivery from 38 to 77% during the same period . This poThe intervention package was developed based on available evidence that up to two thirds of neonatal deaths could be saved by improved uptake of EBPs described by the Every Newborn Action Plan (ENAP) . PreviouOur theory of change is documented in the published protocol . The PTBDuring the study period, each country used a Program Impact Pathway (PIP)  for eachwhiteboard video we created to teach providers about the importance of accurate maternity register data [Rather than introducing parallel data systems, we committed to using an existing data system, namely maternity registers. Improvements in consistency, completeness, and maternity register data use in the study sites were critical to establishing baseline measures and to achieving a demonstrable impact on preterm survival. In addition, we felt that improving these measures in existing data sources would bring sustainable change to our sites, as outlined in a ter data .We conducted an initial Data Quality Assessment (DQA) during the pre-trial facility assessments. We documented what components of the existing data systems warranted additional investment and targeted strengthening . These fThe DS team included 5 data specialists in Kenya and 2 in Uganda. The initial DS training lasted for 2 days in Kenya and 3 days in Uganda in May 2016. DS training included an overview of the health information system and guidelines with facility clinical leadership, health records officers and hospital maternity in-charges, followed by site trainings with maternity nurses and health records staff. We also introduced standard tools to improve GA assessment (GA wheels and tape measures) and reviewed indicator definitions and standardization based on national guidelines.According to the study protocol, we planned a single DS training at the start of the study; however, after the initial DS training, most providers still lacked adequate knowledge and skills in capturing the relevant data elements. Thus, we modified our efforts in all facilities. Two additional 2-day refresher DS trainings were offered during the course of the study, once per year of implementation. In Uganda, additional feedback on data quality issues, data source completion and adherence to indicator definitions was provided during the monthly data collection sessions for all facilities. In Kenya, refreshers were conducted less formally during monthly data mentoring and collection visits with staff available.In addition, there was a lack of inpatient maternity patient charts for recording clinical notes, which are critical for patient management and monitoring QI processes. The project therefore provided a total of 79,200 and 13,000 maternity charts in Uganda and Kenya respectively, and 11,880 neonatal charts, divided into two batches for Kenya and three for Uganda, based on annual utilization of each facility to ensure availability of patient records. In Uganda, the maternity chart was developed based on the existing Ministry of Health (MoH) charts used at the national referral hospital, while we adopted the neonatal chart previously used by the Uganda Pediatric Association for neonatal surveillance in newborn care units. In Kenya, the county’s standard chart was provided.We worked with in-country stakeholders to develop and iterate a Data Dashboard to improve data use and dissemination of our study data. During the intervention phase, the DS team visited all facilities monthly to share dashboard reports and review data with data officers, providers, and maternity ward in-charges.DQAs were conducted after every 6 months, twice in Uganda, and three times in Kenya to collect data on specific DHIS-related indicators and assess gaps between reported and actual indicators  across all control and intervention facilities. We followed the DQA approach utilized in HIV treatment data monitoring systems .We used the WHO Implementation Guide for the Safe Childbirth Checklist to guide our approach to adaptation and roll-out of the checklist . The PTBBefore launching the intervention package in all facilities, a pilot trial of each country’s mSCC provided additional input to optimize content and roll-out. The finalized mSCC was introduced by the clinical study teams during initial DS training activities at all 23 sites. The mSCC was included in the maternity inpatient chart or record for each woman in all facilities.Following low response to an initial roll-out assessed at 8 months at all facilities, modifications on use of the mSCC was deemed necessary. The Kenya team decided to provide a small monetary incentive (US$.50) at each study facility (intervention and control), shared among maternity and newborn unit health workers, for each mSCC completed. In Uganda, care providers requested that the checklist be split up by pause point and integrated into the maternity patient chart, rather than be placed at the beginning or end of the chart. For example, the triage section was placed at the start, while the before discharge section was placed at the end. In Kenya, the checklist was incorporated as part of the patient chart and was placed at the front page. Subsequently, in the intervention facilities in both countries, mSCC completion was included among the QI indicators. This led to increased awareness of the importance of the mSCC and improvements in its utilization among intervention hospitals.Study personnel in both countries monitored mSCC completeness and uptake by each of the 5 pause points either by convenience or purposive sampling. In Kenya, study data staff reviewed all maternity charts of preterm cases eligible for follow-up each month and abstracted data from the checklists of preterm cases, where available. These data were regularly reported back to facilities, and were included in the Data Dashboard.In both Kenya and Uganda, key stakeholders including midwives, nurses, doctors, obstetricians, MoH officials and study clinicians met to adapt the PRONTO curriculum to meet the local training gaps and address critical EBPs related to preterm labor and birth Table . PRONTO Five mentors/trainers in Kenya and 10 in Uganda underwent a two-step process for becoming PRONTO mentors. The process involved an initial 5-day introduction to simulation facilitation and part one of the PTBi curriculum training, followed by a second 2-day advanced facilitator training 4 months later. In alignment with facility levels and staffing, 5 mentors (nurse and midwives) in Kenya were chosen from among 10 candidates who completed the initial training of mentors. Mentors were selected based on knowledge in obstetric and newborn care, prior EmONC training, and effective facilitation and leadership skills. Mentors had to be outgoing, empathetic, compassionate, and engaging, as well as willing to work in different facilities in the specified geographical areas. In Uganda, 20 candidates were invited to participate in training based on recommendations from stakeholder institutions and represented a diverse group of providers  including both regional and national levels. The 10 Ugandan mentors were chosen based on knowledge in obstetric and newborn care, prior EmONC training, facilitation skills, and availability. The mentors were an interdisciplinary team of 2 nurses, 1 medical officer, 4 obstetricians, 2 pediatricians, and 1 neonatologist. Half of the mentors were from national level health facilities, while the other half of mentors were from the implementation sites to enable continuous mentorship and sustainability.The training mode of delivery varied, by design, between Kenya and Uganda. Kenya utilized an in situ mentoring program whereby each intervention facility received low-dose high-frequency/4-day per week mentorship by a pair of mentors who rotated among intervention sites during the study duration. They spent a combined total of 9–12 weeks at each intervention facility over the 31 months study duration in Kenya. Visits included bedside mentoring, video-recorded in situ simulations and debriefing, knowledge reviews, skills stations, teamwork activities, and mSCC support as well as QI roles.In Uganda, a high-intensity/less-frequent shorter modular strategy was selected by stakeholders engaged during study planning. A modular-based training program was paired with 4 2-day long simulation refresher trainings and mentorship visits. Initially, the refresher trainings were 1-day sessions but later (during the second session) modified to 2-day long activities to allow bedside mentorship which was required for knowledge translation and skills application on patients rather than simulation alone. The 2 modules and refresher training visits were spread out during the study period, and similarly amounted to approximately 6 weeks of mentorship  and an obstetrician for Uganda.Use of the mSCC was integrated into all PRONTO clinical activities to provide facility staff with continued opportunities to reinforce its use. Any change ideas that arose from these PRONTO activities were integrated into QI efforts.In addition to simulation debriefing and feedback during mentorship visits, data related to total providers trained, number of simulations, and knowledge reviews conducted were displayed on the Data Dashboards. Knowledge review data also contributed to the content for bedside clinical mentorships.Each facility in the intervention arm created a designated QI team comprising facility leadership, health record staff and midwives from the labor ward, postnatal ward and the neonatal special care unit (8–12 people for Uganda and 3–5 people for Kenya). A QI technical consultant helped provide foundational training in QI methods.Individual facility QI teams carried out Plan-Do-Study-Act (PDSA) cycles. Teams met twice a month for approximately 2 h with the support of a QI coach  to identify and prioritize problems or bottlenecks in the facility, determine solutions/change ideas, track changes and outcomes, and make adjustments based on the results. The QI coach was a study team member who had expertise in supporting PDSA cycles and use of QI tools, including the documentation journal, process mapping, and annotated graphs for process results tracking. The PRONTO mentor participated in QI activities and had both relevant clinical expertise and a clear understanding of the local health system.The study allocated nominal funds (US$100 per intervention facility each month) to address stock outs of key medical supplies, items or equipment that were identified as necessary to implement the identified change ideas. However, in Uganda, due to bureaucracies in procurement it was difficult to provide the needed items in real time, thus supplies were delivered in 2 batches: in the initial facility strengthening and midway the implementation phase. In Kenya, for change ideas that needed funding, study facilities wrote short proposals that described the problem identified and the change concepts. This, upon approval by the country program leadership, was processed to procure the items needed to execute the change idea.QI teams across facilities in each country (known collectively as the QI Collaborative) participated in a total of 5–6 joint learning sessions during the intervention period to discuss core learnings and QI indicators. A total of 5 learning sessions were conducted in each country Fig. . Each leThe first learning session marked the beginning of the QI Collaborative implementation, and involved orientation of the QI teams on the QI indicators and their definitions, sharing the baseline process data, and agreeing on the end targets. Three process indicators were identified initially as important to contribute to a decrease in neonatal mortality among preterm infants: KMC uptake, GA estimation, and use of ACS among eligible women. Modifications were made in each country by selecting additional process indicators based on national priorities and needs Table . PartogrDuring learning sessions, QI teams were taught use of the QI tools, like process mapping to identify system bottlenecks, how to develop and test change ideas, monitoring, and decision-making. QI/clinical mentors and facility leadership attended the learning sessions to identify the persistent gaps they could work on for improved performance. For example, in Uganda, more midwives were allocated to the neonatal special care unit in 3 health facilities, while in 2 other facilities, bigger rooms closer to maternity wards were allocated to be used as neonatal special care units.Elements of the package, specifically the Data Dashboard, PRONTO, and the mSCC were integrated with QI efforts. First, the Data Dashboard generated figures to display the data visually, which better informed facility teams on progress, and highlighted remaining performance gaps. Second, areas of improvement were identified through PRONTO training and mentorship, which were shared with the QI teams. Lastly, the mSCC served as a data source to document selected QI indicators.QI materials included the documentation journal for tracking progress of performance of various indicators. The data sources for QI indicators included the patient charts, the maternity registers, and the mSCC. QI data were displayed on the Data Dashboard and shared monthly with data officers, care providers, and maternity ward in-charges.We achieved fidelity to the expected number of trainings for the intervention components as initially described in the protocol . SpecifiWe have summarized additional measures of fidelity in Table The PTBi CRCT yielded significant results—a 34% reduction in the odds of combined fresh stillbirth and neonatal mortality among eligible neonates in the intervention facilities, relative to the control facilities . UnderstFirst, EBPs promoted through an intervention package with multiple mutually reinforcing components Fig.  can offeSecond, local leadership engagement and adaptation of the intervention package were important both to align with national guidelines and priorities and to increase local ownership. In our case, leadership from both medical and political domains, and engagement from other national and regional stakeholders right from the start was critical. During the planning phase of each intervention component, key stakeholders were engaged. For example, key members of the national cadre of EMONC trainers, professional societies, the country’s first neonatologist, and regionally based obstetricians and pediatricians were all included in the PRONTO curriculum development consultation and invited to train as mentors. In addition, during implementation activities, local leadership was invited to PTBi trainings and QI Collaborative learning sessions, as well as special events about prematurity. Their involvement improved the general awareness of the burden of preterm birth among leaders, placing prematurity on top of their newborn health agenda. Local ownership by facility-based staff also helped to effectively integrate the interventions and address system bottlenecks, such as the infrastructure and staffing for the neonatal care units. QI change ideas, for example, catalyzed system improvements which required approval from facility administrators, such as relocating neonatal special care units closer to the maternity ward. During this process, local champions emerged and helped in accelerating the uptake of EBPs, at the facility and system level. These champions were critical to the success of a program and efforts should be made to identify and mentor champions at all levels of the system early on.While we were thoughtful in the planning, implementation, and evaluation of each of the components from the start, we designed the package to be flexible in nature to enable adaptation and responsiveness to each country’s needs and health system challenges. Each component of the intervention underwent several adaptations, which were important to align the program with national strategies and the local context Table . For exaOur third lesson highlights the importance of working with and strengthening existing health system resources, so as to enhance local ownership and potential for replicability. We believe relying on and strengthening the existing facility registers as the primary data source is an important strength and has had lasting impact. This effort contributed to establishing a more reliable and sustainable system for monitoring outcomes, which has been documented by other QI initiatives . FurtherIt is worth noting that this intervention took on a regional approach through a network of facilities as collaborators. While this was part of the design for practical research reasons, it may have had unintended benefits in strengthening the relationships among facilities. This collaboration improved communication among providers across facilities . Moreover, this regional approach also allowed us to liaise with other organizations working in the area implementing QI or other RMNCH interventions. For optimal impact, we recommend similar efforts to consider this regional network approach covering a referral system in order to maximize outcomes. We recognize, however, that as quality improved in the intervention facilities, more referrals came in from neighboring hospitals and lower level facilities. This increased volume had the potential to compromise the quality of care provided.Despite these lessons learned, several challenges existed. For example, most of the intervention package was implemented using existing personnel to enable sustainability. Besides the PRONTO mentors and QI consultant, there were no additional staff brought in by the project, which is often part of large intervention studies , particuPublishing more extensive details for complex interventions are important for reproducibility and a more robust understanding of the various elements that contribute to the findings , 34. ThiAdditional file 1: Supplemental Table 1. The TIDieR (Template for Intervention Description and Replication) ChecklistAdditional file 2: Supplementary Table 2 Equipment and Supplies Provided by Country


Scientific Reports 10.1038/s41598-019-54408-3, published online 05 December 2019Correction to: The original PDF version of this Article contained an error in the order of corresponding authors. This has now been corrected in the PDF version of this Article.Additionally, the Author Contributions section in this Article was incorrect.“Liuting Zeng and Kailin Yang dominate the concept and carry out a comprehensive design; Tingting Bao and Bing Jin participant in the concept and design. Kailin Yang, Liuting Zeng, Tingting Bao and Zhiyong Long are responsible for data analysis and interpretation. Liuting Zeng, Kailin Yang, Tingting Bao drafted the paper; Bing Jin supervised the study; all authors participated in the analysis and interpretation of data and approved the final paper. Kailin Yang, Liuting Zeng and Tingting Bao should be considered joint first author.”now reads:“Liuting Zeng and Kailin Yang dominate the concept and carry out a comprehensive design; Tingting Bao and Bing Jin participant in the concept and design. Kailin Yang, Liuting Zeng, Tingting Bao and Zhiyong Long are responsible for data analysis and interpretation. Liuting Zeng, Kailin Yang, Tingting Bao drafted the paper; Bing Jin supervised the study; all authors participated in the analysis and interpretation of data and approved the final paper. Kailin Yang, Liuting Zeng and Tingting Bao should be considered joint first author. Bing Jin is the first corresponding author because he supervised the study.”


The use of artificial intelligence (AI) in medicine will generate numerous application possibilities to improve patient care, provide real-time data analytics, and enable continuous patient monitoring. Clinicians and health informaticians should become familiar with machine learning and deep learning. Additionally, they should have a strong background in data analytics and data visualization to use, evaluate, and develop AI applications in clinical practice.The main objective of this study was to evaluate the current state of AI training and the use of AI tools to enhance the learning experience.A comprehensive systematic review was conducted to analyze the use of AI in medical and health informatics education, and to evaluate existing AI training practices. PRISMA-P  guidelines were followed. The studies that focused on the use of AI tools to enhance medical education and the studies that investigated teaching AI as a new competency were categorized separately to evaluate recent developments.This systematic review revealed that recent publications recommend the integration of AI training into medical and health informatics curricula.To the best of our knowledge, this is the first systematic review exploring the current state of AI education in both medicine and health informatics. Since AI curricula have not been standardized and competencies have not been determined, a framework for specialized AI training in medical and health informatics education is proposed. Artificial intelligence (AI) is one of the most disruptive innovations in health care, and the topic has attracted the attention of physicians, clinicians, researchers, and medical device industry professionals. Recent advancements in machine learning (ML) and deep learning (DL) algorithms and cloud computing have increased the adoption of AI. Consequently, applications that can handle a large number of unstructured data sets and solve complex problems have become a part of daily clinical practice.Most AI applications process data and run self-learning algorithms behind the scenes. Although some AI applications provide data-driven recommendations to clinicians, others may not offer an option to accept, reject, or modify the output. The recommendations AI applications provide through statistical correlations may not be the best option because human-made AI algorithms may be flawed. To use and screen AI-based decisions, clinicians and health informaticians who develop AI applications should have an excellent understanding of the underlying AI concepts. This paper will focus on the emerging need for formal AI education in medicine and health informatics.Intelligence requires the capacity to perceive contexts, associate contexts to actions, and act. Even though the concept of machines that imitate intelligent human behavior is not new, AI has recently become a topic of interest . As an aAI, ML, and DL are closely related, and the absence of universal definitions might be confusing; however, the difference between AI, ML, and DL is simple. AI is defined as “the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages” . AI-baseThe main purpose of this study was to investigate peer-reviewed publications focused on AI education and to determine objective assessment methods for AI skills and competency training for medical and health informatics professionals. The impact of AI on the learning experience was evaluated to assess the need for AI education. As medical education, clinical informatics education, and health informatics education are closely related, medical and health informatics education trends were analyzed together. The American Medical Informatics Association has been working closely with the Commission on Accreditation for Health Informatics and Information Management Education to determine health informatics competencies; clinical informatics became a medical subspecialty in 2011 . AlthougUsing a replicable systematic search strategy, a full-text review was performed between November 2019 and February 2020. PRISMA-P  systematic review methodology introduced by Moher et al  was usedThe combination of five groups of keywords was used to search PubMed, IEEE  Xplore Digital Library, CINAHL  Plus, and ScienceDirect databases: (1) medical education, (2) medical training, (3) artificial intelligence, (4) machine learning, and (5) deep learning . OverallBased on the following inclusion and exclusion criteria, the selection process was applied. Peer-reviewed research articles, review papers, conference papers, case reports, correspondences, discussions, viewpoint papers, editorials, mini-reviews, and short communications papers that focused on AI tools to enhance the learning experience in medical and health informatics education or teach AI as a new competency published after 1990 were included . Book chOne of the goals of this systematic literature review was to evaluate existing studies and determine the current state of AI education. The selected papers were used to identify the answers to three research questions.The first research question is the following: what topics are discussed in peer-reviewed publications that focus on medical and health informatics education and AI? To answer this question, the selected publications were classified based on their education foci and the characteristics of included studies were summarized in Of 26 publications, 16 (61%) investigated the use of AI applications in medical education (Category 1) and 10 publications (39%) evaluated AI education in medicine  Levels of Evidence ranking scheme . Of 26 aFinally, our third research question was the following: what is the status of AI education in medicine and health informatics? In our study, 10 articles discussing AI education were identified. Since this topic is a relatively new area in medical and health informatics education, our findings were consistent with recent developments.In their review, Paranjape et al  summarizA review conducted by Park et al  emphasizAs discussed earlier, ML is an AI technique to process massive amounts of data and make predictions using computers. Kolachalama and Garg  proposedThe radiology community embraced AI and ML long before other medical specialties, and pioneered the usage of AI algorithms in advanced imaging applications. A multicenter survey of undergraduate medical students determined students' optimistic views about the implications of AI applications for radiology . The FreThe technological advancements in computer and software technologies; digitization of health care data; and methodological developments in information science, philosophy, mathematics, linguistics, and psychology disciplines accelerated medical research programs that focus on ML and commercialization. Maturation of AI technologies changed the roles of clinicians, and novel decision-making processes in medical settings and innovative AI-based protocols have the potential to provide diagnostic and treatment decisions by analyzing complex data sets .Over the last decade, many AI researchers have concentrated on developing a proof of concept system for clinicians and patients. There is an increase in the number of studies that evaluate the effectiveness of intelligent reasoning. Intelligent monitoring technologies require new algorithms to detect anomalies, predict patterns, and make decisions.A recent independent report prepared for the UK Secretary of State for Health and Social Care explored how the health care workforce could be prepared to use digital technology. The report emphasized the skills gap in the workforce, and made some recommendations about the integration of digital health care technologies, AI, and data analytics in undergraduate curricula ,43.Clinicians should have a realistic view of AI, and become familiar with the right tasks for AI in health care. Formal training for medical and health informatics students should enable them to develop AI algorithms, use AI technologies in a competent manner, and keep bias out of AI tools. Any new AI technology might encounter something new for which it has no experience, and therefore the physician should be able to assess problematic decisions and take necessary precautions when needed. Recent discussions about the need for medical education reform emphasize the shortcomings of the current model of education . OverrelWhile machine learning and AI algorithms are able to handle high-dimensional data classification problems and medical image interpretation, their success rates in risk prediction and diagnosis are lower. Consequently, there is a need to determine the most appropriate application areas for AI in health care .There are different ways to implement AI in clinical practice, and clinicians and health informaticians need formal training to use the right approaches. Health informaticians and physician champions who design and develop AI-based protocols need to have a good understanding of complex algorithms, methodologies for data quality assessment, probabilistic forecasting, and comparative model assessment to work with engineers and develop reliable AI applications. Moreover, clinicians who use AI applications should become familiar with potential challenges.AI-based relational time pattern analysis replaced simple threshold-based diagnostic rules. Current medical education and health informatics curricula still do not provide the ability to understand AI communications and necessary skill sets to develop AI systems that can detect and analyze relational time patterns .The ability to interpret AI algorithms' mistakes and formulate the best strategies to correct these applications requires specialized training. Consequently, medical and health informatics education must emphasize algorithm-based platforms, and include relevant data analytics and AI topics in their curricula. Moreover, computer science and health informatics programs should consist of health care–focused digital skills training.To the best of our knowledge, no previous research has investigated the use of AI tools to enhance the learning experience and AI education of medical and health informatics students. The main findings of this systematic review are as follows: (1) Although there are several recommendations on the integration of AI into medical and health informatics curricula and some academic institutions implemented experimental training programs, AI and ML education are not a part of traditional medical and health informatics curricula yet. (2) Current medical education and health informatics accreditation standards do not require AI training, and AI competencies have not been determined. (3) Using the OCEBM Levels of Evidence classification table, the majority of studies were classified as Level IV and V, which indicates poor reference standards.Several efforts were made to design an optimal systematic review process; however, there were still many limitations. It is probable that some studies might not be listed in the peer-reviewed academic literature databases or might be published in a non-English language. Although this is a systematic review of the field, AI is a new technical discipline, particularly in medicine, and therefore the number of articles that met the inclusion criteria was limited.Overall, the selected publications did not provide specific details about different jobs' requirements and curriculum needs. The emergence of intelligent systems in health care requires new learning modalities. Even though several organizations, agencies, and work groups such as the International Medical Informatics Association , the ComA specialized AI education framework for various professional fields would be useful; using the results of this systematic review, we propose a framework for specialized AI training for different domains . Medical