Learning theory is an essential component for designing an effective educational curriculum. Reviews of existing literature consistently lack sufficient evidence to support the effectiveness of digital interventions for health professions’ education, which may reflect disconnections among learning theories, curriculum design, use of technology, and outcome evaluation.The aim of this review was to identify, map, and evaluate the use of learning theories in designing and implementing intervention trials of health professions’ digital education, as well as highlight areas for future research on technology-enhanced education via the establishment of a development framework for practice and research.We performed a systematic search of Medical Literature Analysis and Retrieval System Online, Excerpta Medica database, Cochrane Central Register of Controlled Trials (Cochrane Library), PsycINFO, Cumulative Index to Nursing and Allied Health Literature, Education Resources Information Center, and Web of Science for randomized controlled trials (RCTs) published between 2007 and 2016.21=8.2; P=.002) and validated instruments  have shown effective acquisition of learning outcomes.A total of 874 RCTs on digital health education were identified and categorized into online-offline, mobile digital education, and simulation-based modalities for pre and postregistration health professions’ education. Of these, 242 studies were randomly selected for methodological review and thematic analysis. Data were extracted by one author using a standardized form, with a  random sample extracted by a second author, in duplicate. One-third  of the studies reported single or multiple learning theories in design, assessment, conceptualization, or interpretation of outcomes of the digital education interventions. Commonly reported learning theories were problem-based learning , social learning theory , and cognitive theory of multimedia learning . Most of these studies assessed knowledge , skills , and performance  as primary outcomes with nonvalidated assessment tools . Studies with reported learning theories (χWe proposed a Theory-Technology Alignment Framework to safeguard the robustness and integrity of the design and implementation of future digital education programs for the training of health professionals. Digital education is one of the most popular and rapidly evolving approaches to teaching and learning in health professions’ education. It offers a wide range of intervention modalities using information and communication technologies, such as computer-assisted learning, mobile learning, and digital simulation–based learning, which enable individuals to acquire knowledge and skills in a timely and cost-effective manner with greater personal control ,2. The MImplementation of learning theories in the design of digital education could reduce such uncertainties as they provide structured theoretical and practical foundations that help educators develop the curricula, pedagogies, and assessments that are most relevant and conducive to student learning ,11. LearThere is a paucity of research on the extent to which digital intervention for health professions’ education design integrates educational theory. The lack of understanding of how learners acquire knowledge via different digital modalities makes it difficult to determine the appropriate outcomes to measure when evaluating the effectiveness of such interventions. Therefore, this methodological review aimed to address this important but often neglected area of digital health professions’ education. We carried out a critical analysis of digital interventions within health professions’ education to determine the extent to which learning theories were explicitly used in the intervention design and evaluation and examine how these theories have been implemented. Finally, on the basis of the resulting findings, we conceptualized a development framework for augmenting learning theories in the design of digital interventions for health professions.online-offline interventions. Studies that used mobile phones, tablets, personal digital assistants, and other handheld devices for delivering educational content were grouped as mobile digital education interventions. Finally, studies that utilized simulation in the learning intervention, such as VR, VPS, and DGBL, were categorized as  digital simulation–based education.Digital education includes a variety of technologies such as offline and online computer-based digital education, digital game–based learning (DGBL), massive open online courses, virtual reality (VR), virtual patient simulation (VPS), psychomotor skills trainers, and mobile digital education . For thiOur study is a methodological review that adopts both a quantitative and a qualitative evaluative approach. This review is part of a global evidence synthesis initiative on digital health professions’ education . A systeA total of 874 intervention studies, published between January 2007 and August 2016, on different areas of digital education, were identified through our database search. We included only pre- and post-registration health professions as listed in the Health Field Education and Training (091) of the International Standard Classification of Education  . LearnerData were extracted for each included study. We developed the data extraction form through pilot testing and revised it further according to the feedback from coauthors. We extracted information including random number, mode of digital learning, first author, year of publication, title of the study, name of the journal, sample size, study population, setting, country where research was originally conducted, primary outcomes, measurement instrument, validation of measurement instrument, and the theory mentioned in the included studies. Data were extracted by the first author (SB) and verified by the second author (MS), and discrepancies were resolved through discussion, with adjudication by a senior author (AH) when necessary.pedagogical or learning principles but did not name the theory or describe its relevant features, was not considered as having used a theory and was excluded from this review. Decisions on theory used were based on consensus of the working group in case of uncertainty .This study investigated the use of learning theory reported in the interventions of 3 modalities of digital health professions’ education. To be judged as having a theory used, 3 criteria had to be met. First, any study that explicitly named a learning theory used in the design of intervention or learning evaluation was considered as theory used. Second, any study that described the use of pedagogy and the theoretical framework relevant to learning theory in intervention design and learning evaluation was considered as theory used. Third, any study that did not explicitly mention a learning theory but had clearly employed a learning theory in intervention design and learning evaluations was considered as theory used. In instances where a study merely mentioned that its intervention or evaluation design was based on P<.05 was considered as statistical significance. All analyses were performed using Stata software .Data were directly entered into Microsoft Excel and subsequently cleaned for invalid entries. The analysis was mostly descriptive; we summarized data as frequency and percentage for categorical items and median and interquartile range (IQR) for continuous items. We analyzed predefined study characteristics of all included digital learning interventions as previously described in data extraction section. We quantified associations using the Chi-square test and 2-sided We evaluated 242 studies, published between 2007 and 2016, from 3 modalities of digital health professions’ education interventions: online-offline–based digital education , mobile digital education , and digital simulation–based education  studies that reported learning theories, a total of 42 theories were applied in the design of digital health professions’ education interventions . The mosGiven the vast number of theories reported, we further organized each theory into a set of thematic categories on the basis of its nature and characteristics. Among the 42 identified theories, only 13 referred to a specific learning theory. Of these, 7 theories were categorized under cognitivism, which focuses on the inner mental activity and information processing of learners. Furthermore, 3 theories were organized into the constructivism category. The remaining 3 theories were standalone idiosyncratic models of learning. A total of 9 theories were categorized under design-based learning, which comprises a combination of multiple learning theories for explaining learning processes and pedagogy practices. A total of 9 theories were categorized under behavior-change theories that explain the processes of health-related behavioral and attitudinal transformation. Furthermore, 3 theories were organized into the social sciences category, 3 into the decision-making and therapeutic framework category, 3 into the learning style category, and 2 theories into the motivational theory category. .21=12.6; P=.006) and significant primary outcomes  were statistically associated with the reporting of learning theory  of the studies were judged to have a relatively clear use of theory in interventions . Among tA closer examination of different digital modalities revealed that among offline-online intervention studies, (n=55), 37 out of 55 studies (67%) clearly elaborated on the purpose of the theory used, 29 out of 55 studies (53%) used validated measurement instruments to assess learning outcomes, and 17 out of 55 studies (31%) reported effectiveness of digital interventions compared with the conventional method of learning. Among the mobile digital intervention studies, all included studies (n=9) discussed the purpose of the theory used, 3 studies used a validated instrument to assess the outcomes, and 2 studies reported significant results. Among the digital simulation–based intervention studies (n=17), 11 studies out of 17 (65%) were judged to have employed theory in their interventions, 9 studies assessed outcomes implying valid tools, and 5 studies reported statistically significant results .This is the first ever study that comprehensively reviewed the application of learning theory within the design and evaluation of digital health professions’ education interventions, inclusive of online-offline–based education, mobile digital education, and digital simulation–based education modalities. Our analyses highlight 4 serious concerns that may hamper the developmental integrity of this fast growing industry, including (1) design and implementation of digital health professions’ education interventions without integration of appropriate pedagogy frameworks , (2) poor selection of learning theories for developing or supporting education interventions, (3) not selecting appropriate theory(s) for different modalities of digital health education such as computer-assisted learning, Web-based learning, mobile learning, and others, and (4) inappropriate usage of learning theory(s) and a nonvalidated assessment instrument that results in the mismatch of learning outcomes. The issues identified with the lack of learning theories and inappropriate application of learning theories to the development of digital education for health professions impede high-quality research into the efficacy of digital learning in this area.Specifically, our findings reveal that the interventions for health professions’ education did not utilize a learning theory or pedagogical framework in curriculum design, program implementation, or learning evaluation. In fact, only one-third of the intervention studies included in our review were informed by learning theories; nevertheless, the purpose of theory utilization was unclear in many instances. This phenomenon may be explained by the intentionality of interventions. All published articles focused primarily on the use of technology and its effectiveness on learning rather than carefully applying learning theory and pedagogy in the design of digital curriculums . ConsequWe also observed a lack of clarity and explanations for the selection of learning theories and pedagogy that informed digital education design, implementation, and evaluation . Poor seAnother important finding derived from our review is the improper selection and reporting of measurement tools and the use of a nonvalidated instrument. Although certain psychological or attitudinal constructs of learning outcomes, such as self-efficacy, often demand the use of specific self-assessment instruments, validation may not always be feasible or practical. In addition, nonvalidated self-assessment tools have also been widely criticized for their lack of accuracy . It is tMoreover, most studies in our review assessed knowledge followed by skills and performance as primary outcomes. We observed a high degree of incongruities  among the choice of learning outcomes, the underlying learning theory, and the use of measurement instruments for assessing such learning outcomes  were significantly less likely to use validated measurement instrument. Similarly, primary outcomes were statistically significant in a majority of the studies  that applied learning theory and validated assessment tools. In addition, more than half of the studies reporting a nonvalidated assessment tool may indicate a poor understanding of validity theory  and poor contextualization and application of validity and reliability in medical education . TherefoDespite the important findings reported, this review comprises some limitations. First, this is not a systematic review of theory-used in digital learning, which might increase the likelihood of missing some important study(s) pertaining to the topic. Second, we reviewed only one-fourth of published studies from 3 modalities; however, digital learning includes various modalities, but feasibility restriction does not permit us to include all published studies from different modalities of digital education. Nonetheless, future research may aim to focus on these issues, including proper reporting of theory-used and measurement tools for assessment. Given the multimodal nature of digital education, it is recommended that unless we have the solid theoretical guidelines for development, implementation, and evaluation, we are unlikely to achieve the desired learning goals .intended learning outcomes that they would like students to acquire upon the completion of education and training. On the basis of these outcomes, they proceed to curriculum design by incorporating various teaching and learning activities that they believe would lead to the intended learning outcomes, and thereafter the assessments and evaluations procedures are established to evaluate whether students have attained the intended outcomes, as well as assess standards of performance. We argue that this widely adopted practice known as constructive alignment [learning theory foundation, one that includes a collection of the most relevant learning theories to ensure the effective choice and application of teaching and learning activities. This would empower educators to better conceptualize and create a conducive pedagogical framework and effective learning environment to help students achieve success. Once a fitting teaching framework and learning environment are clearly delineated and established, they can then be augmented by using appropriate technology-assisted pedagogy, which effectively aligns with and supports learning theory foundation and curriculum design for developing theory-driven assessments and evaluations for adequately measuring students’ performance in accordance to the original intended learning outcomes.The appropriate application of learning theories in digital education is essential to ensure curriculum integrity and critical to successful learning outcomes ,47. Therlignment  is too sintended learning outcomes of (1) evidence-based clinical diagnosis with (2) team-based problem solving. To achieve these outcomes, one can apply a learning theory foundation that incorporates team-based learning theory with the Mayer’s cognitive theory of multimedia learning to inform curriculum design. This may comprise pedagogical instructions that utilize visual and auditory information processing for making proper patient diagnosis in a team-based environment. Furthermore, a set of theory-informed teaching and learning activities, matching technology-assisted pedagogy, such as the use of digital patient records including medical charts, x-rays, and computed tomography scans, coupled with voice recordings of patient intake assessments and patient-physician communications, can be used to facilitate the first intended learning outcome. Moreover, an online discussion forum with a built-in monitoring and feedback mechanism for course instructors can be created for trainee groups to deliberate on the digital patient medical information they have and to share their analysis of problem, to come to a joint diagnosis; this will serve to facilitate the second intended learning outcome. With this alignment and integration of curriculum design and technology-assisted pedagogy, an appropriate assessments and evaluations protocol can be developed to ensure that all intended learning outcomes are properly assessed. To assess the first outcome, evaluation on individual trainee’s understanding and application of digital patient medical information in clinical diagnosis may be conducted through short-answer questions and higher order MCQs that test the cognitive processes of information analysis and knowledge application. To assess the second outcome, continuous monitoring of team-based discussion, as well as evaluation of written reflections on group processes, and group-based problem-solving capacity via stimulated online clinical meetings may be conducted.Putting the TTAF in practice, a digital education intervention for medical trainees may aspire toward the In short, we recommend that instead of placing technology at the center of digital education, one must begin at the fundamental roots of learning theory and pedagogy. Therefore, the TTAF could be useful for designing an effective digital education intervention.Our study has opened the doors for future research by highlighting many problems in digital health professions’ education research and its different modalities. This multifaceted problem can be tackled through effective utilization of appropriate learning theories to foster stronger integration among intended outcomes, curriculum design, pedagogy activities, and evaluations with pertinent computer-assisted technologies. Imperatively, high-quality digital research using a clear theoretical framework, well-defined outcomes, and standardized assessment tools with adequate validity evidences is urgently warranted with proper reporting in methodology. Our review serves as an important guideline for researchers, educators, policy makers, and program designers to develop an effective intervention for the training of health professionals, and if applied proficiently, it would assist in advancing the field of digital research in medical education by addressing the various methodological shortcomings that exist in current interventional studies.


During the Dutch winter famine of 1944–1945, an interesting observation was made about the offspring born during this time—They had an increased risk of developing metabolic syndrome and other chronic diseases. Subsequent research has confirmed this finding as well as noting that health outcomes for many diseases are different, and often worse, for women. These findings, combined with the lack of enrollment of women in clinical trials and/or analysis of sex-specific differences are important factors which need to be addressed. In fact, Women’s health research and sex differences have historically been overlooked or lumped together and assumed equivalent to those of men. Hence, a focus on women’s health and disease prevention is critical to improve the lives of women in the 21st Century. In this review, we point out the critical differences biologically and socially that present both challenges and opportunities for development of novel platforms for precision health. The technologic and scientific advances specific to women’s precision health have the potential to improve the health and wellbeing for all females across the world. Women’s health, when in balance, is cyclical. In Celtic lore, the Triqueta (Trinity knot) represents three unique phases of a woman’s life: Maiden, Mother, Crone; and each stage presents unique health changes. Fundamental biologic sex differences influence even the microscopic composition of the female body ,2,3. Theprecision health offers the promise to provide personalized, female-specific lifestyle coaching, personal health education, and early disease detection to mitigate the consequences of diseases throughout a woman’s life. This contrasts with precision medicine which focuses on molecular targeting to diagnose and treat diseases thereby affecting the outcome after the onset of disease.Despite advances in medical technology and health care, the death rate among middle-aged American women is increasing . The difNovel devices and mobile applications are being developed specifically for women to optimize their health and provide access to information for prevention of disease. Indeed, it is predicted that health technologies for women will likely comprise a $50 billion industry by 2025. These personalized diagnostics, smart-tools, and evidence-based recommendations have the propensity to change how we understand and address women’s health with a focus on well-being and prevention. In the emerging “FemTech” market, diagnostics that utilize various body fluids  can now detect diseases  as well as monitor hormone levels and even nutritional status . In thisYoung women develop many of the health habits during puberty that last for their lifetimes. The first signs of female puberty typically start around age 9 with a growth spurt and increased fat deposition. Shortly thereafter, breast development begins accompanied by maturation and growth of reproductive organs. First menstruation (menarche) and ovulation are the most definitive signs of puberty in women. In the Northern Hemisphere, 95% of females reach menarche between 11 to 15 years of age . Puberta2) both surge just prior to ovulation, to stimulate the release of an egg/oocyte from a dominant follicle and this occurs around day 14 of an average 28 day cycle [The typical menstrual cycle is on average 28 days . The cycle is divided into four phases: follicular, proliferative phase, ovulatory and secretory phases. Luteinizing hormone (LH) and estrogen  have increased metabolism of all steroid hormones . In a thNeisseria gonorrhea are now increasing [Despite decades of declining incidence, the rates of syphilis and creasing . Inferticreasing . Emergincreasing . The rescreasing . EmployiEndometriosis is defined as the presence of endometrial glands and stroma in organs other than the native endometrium. Displacement of endometrial tissue cause painful menstrual cycles and can contribute to infertility. Currently, this disease is diagnosed using clinical symptoms and nonspecific findings on ultrasound, pelvic exam, and magnetic resonance imaging; however, there are no features or biomarkers currently available to definitively diagnose endometriosis without histologic confirmation with biopsies obtained via laparoscopic surgery . EmerginAccording to data from the Centers for Disease Control and Prevention (CDC), 10% of women (6.1 million) in the US have difficulty getting or staying pregnant ,25. With2, LH, thyroid-stimulating hormone, free thyroxine, prolactin, and testosterone is offered by a new company (Modern Fertility) [Ovarian reserve testing refers to an indirect estimate of the number and quality of ovarian follicles as an estimate to predict fecundity. As more women have children at older ages, there is an increased demand for predicting their fertile window. Ovarian reserve can be estimated through measurement of serum markers as a proxy for the function of somatic cells within follicles ,25,26. Ortility) . The labTracking the menstrual cycle, especially ovulation, is critical for timing intercourse in order to conceive ,12,28. NAs research continues to understand more on the molecular mechanisms of chronic diseases, companies such as Celmatix are offering genomic DNA tests to link single nucleotide variants to diseases such as endometriosis, polycystic ovarian syndrome, idiopathic infertility, hyperstimulation syndrome, and recurrent pregnancy loss. The implications of detection of single nucleotide variants remain unclear given the contribution of epigenetics and environmental factors involved in disease development . For exaHuman pregnancy is, on average, 40 weeks of gestation starting from the first day of the last menstrual period. It is generally divided into three trimesters, each lasting about 12–14 weeks, which corresponds with distinct phases of growth and development of the fetus.Early detection of chromosomal and genetic abnormalities has been part of prenatal screening since the mid-1960s. The most common chromosomal abnormalities that are the target of screening are chromosomal abnormalities in which an extra chromosome is present (trisomy). The primary chromosomal abnormalities targeted by screening are Down syndrome (trisomy 21), Edwards syndrome (trisomy 18), and Patau syndrome (trisomy 13). Prenatal screening initially began amniocentesis, a procedure whereby fluid is collected via a large bore needle inserted into the amniotic sac. Later, a panel of predictive serum markers was developed first targeted at specific high-risk populations for early detection of chromosomal abnormalities. Recently, cell-free DNA testing is an emerging non-invasive alternative to amniocentesis and chorionic villous sampling to detect chromosomal conditions as well as sex chromosomal conditions like Turner and Klinefelter syndrome . PrenataLabor commences with uterine contractions to prepare the cervix for delivery of the baby and placenta. There are now devices changing the way pregnancy and delivery are monitored, such as Bellabeat Shell which can amplify the sound of a baby’s heartbeat, and a wearable pregnancy sensor (Bloomlife) that can count and time contractions (either Braxton Hicks or true labor).The post-partum period brings unique challenges which have become the target of emerging technologies, including balancing career and motherhood, breastfeeding, weight and stress management, and urinary incontinence.Breastmilk contains critical nutrients for proper nutrition of the human infant. The nutritional composition of breastmilk is heavily influenced by the nutrition of the mother. For example, docosahexaeonoic acid  is critical for the structure, development and function of the central nervous system as well as for cognitive function, however, its level in breast milk varies based on maternal intake . LikewisThe stress and damage caused during pregnancy and labor can affect the pelvic floor (the supportive muscles between the tailbone and public bone) that and impact daily life activities . Pelvic 2 fall due to their decreased production by the ovaries. Sleep is often disturbed by hot flashes and the effects of hormone fluctuations. Depression and mood swings often also are common among this age group. Bone density, the cardiovascular system, and breast cancer risk are also influenced by menopause, particularly an early onset menopause. Saliva tests are available to determine hormone levels  to monitor perimenopausal hormonal changes . Prediction of the onset of menopause has also been explored, with increased focus on anti-Mullerian hormone (AMH) levels, although the precision of AMH levels for prediction of menopause remains limited [Health concerns that present during the peri-menopausal and menopausal transition can significantly impact the quality of life of women as levels of E limited ,41. The  limited .Emerging technologies and mobile applications focused on providing support and insight into the emotional and physical changes of menopause are increasingly available. For example, Menopause View and MyFlo mobile applications provide symptom trackers, educational resources, and peer support through this transition. In addition, Madorra has developed a non-invasive device to treat vaginal dryness. Given the relative lack of available resources and technologies available for women’s needs in this stage, additional focus on the needs specific to peri and post-menopausal women is warranted.2) has an overall cardio-protective role which helps prevent vascular dysfunction and cardiovascular disease. Due to the influence of E2, women’s blood pressure and platelets react differently under stress: buildup of platelets and reduction in microvascular blood flow can cause ischemia in critical organs such as the heart [2 in early menopause, they have a 33% higher risk of coronary heart disease and stroke [Currently, more women die of cardiovascular disease than men . Cardiovhe heart . Therefohe heart ,44,45,46d stroke . The Womd stroke ,49. Reced stroke . Wearabld stroke . Rigorou2 fluctuations lead to downregulation of metabolism and hunger-regulating hormones such as cholecystokinin (CCK), leptin, and peptide YY (PYY). Progesterone also increases cravings and energy intake [In the United States, severe obesity (Body Mass Index (BMI) > 35 kg/m) is more prevalent in women than men. Sex-specific risk factors include monthly and lifetime hormone fluctuations, menopause, and pregnancy . Specifiy intake . Female-Precision Health for women is also applicable for cancer risk assessment and the early detection of cancer ,52. For Cervical cancer is a malignant neoplasm of the uterine cervix, most commonly squamous cell carcinoma, but also includes endocervical adenocarcinoma. Human Papillomavirus is the causative agent of more than 99% of cervical squamous cell carcinomas, and is part of the screening algorithm . DetectiBreast cancer is a malignant proliferation of cells within the breast tissue. The heterogeneity of the disease is apparent morphologically under the microscope, but also increasingly by hormone markers and molecular/gene expression . HormoneOvarian cancer is a term that encompasses an array of benign to malignant entities. While high-grade ovarian cancer has the highest rate of mortality amongst gynecologic malignancies, early detection and distinction from benign mimics is lacking. Detection of ovarian cancers is notoriously difficult since physical symptoms often present late . FurtherIt is becoming increasingly apparent that sex-specific research and testing is important to provide individualized recommendations is needed to address mental health issues specific to women—Including postpartum depression, hormone balance and depression, and gender-specific depression management ,76. TwicAfter the discovery that the effects of Gamma-amino butyric acid (GABA), and drugs associated with GABA, can be affected by sex hormones, there has been increasing awareness regarding the dosage and metabolism of medications in women . AdditioProgressive dementia and cognitive impairment are hallmarks of Alzheimer’s disease (AD)  and two-A plethora of new technologies are being developed for women in the FemTech space which have the potential to make an impact on improving women’s health. Limitations to the technology include the currently high device costs, limited data on the accuracy and unbiased clinical trial and actual use trials. In addition, regulatory and technical challenges will pose challenges to implementation and use. Nevertheless, these emerging devices and resources may provide a unique opportunity to provide access to education and testing especially among low-resource and marginalized populations who would otherwise be unable or unlikely to access care. There is potential to continue to link these technologies to health professionals and conventional medicine practice to improve health outcomes, especially for the unique health challenges facing women throughout their life.


Over the years, technological and process innovations enabled active case finding (ACF) programs to expand their capacities and scope to have evolved to close gaps in missing TB patients globally. However, with increased ACF program’s operational complexity and a need for significant resource commitments, a comprehensive, transparent, and standardized approach in evaluating costs of ACF programs is needed to properly determine costs and value of ACF programs.Based on reviews of program activity and financial reports, multiple interviews with program managers of two TB REACH funded ACF programs deployed in Cambodia and Tajikistan, we first identified common program components, which formed the basis of the cost data collection, analysis, reporting framework. Within each program component and sub-activity group, cost data were collected and organized by relevant resource types . Total shared, indirect and overhead costs were apportioned into each activity category based on direct human resource contribution (e.g. a number of staff and their relative level of effort dedicated to each program component). Capital assets were assessed specific to program components and were annualized based on their expected useful life and a 3% discount rate. All costs were assessed based on the service provider perspective and expressed in 2015 USD.Over the two program years (April 2013 to December 2015), the Cambodia and Tajikistan ACF programs cumulated a total cost of $336,951 and $771,429 to screen 68,846 and 1,980,516 target population, bacteriologically test 4,589 and 19,764 presumptive TB, diagnose 731 and 2,246 TB patients in the respective programs. Recurrent costs were the largest cost components (54% and 34%) of the total costs for the respective programs and Xpert MTB/RIF (Xpert) testing incurred largest program component/activity cost for both programs. Cost per screening was $0.63 and $0.10 and cost per Xpert test was $25 and $18; Cost per TB case detected (Xpert) was $373 and $343 in Cambodia and Tajikistan.Results from two contextually and programmatically different multi-component ACF programs demonstrate that our tool is fully capable of comprehensively and transparently evaluating and comparing costs of various ACF programs. The ambitious End TB Strategy aims for reduction in tuberculosis (TB) incidence and mortality rates to 90 and 95% of 2015 estimates by 2035; yet, current estimated annual rate of decline for TB incidence (2%) and mortality (3%) is far below the rate needed (10% or more) to achieve these targets.In the past decade, multiple technological and process innovations enabled ACF programs to expand the capacity and scope beyond community symptom screening. Some examples of these innovations include involving mass media, educational programs, community engagements to increase awareness of the TB problem –7; use oWhile there is growing evidence demonstrating the public health impact of ACF programs, evidence on costs of ACF programs is far too scarce , 15. ThiTherefore, the overall aim of our study was to develop a generalizable and comprehensive cost data collection, analysis, and results reporting framework that can be adopted for retrospective cost assessment of a wide range of ACF programs. The use of such a tool can facilitate systematic and transparent comparison of costs–total program costs, service  unit, and program yield costs–of ACF programs. To demonstrate the data categorization, analysis, and reporting process using this framework, we report costs of two operationally different multi-component ACF programs targeting different geographical areas and populations that operated in Cambodia and Tajikistan supported by the TB REACH  wave 3 funding cycle.The TB REACH initiative of the Stop TB Partnership (UNOPS) is a multi-lateral funding mechanism supported by the Global Affairs Canada and the Bill and Melinda Gates Foundation with specific aims to improve TB case detection and care cascade in high burden countries by funding innovative projects. During the wave 3 funding cycle, TB REACH funded a total of 32 projects across 24 countries. In this study, we used two TB REACH wave 3 funded programs in Cambodia and Tajikistan to demonstrate the development, data collection and analysis processes of our costing framework. These two programs were one of the more complex, multi-component programs that targeted widely different geographic areas , population and health systems , and epidemiological settings .. Fir. Fir29].Results from our study should be interpreted with the following notion of limitations. First, we were only able to evaluate program costs and activity outputs retrospectively, limiting our analytic option to the top-down method. Therefore, we were not able to capture uncertainties in cost estimates that may arise due to variabilities in operations and workloads that may depend on operational settings conditions., 26, 30 Our evaluation of costs of two contextually and programmatically different ACF programs in Cambodia and Tajikistan demonstrate that our comprehensive costing data collection, analysis, and reporting framework can be used to evaluate and compare costs of a wide range of ACF programs in a highly transparent manner. Furthermore, in efforts to standardize costing practices, a similar approach can be adapted to evaluate other TB control programs  and services. However, retrospective top-down assessment of costs restricts the scope of how costs can be assessed and do not allow evaluation or monitoring of factors that influence the costs of operations. More research is needed in this regard, early and periodic (e.g. quarterly or monthly) assessment of costs using the proposed framework is highly encouraged.S1 File(DOCX)Click here for additional data file.S2 File(DOCX)Click here for additional data file.S1 Table(DOCX)Click here for additional data file.


Over the past two decades, we have seen a nationwide increase in the use of medical-legal partnerships (MLPs) to address health disparities affecting vulnerable populations. These partnerships increase medical teams’ capacity to address social and environmental threats to patients’ health, such as unsafe housing conditions, through partnership with legal professionals. Despite expansions in the use of MLP care models in health care settings, the health outcomes efficacy of MLPs has yet to be examined, particularly for complex chronic conditions such as HIV.This on-going mixed-methods study utilizes institutional case study and intervention mapping methodologies to develop an HIV-specific medical legal partnership logic model. Up-to-date, the organizational qualitative data has been collected. The next steps of this study consists of: (1) recruitment of 100 MLP providers through a national survey of clinics, community-based organizations, and hospitals; (2) in-depth interviewing of 50 dyads of MLP service providers and clients living with HIV to gauge the potential large-scale impact of legal partnerships on addressing the unmet needs of this population; and, (3) the development of an MLP intervention model to improve HIV care continuum outcomes using intervention mapping.The proposed study is highly significant because it targets a vulnerable population, PLWHA, and consists of formative and developmental work to investigate the impact of MLPs on health, legal, and psychosocial outcomes within this population. MLPs offer an integrated approach to healthcare delivery that seems promising for meeting the needs of PLWHA, but has yet to be rigorously assessed within this population. Four in five physicians in the US agrees that patients’ social and legal needs are as important to address as their medical issues. Eighty-five percent of primary care providers nationwide report that unmet social and legal needs lead directly to inferior health outcomes. Yet 80% of physicians surveyed lacked confidence in their ability to address such needs, impeding their ability to provide quality care . In respIn June 2018, the authors were awarded a two-year R21 study funded by the National Institute of Mental Health (#1R21MH115820–01) to develop a culturally-appropriate MLP intervention package to HIV continuum of care outcomes. The specific objectives of this ongoing study are: (1) To identify existing best practices among current MLPs that can be tested, replicated, and scaled-up as evidence-based practices for serving the plurality of PLWH; (2) To assess the effects of the identified MLP practices on: a) appropriately addressing the legal issues through clients’ satisfaction and case outcomes, b) reducing the legally-related psychosocial burdens for PLWH, and, c) increasing positive movement in the HIV continuum of care  among PLWH. (3) To develop an MLP-comprehensive HIV care diffusion model and its benchmarks of success to achieve positive movement in the HIV continuum of care  for diverse sectors of PLWH.The aim of this paper is to describe the methodology to design a structural intervention to improve HIV care continuum outcomes for people living with HIV. Structural interventions in the field of HIV are usually reported post-hoc with limited or no insights into how they are built and implemented. Drawing on the fields of implementation science in public health and medical anthropology, our study protocol draws on institutional ethnography and intervention mapping methodologies.HIV treatment and outcome disparities in the US and globally are attributed to syndemic factors, including lack of testing and access to care, discrimination, poor mental health, substance use, violence, and economic hardship –9. Fig. Institutional ethnography is an inductive method of inquiry that begins by looking at specific experiences, behaviors, and practices of individuals and then works outward to draw conclusions about the codes, systems, and structures by which they are governed , 15. ThiOur institutional ethnographic methodology draws on this dual focus by examining larger social phenomena from the perspective of multiple social actors and institutions. In our study, the social phenomena are the interactions and formal and informal systems governing medical-legal partnerships. The institutions are health care facilities inclusive of hospitals, federally qualified health centers, community health centers, and, HIV/AIDS service organizations. The social actors include: lawyers, legal aid staff, health care and social service providers screening for health-harming legal needs. Using the outward institutional methodological strategy that Smith and others have argued allow us to build institutional case studies of MLPs for PLWH on two main levels: the organizational and community levels –18. The In order to build our MLP intervention model for PLWH, we are incorporating institutional case studies and Intervention Mapping –21. InteWe began collecting data for our institutional case studies in August 2018 and will continue until May 31, 2019. Our institutional case study methods are divided into three components: 1) organizational qualitative data; 2) organizational-provider-client quantitative data; and, 3) provider-client qualitative data.We conducted interviews with 19 MLP key informants, visited four organizations involved in MLPs, and conducted four structured meetings with a Scientific Collaborative Board. Data was collected from July 2018 to January 2019.Based on the formative work conducted prior to the start of the study and meetings with the leadership at the National Center for Medical Legal Partnerships, we selected an initial sample of six MLPs working with PLWH within 100 miles radius of the authors’ host institution and, through referral, we contacted four additional MLPs. These ten MLPs are located in New York, Pennsylvania, Massachusetts, Hawaii, Wisconsin, Texas, Florida, New Jersey, and Washington, DC.Consisted of providers involved in MLPs that provide services to people living with HIV. We conducted interviews with 19 key informants. Key informants represented: lawyers, health providers, social workers, administrators, and researchers with specific expertise and experience in the MLP approach to care. These key informants, selected because of their first-hand knowledge and understanding of the community, provided insight into the issues facing PLWH and the MLPs that serve them and suggested various ways to address these challenges. When possible, we attempted to interview two key informants per MLP. The levels of experience with MLP services for PLWH ranged from 4 years to more than 20 years.The interview guide was designed by the authors and it contained ten questions and four probing questions. Sample questions included: How would you describe the MLP approach in your agency? What factors contribute to the success of the MLP and why? What are the main challenges associated with MLP implementation? The interviews were conducted in person or by phone, with the authors taking turns asking questions. Unscripted probing and follow-up questions are customary in key informant interviewing. During each interview, the authors took notes independently and subsequently compared recorded informant responses for accuracy. Discrepancies in the notes were clarified within the same week of conducting the informant interviews.The SCB consisted of a core group of MLP practitioners who are experts in their fields, including clinical care, legal services, administration, and social services. The SCB met to review findings from interviews and site visits and to provide guidance on next steps, including preparation for subsequent NIH grant submissions. The SCB meeting with the full board lasted approximately 2 hours. We also had subsequent individual meetings with SCB members. The authors took notes from the meetings.The analysis for organizational qualitative data followed a comparative approach, viewing data collection and analysis as a single concurrent process in which the method is fluid and evolves as the understanding unfolds from the data, a common approach in institutional ethnography. We first focused on “within” differences among key informants’ MLPs with regard to the provision of HIV services. Once we determined the organizational structure and organizational goals, we identified the various approaches to MLP HIV services and the rationale behind each approach. The researchers identified key components of MLP structure and implementation that were later validated by the SCB. We also gathered field notes and reflections from the sites visits. The authors listed, described, and discussed organizational-level best practices, including practices related to the continua of care and screening strategies for health harming legal needs; agency structure and staffing, including staffing of legal aid providers and operational hours; communication and information sharing among partners within the MLP; and written informational materials for clients and providers, including content of biomedical HIV prevention marketing initiatives. The above analysis was completed by February 15, 2019.From February 1 – May 15, 2019 we will conduct an online quantitative survey with 100 health and legal MLP service providers to collect organizational-provider data on MLPs serving PLWH.We identified MLP providers through the NCMLP database. The database includes names of existing MLPs, geographic locations , and voluntary contact information . A total of 294 MLPs have been registered by the NCMLP. Using a script approved by Temple University IRB, we are soliciting MLP service providers’ participation via email, one 3-min phone call, and one follow-up email. This personal approach will likely increase survey completion. The recruitment phone script and email contain information about the study and benefits associated with completion. We are also creating a moderate study-specific website page, which will include an attractive and interactive introduction and overview of the study, and hypertext-enabled links that allow viewers to contact study staff to inquire about participation, ask questions, and solicit feedback. Those who consent and complete the enrollment process will be asked for contact information  to arrange compensation and/or express interest to be contacted for the dyad in-depth interviews (Component #3). At the conclusion of the survey, participants will be offered a $30 gift card and thanked for their time.Part I inquires about general MLP information  and providers’ characteristics . Part II focuses on seven standard NCMLP performance measures for the prior 12 months . Part III will focus on identifying any existing services specifically geared towards PLWH and frequency of contextual barriers and facilitators to HIV medical adherence for their client population. Part IV focuses on examining health-harming legal, legal challenges affecting patient-clients, barriers to accessing legal services, and strategies for engaging PLWH. Part V consists of assessing MLPs level of readiness to adopt, implement and sustain a potential MLP HIV specific model of practice.Overall, questions will assess best practices, identify new tools, and serve to develop effective MLP approaches to addressing disparities in HIV treatment and health outcomes. The survey instrument consists of five parts. The piloting of the above survey with attorneys, graduate students of health and social services, members of the Scientific Collaborative Board, suggest a completion time between 15 and 20 min. At the end of the survey all participants will be asked if they would be willing to participate in a follow-up research component .Using the quantitative data generated, we will conduct two primary analyses. The first level of analysis will focus on identifying what MLPs consider organizational best practices  for HIV treatment and care. To achieve this, we will rank MLPs based on their level of exposure to HIV patients and services provided to HIV patients (answers to questions in Part III) vs. the level of performance as determine by the NCMLP performance measures (Part II of the survey). Using this composite indicator of MLP performance on HIV services, we will conduct a series of basic monomial logistic regression analyses to determine differences between MLPs with high and low performance .1: MLP service structural variables will not be associated with the likelihood of patients with at least one health-harming legal need who were treated by the healthcare organization; H2: Providers’ characteristics will not be associated with the likelihood of patients with at least one health-harming legal need who were treated by the healthcare organization; and, H3: The frequency of health-harming legal needs will be lower among providers with high exposure to HIV positive clients than their counterparts with lower exposure. If the evidence supports hypotheses 1 and 2, there is no need for tailoring the intervention to reduce the likelihood of health-harming legal needs. If H1 and H2 are rejected, we will conduct further analyses to determine determinants within MLP structure and provider characteristics that may reduce the likelihood of health-harming legal needs. The findings from testing H3 will provide insights into the relevance of exposure to HIV positive cases during the training of MLP providers as part of the development of the HIV MLP diffusion model.The second level of analysis will focus on proximal and distal determinants of health-harming needs. We will also conduct exploratory logistic and linear regression modeling, after assessing that that the appropriate assumptions are met, to test the following hypotheses: Hn = 25); medical and clinical providers (n = 25); (c) legal services providers; and, (d) administrators (n = 25). This sampling approach will allow us to have equal representation of the sectors within MLP practices. A sample size of 100 will allow us to conduct the above proposed exploratory analyses between continuous variables, where there should be at least 10 observations per variable; thus, if conducting an analysis of four independent variables, there should be a minimum sample size of 40. Moreover, before conducting the above exploratory regression analyses, we will examine the variance in each variable to make sure that the basic assumptions for regression analysis are met. We will use the Bonferroni correction, a multiple-comparison correction used when several dependent or independent statistical tests are being performed simultaneously to adjust the alpha coefficient to minimize Type 1 error in our analyses. Because of the exploratory nature of the proposed protocol, it was not appropriate to conduct power sampling calculations. We will conduct post-hoc power calculations to determine the likelihood of Type 2 error to our analyses.We will recruit a sample of 100 MLP providers divided into four categories by type of function within MLPs: (a) health and social services providers  who complete the online survey under Component #2, based on the answers to the last question in the survey. After consent procedures, MLP providers will participate in the in-depth interview through phone, on-line visual-teleconferencing, and/or in person. MLP providers will recruit PLWH clients into the study by providing the potential participants with a unique ID number . Fifty PLWH-MLP clients  will participate in in-depth interviews after completing consent procedures.n = 5), (b) elderly, 60 years and older, sexual minorities (n = 5), (c) young and adult sexual minorities (n = 5), (d) injecting substance users (n = 5), (e) non-injecting substance users (n = 5), (f) transgender young and adult men and women (n = 5), (g) individuals involved in sex worker (n = 5), (h) recent (documented or undocumented) migrants (n = 5), (i) ethnic minority individuals (n = 5), and, (j) dual diagnosis with HIV and another chronic illness or mental health condition (n = 5).We are seeking to recruit a diverse sample of PLWH in terms of sex, age, race/ethnicity, risk to sexual health, and representativeness of a larger population of HIV positive individuals. To be eligible PLWH have to meet the following criteria: 1) Have received MLP services in the past year by MLP provider who participated in the dyad interview; 2) Self-reported being HIV-positive when receiving MLP services; 3) Age 18 or older; 4) Able to speak English and/or Spanish. To achieve a diverse sample of PLWH, we will engage MLP providers in following a theoretically relevant quota sampling frame to select participants , 33. We Qualitative impacts of MLPs on HIV care will focus on identifying illustrative case studies, briefly defined as cases within the dyads narratives that represent the most typical and most deviant, as defined by Yin (2017) within the sample demonstrating the range of variations in MLPs addressing the legal issues through clients’ satisfaction and case outcomes [outcomes .Concordant-discordant perspectives on reach and effectiveness of MLPs for HIV care will focus on examining each dyad answer, and classify the answers in terms of concordance to determine the level of agreement on MLP effects in increasing positive movement in the HIV continuum of care [ of care  among PLConfirming and disconfirming cases – cross validation analysis will allow us to detect MLP cases that demonstrate and support findings and inferences from Components 1 and 2 on the structure and MLP practices, while simultaneously actively searching in the qualitative data cases that contradict, disconfirm and/or provide alternative explanations for researchers’ working inferences on the quantitative analysis under Component 2 [ponent 2 , 37.Recommendations for HIV MLP diffusion model. We will conduct content analysis on domains #4–6  Information process exchange; 5) Provider-patient communication: shortcomings and successes; and, 6) Current HIV strategies at MLP), to specify recurrent themes, lessons learned (dos and don’ts), determinants of behavioral change and concrete strategies used and/or experienced by participants to promote positive movement in the HIV continuum of care. The content analysis technique has been used before for identifying different elements of programs and interventions [ventions . These wInterviews will be transcribed (and translated into English for Spanish speaking interviews) and entered into Dboose, a cloud-based software package specifically designed to handle textual data and its analysis. A codebook will be constructed by two independent coders, including coding families based on the above 6 primary topics. We will take a sample of 10 transcripts, which will be coded collectively by two independent coders, members of the research team, and a research assistant. This will serve to refine the codebook. From this coding exercise, a thematic matrix will be developed. We will conduct 4 analyses using the dyads interviews:In order to conduct the above analyses, the qualitative data will be recoded according to each of the four analyses, thus expanding the original thematic matrix. This second level of advanced coding will be conducted by the investigative team.The institutional case studies and mixed methods data generated through components 1 to 3 will serve as the evidentiary foundation for the design of the MLP intervention. The process of designing interventions is as important and must be as rigorous as the evidence collected. For this reason, we selected using the intervention mapping approach . In a reIn general terms, intervention mapping consists of using an iterative path from problem identification to problem solving or mitigation. Each of the six steps of intervention mapping comprises several tasks each of which integrates theory and evidence. The completion of all of the steps will serve as the blueprint for the HIV-MLP model based on a foundation of theoretical, empirical and practical information.Table The intervention mapping steps in Table Intervention mapping has been a successful implementation science methodology for the development of in health-risk reduction and health promoting behaviors. Most of the interventions designed using intervention mapping range in levels of intervention from individual, families, and practitioners to organizational and community levels practices; and in types of conditions ranging from prevention of communicable diseases and early detection of chronic conditions to behavioral management of long-term chronic conditions , 22–25. Because of prior studies design limitations, prior assessments of MLPs cannot isolate the specific effects of MLP services on health outcomes. Most published MLP studies do not measure patient health outcomes or specify the causal connections between MLP and observed effects. To our knowledge, this study is the first U.S. federally funded study on medical legal partnerships. Our methodology is one of the first to empirically document best practices for delivering MLP services to PLWH, outcomes in the HIV care continuum and related psychosocial issues. Our study methodology considers the legal continuum as key component in the delivery of structural interventions. By developing an MLP HIV specific diffusion model, this study will contribute towards the field of HIV structural interventions. The findings from this study will provide insights into the relationships among legal challenges, legal involvement and HIV outcomes for PLWH. The lack of studies of the impact of medical legal partnerships on PLWH, despite their increased use, gives our methodology the potential to be of high public health impact.Additional file 1. Data collection instrument.


A systematic method was used to review the existing epidemiologic literature and determine the state of the scientific evidence for potential adverse health outcomes in populations living near oil and natural gas (ONG) operations in the United States. The review utilized adapted systematic review frameworks from the medical and environmental health fields, such as Grading of Recommendations, Assessment, Development and Evaluations (GRADE), the Navigation Guide, and guidance from the National Toxicology Program’s Office of Health Assessment and Translation (OHAT). The review included 20 epidemiologic studies, with 32 different health outcomes. Studies of populations living near ONG operations provide limited evidence  of harmful health effects including asthma exacerbations and various self-reported symptoms. Study quality has improved over time and the highest rated studies within this assessment have primarily focused on birth outcomes. Additional high-quality studies are needed to confirm or dispute these correlations. The United States has significantly increased its capacity for oil and natural gas (ONG) development through the technological advancements of directional drilling and hydraulic fracturing, with natural gas production reaching a high in 2017 and 2018 .There currently exists limited research and conflicting scientific information on the health risks for those living next to these operations. The industry surrounding ONG expanded faster than evidence-based epidemiologic research could respond ,5. EarlyOur study is the first of its kind to systematically review the entirety of existing epidemiologic literature on the associations between living near ONG development and the potential for harmful health effects. We weigh the level of evidence for each health outcome and aim to present a clear assessment of the methodological rigor, study strengths, and weaknesses, to identify approaches to future research. The scholarship published to date varies in the types of ONG operations studied, the populations of interest , the health outcomes measured, and the quality of the methods used. While Saunders and colleagues do raise important methodological concerns about many of the articles they review , no exisThe need for public health scientists to systematically evaluate the body of a literature base for an important issue, with limited resources, is necessary to assist in science-based regulatory decision making. Often, these issues are not entirely characterized and may include multiple chemical stressors  and variable health outcomes. The current established systematic review frameworks focus on an in-depth evaluation of the toxicological and epidemiological literature for a specific chemical and/or health outcome, however, this approach is unable to be applied directly to the epidemiological literature surrounding ONG development. Therefore, we have adapted these approaches to better answer this environmental health question.The steps used to conduct the review were adapted from various established systematic review frameworks for the medical and public health fields, including as Grading of Recommendations, Assessment, Development and Evaluations (GRADE)  and MetaThe scope of this literature review is defined by a PECO  question : “In humThe PECO question informed our exclusion criteria and studies were excluded if one or more of the following five criteria were met: (1) exposure to ONG chemicals was not directly measured in, or estimated for, study subjects ; (2) the study failed to quantify associations between exposures and a specific health outcome ; (3) the study did not include original data or observations ; (4) the study did not define ONG operations to include any or all processes associated with the upstream development and production of ONG, including but not limited to horizontal drilling and hydraulic fracturing; or (5) the study did not take place in the U.S.PubMed was the primary research database used to obtain articles. We identified relevant records using the following PubMed search terms:  AND (epidemiolog* or symptom*)) OR  AND  AND  NOT  AND (“2013/01/01”[PDAT]: “2018/10/01”[PDAT])) AND Humans[Mesh]. We verified that no relevant study was published before 2013, and any studies published after our search date of October 1, 2018 were not included in the assessment. In total, 1253 articles were returned by the search and all were screened for eligibility . Review A modified systematic review framework was used to rate the level of certainty (or the certainty in an estimate of effect) for each health outcome . We deveWe rated study findings as having low, moderate, or high certainty that the estimated effect was close to that of the true effect. The findings of observational epidemiologic studies were initially ranked as low certainty and were upgraded according to fourteen (14) study evaluation questions that assessed various domains . These cWe derived weight-of-evidence conclusions using standards outlined in GRADE , the CocTwenty (20) studies met our criteria of a human health epidemiologic study evaluating the potential health effects associated with living near ONG operations in the United States . Weight-Across all health outcomes, four of the 20 studies received a moderate level of certainty rating. All others received a rating of low certainty. The majority of the studies were retrospective cohort (six studies) or ecological (six studies) study designs. There were five cross sectional studies, two nested case controls, and two case-controls. The average score across all studies was 6, with a score range from 2 to 9 .This review identified nine studies comprising 12 low to moderate certainty findings that identified the relationship between women who lived near ONG operations and the likelihood that their child was born with birth defects or other types of adverse health outcomes at birth. Two studies evaluated birth defects  in infants of mothers who lived at varying proximities to ONG development during pregnancy ,33. ThesEight studies evaluated adverse birth outcomes ,38,39,40Birth outcomes have received the most scholarly attention for this topic, due to the relatively easy access to birth certificate or birth health records data, and the ability to pinpoint exposures to ONG operations during the 40-week gestation period . While t10) during upstream development activities. Some of these VOCs have the potential to cause developmental effects in test animals following high levels of exposure—generally at much higher levels than what has been observed for individual VOCs at ONG operations [ONG operations can emit volatile organic compounds (VOCs) into the air and contribute to increased particulate matter 10 micrometers or less in diameter  and found conflicting evidence for both categories. The two hospitalization studies used ecological study design, which is limited since the estimation of exposure is based on an average in the population. The three other studies documented self-reported symptoms. Health outcomes were not determined by a medical provider.We identified four studies that assessed the relationship between living near ONG development and the likelihood of neurological health effects ,53,55,57We found limited evidence of a positive association between general multiple self-reported symptoms and living near ONG development, with two studies assessing this relationship ,57. The Two epidemiologic studies evaluated a variety of indicators of psychological well-being, including depression, anxiety and sleep disturbances ,61. MeasThere was mixed evidence for self-reported dermal symptoms, self-reported psychological symptoms, and cardiovascular hospitalizations. Other health effects, including neurological and all hospitalizations, diagnosed sleep disturbances, and self-reported cardiovascular symptoms, had insufficient evidence due to a single low-rated study per outcome. There was a demonstrated lack of evidence (no association) for gastrointestinal self-reported symptoms. Three studies evaluated self-reported dermal symptoms, such as rash, irritation, burning, itching, and hair loss, in relation to ONG in Pennsylvania, resulting in mixed evidence ,55,61. SIn this paper, we summarized the observational epidemiologic literature on the health effects of populations living near ONG operations and assessed the methodological rigor of the studies published to date. Specifically, we used a modified systematic review framework, adapted from GRADE, the Navigation Guide, and guidance from OHAT, to determine the level of certainty that the study findings represent the true effect of exposures to ONG-related substances, and to make overarching weight-of-evidence determinations for a variety of health outcomes.The strength of our review lies in its transparency and objectivity. We adapted previous systematic review guidelines to make the criteria for evaluating studies as clear as possible. We considered a wide variety of study evaluation questions to represent those domains. Our review framework can also be applied to other research questions in environmental health. For researchers, policymakers, and public health practitioners, this type of review can swiftly help elucidate key findings and gaps in the knowledge base that need to be addressed.We found 20 published epidemiologic studies that evaluate potential associations between ONG operations and health outcomes. These studies assessed 32 different health outcomes ranging from self-reported symptoms to confirmed disease diagnoses. Since only a few outcomes were covered by multiple studies, there was insufficient weight of evidence for most health outcomes. We found studies of populations living near ONG operations provide limited evidence  of harmful health effects including asthma exacerbations and various self-reported symptoms. For all other health outcomes, we found conflicting evidence (mixed), insufficient evidence, or in some cases, a lack of evidence of the possibility for harmful health effects.There are important limitations to our approach. First, it is not a meta-analysis as the current line of inquiry, including different exposure measures (and surrogates), health outcomes, and geographic/geologic locations, is not suited to conducting a meta-analysis. Second, although we clearly stated our criteria for upgrading a study to a moderate or high level of certainty ranking, the number of study evaluation questions and the ranking cutoffs may still be viewed as arbitrary since Rooney et al. (2016) compares these systematic review methods and notes that the scoring of studies may be influenced by the number of elements and may not account for the differences in relative importance across the risk of bias domains . Study cThe majority of findings from the studies were ranked as low certainty, primarily due to limitations of the study designs that make it difficult to establish clear links between exposures to substances potentially emitted directly from ONG operations and the health outcomes evaluated. These limitations are inherent to observational epidemiologic studies and include indirect exposure measurements, confounding bias, and subjective methods to determine health outcomes. The field of environmental health incorporates these types of studies along with exposure and risk assessments to inform public health and policies. In addition to these factors, differences in the observational epidemiologic study types  make it difficult to compare results across studies with various health outcomes. These epidemiologic studies may also reflect the interactions of non-chemical or chemical stressors that may or may not be related to ONG operations that can contribute to adverse health outcomes in a population. Study quality has improved in recent years with better exposure measures and more thorough methods to account for possible confounders.Although these observational epidemiologic studies alone are not sufficient to determine causality, they provide helpful information to direct further investigation into the public health implications of ONG activity near residential areas. Taken together, these studies make it clear that the identities and exposure levels of substances people are exposed to when living, working, or going to school near ONG development have not been well characterized. Epidemiologic studies that include more controlled designs with direct measurement of exposure and diagnosed health outcomes are needed to confirm or dispute the associations published in the literature. Incorporating a health impact assessment framework within an epidemiologic study may be useful. One such framework, developed by the Agency for Toxic Substances and Disease Registry (ATSDR) can be used to assess the health impacts of multiple chemicals and stressors .Additionally, we have little empirically driven understanding of the factors  that drive ONG-related exposure patterns and vulnerability to such exposures. For example, there may be regional differences across the U.S., with varying technological controls or regulatory environments. Researchers should integrate community members ,69,70 anONG regulatory policy has not been informed by robust epidemiologic research literature. Now, 15–20 years since the widespread application of hydraulic fracturing and horizontal drilling in states as diverse as Colorado, Pennsylvania, Texas, and Kansas, the epidemiologic literature on the potential health effects of ONG operations is still inadequate to definitively guide policy, as evidenced by the mainly low certainty and conflicting studies reviewed here. Regulators and policymakers, then, should work with public health researchers to pose specific questions that need to be answered, and partner with public health officials to evaluate the public’s concerns. Public health officials should continue to monitor health concerns in areas with substantial ONG operations through centralized data collection and analysis. Multi-state collaborations should be considered to collect consistent data from differing oil and gas basins across the United States with the aim to more comprehensively evaluate the potential for adverse health effects.


Purpose: This article outlines the process of establishing the Trans Youth Care Research Network, composed of four academic clinics providing care for transgender and gender-diverse (TGD) youth. The Network was formed to design and implement research studies to better understand physiologic and psychosocial outcomes of gender-affirming medical care among TGD youth.Methods: Formed in response to both the Institute of Medicine's report recommendation for an increase of data concerning sexual and gender minority populations and a transgender-specific NIH program announcement, The Center for Transyouth Health and Development at Children's Hospital Los Angeles, the Gender Management Service at Boston Children's Hospital, the Child and Adolescent Gender Center Clinic at Benioff Children's Hospital in San Francisco, and the Gender and Sex Development Program at Lurie Children's Hospital of Chicago established a collaborative research network that subsequently designed a longitudinal observational study of TGD youth undergoing medical interventions to address gender dysphoria.Results: Two cohorts, youth starting puberty blockers and youth starting gender-affirming hormones, are participating. Psychosocial measures that span multiple domains of mental and behavioral health are collected from youth and parents. Physiologic data are abstracted from patient's charts. Baseline and follow-up data of this large cohort will be disseminated through conferences, abstracts, posters, and articles.Conclusion: Since initiation of funding in 2015, a total of 497 participants have been enrolled in TYC across the four sites; gonadotropin releasing hormone analogs (GnRHa) cohort youth (n=93), GnRHa cohort parents (n=93), and gender affirming hormone cohort youth (n=311). As the network moves toward data dissemination, its lessons learned have helped strengthen the current study, as well as inform future endeavors in this field. Within this cultural context, where there are few-to-no road maps for TGD youth to follow in exploring their gender, TGD youth are challenged with an additional burden: discovery, disclosure, and stigma management of a gender different from what has been assumed based on their sex designated at birth. At odds with the social construct in which they are immersed, TGD youth may experience discomfort, stress, and anguish at both the societal/distal and individual/proximal levels.3 Many carry diagnoses of attention deficit-hyperactivity disorder, obsessive–compulsive disorder, eating disorders, and others.3–6 Compared to cisgender peers, TGD youth engage in higher rates of self-harm, alcohol, and drug use, and more often have suicidal thoughts and attempts.5–7Distal stressors may include harassment, bullying, violence, loss of partner, family members, or community. These distal stressors are theorized to interact and bidirectionally influence individual/proximal stressors, such as internalized transphobia and negative expectations about the future, which in turn contribute to psychological distress. TGD youth experience high rates of anxiety and depression symptoms compared to cisgender adolescents, including sexually minoritized youth.4 Results from a prospective study by Olson et al. reported that among 101 youth, depression scores were mild-to-moderate in 24% of participants and severe in 11%. Fifty-one percent reported contemplating suicide, and 30% had a history of at least one suicide attempt.5In 2012, Spack et al. documented that among 97 youth, 20.6% had a history of self-injurious behavior, 9.3% a prior psychiatric hospitalization, and 9.3% a history of 1 or more suicide attempts.8 Edwards-Leeper et al. found that a subset of adolescents seen in a transgender clinic seeking medical intervention appeared to be well adjusted based on standardized measures of mental health.9 Paramount to optimizing patient care is understanding the factors that play a role in the mental health of youth with gender dysphoria.Only a handful of studies have documented baseline well-being among TGD youth. Olson et al. found that socially-transitioned prepubescent gender-diverse children 12 years and younger were psychologically well adjusted compared to cisgender peers and siblings.10 This growth is likely the result of a convergence of events, including the building of an accessible transgender community through the Internet, increased visibility of transgender narratives in media, and finally, increased availability of medical interventions. The broader visibility of TGD youth has opened the closet doors for many who would have previously remained nondisclosed about their gender for years, possibly even an entire lifetime.Over the past decade, a groundswell has occurred in the number of TGD adolescents seeking care at gender-specialized clinics throughout the United States and Western Europe.12 only a handful of studies focused on the impact of hormonal intervention on youth with gender dysphoria, particularly minors.13–19 As a result, there is lack of consensus among professionals around timing of initiation of medical interventions, as well as optimal dosing regimens.Although transgender adults frequently report first recognizing their gender identities during childhood and adolescence, there is a consensus gap about the best approach to the care of youth with gender dysphoria. While a fair amount of extant literature supports the safety, positive impact, and importance of medical intervention for transgender adults,Vrij Universiteit Medical Center in Amsterdam disseminated the “Dutch Model” of care, outlining a protocol, including the use of puberty blockers to suppress endogenous puberty in youth with gender dysphoria starting at age 12, followed by the addition of estrogen or testosterone for induction of appropriate puberty starting at age 16.15 Follow-up data of 70 youth between the ages of 11 and 17 years initiating endogenous puberty suppression reported on mental health measures before initiation of blockers and then again before initiation of gender affirming hormones. Findings indicated a decrease in behavioral and emotional problems, a decrease in depressive symptoms, and improved general functioning for youth with gender dysphoria treated with puberty blockers.16The largest body of published data about transgender youth was derived from samples of Dutch youth. In 2006, the 17 While these findings are promising, the cohort is small, ethnically homogenous, and not necessarily generalizable to youth with gender dysphoria in the United States. To inform best clinical practices, the need for ongoing prospective and longitudinal data collection is clear.A follow-up study from the Dutch team examining the impact of puberty suppression followed by gender affirming hormones and gender reassignment surgery in 55 of the same sample (now in young adulthood) showed alleviation of gender dysphoria, steady improvement of psychological functioning, and a sense of “well-being” that was equivalent or superior to age-matched controls in the general population.20 Following the IOM report, the National Institutes of Health (NIH) released a program announcement aimed at better understanding the health of TGD populations in 2012, PA-12-111. For the first time in history, U.S. federal resources were allocated to understand and improve the lives of gender minoritized populations unrelated to HIV.In 2011, the Institute of Medicine's (IOM) Report on the health of lesbian, gay, bisexual, and transgender populations called for increased investigation to better understand disparities within the lives of sexual and gender minoritized individuals, with specific discussion of transgender youth.The Center for Transyouth Health and Development at Children's Hospital Los Angeles, the Gender Management Service at Boston Children's Hospital, the Child and Adolescent Gender Center Clinic at Benioff Children's Hospital in San Francisco, and the Gender and Sex Development Program at Lurie Children's Hospital of Chicago had been informally collaborating for several years before the release of the NIH program announcement. In response to the program announcement, these four centers agreed to work as a network and began formulating a study aimed at answering some of the most pressing questions about the care of TGD youth. All four sites, each possessing significant clinical experience providing medical care for youth with gender dysphoria, were well positioned to provide optimal environments for recruitment and retention of ethnically, geographically, and developmentally-diverse participants across a wide age span.The four principal investigators (PIs) are two pediatric endocrinologists and two adolescent medicine subspecialists. All four sites have strong behavioral health professionals and psychologists on their teams, who serve as coinvestigators (Co-Is), working with PIs in conceptualizing research questions related to mental health outcomes, developing appropriate methodological strategies, choosing measures, and developing data analysis protocols for the planned study. The Network was formed with the purpose of designing and implementing research studies to better understand how medical interventions, social transition, youth resiliency, parental support, and affirmative environments affect the experiences of TGD youth.22 The use of GnRHa for youth beyond the early stages of puberty is also common, particularly in combination with gender affirming hormones. Youth experiencing gender dysphoria in their later or final stages of endogenous puberty may be appropriate for administration of gender affirming hormones to induce masculinizing or feminizing features.Recommendations from the World Professional Association of Transgender Health, the Endocrine Society, and cosponsoring organizations note that youth experiencing gender dysphoria in the early stages of puberty (Tanner stage 2–3) may benefit from the administration of gonadotropin releasing hormone analogs (GnRHa) to suppress further endogenous pubertal development.The Trans Youth Care (TYC) Study was specifically designed to understand the impact of early medical treatment within two developmental cohorts: youth in early puberty with gender dysphoria seeking suppression of their endogenous puberty, and older adolescents and young adults with gender dysphoria seeking initiation of gender affirming hormones. Collecting data from each cohort over time is essential for understanding the long-term health and well-being of youth; hence, we used a longitudinal study design, expanding across 24 months of inquiry at 6-month intervals.In a relatively emergent and fledgling field, prioritizing research questions is exceptionally challenging. Gathering as much data as possible must be balanced with the potential for overburdening participants with voluminous inquiry. This team was interested in the physiological outcomes of medical intervention, as well as understanding the social context of culture; family and peers; stigma and support; resilience; gender roles and expression; relationships and self-identity; traumatic experiences; mental health, including overlap of potentially co-occurring conditions such as addiction; obsessive–compulsive disorder; post-traumatic stress disorder; or developmental differences such as autism spectrum disorder.18 the first version of the Endocrine Society Guidelines (2009) recommended that gender affirming hormones be started in youth with gender dysphoria around the age of 16 years.22 The newest version of the Endocrine Society guidelines, published in 2017, acknowledges that there are compelling reasons to initiate gender affirming hormone treatment in youth with gender dysphoria younger than 16 years of age.23 Since this new recommendation, only a handful of studies have examined the physiologic impact of hormone use among younger adolescents.14In light of existing data that underscore increased well-being among adolescents who initiate gender affirming hormones at younger ages,11 While thrombotic events, hypertension, and lipid profile changes are sequelae exacerbated by aging (meaning fewer incidents will likely occur within our younger population), our team was interested in understanding whether youth had a similar or higher risk for these side effects based on introduction of exogenous hormones in adolescence.In contrast, the safety and impact of gender-affirming hormones used in adult TGD patients undergoing medical transition have been written about comprehensively and have demonstrated that hormones are relatively safe and result in a low risk profile for patients. The potential sequelae of gender-affirming hormone use in adults have been outlined, including risk for thrombotic events and changes in blood pressure, hemoglobin and hormone levels, and lipid profiles.24 their impact on bone health is one of the most important aspects to understand to optimize treatment protocols. In this study, physiologic data being collected include bone density, calcium intake, and weight-bearing exercise among participants to more fully understand the role of GnRHa within a wider context of bone health. A comprehensive list of metabolic, anthropometric, and bone health measures used is listed in While preliminary data concerning the impact of GnRHa have been published,The network hypothesized, from clinical experience, that many TGD youth carry historic and current mental health diagnoses due to the presence of symptoms that may diminish with gender-affirming medical intervention and an affirmative environment. To test this hypothesis, a protocol was developed to collect diagnostic information established before entry into care, along with diagnostic interviews conducted at baseline and at follow-up time points.25 To decrease participant discomfort and potentially preserve data integrity, the team chose to replace the DISC with the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI Kid and MINI), a shorter diagnostic tool that was not reported as distressful by participants.26 The team also opted to include symptom checklists to assess broadband internalizing and externalizing conditions, anxiety, depression, autism features, resilience, quality of life, body esteem, and suicidality. We hope that these symptom checklists will adequately discriminate nongender dysphoria-related, stand-alone mental health diagnoses  versus “mental health” symptoms that are secondary to gender dysphoria that we would expect to improve following medical treatment. Comprehensive lists of the mental health measures are listed in 27Initially, the team chose the Diagnostic Interview Schedule for Children (DISC), a validated, highly structured interview that assesses the presence of more than 30 mental health diagnoses.28 In those cases when we utilize scales scored by gender, we are choosing to score participants according to both male and female parameters as suggested by some.28 The existing measures of gender dysphoria are inadequate on multiple levels.Existing measures of depression, anxiety, coping, resilience, suicidality, and self-harm are not specifically validated for TGD youth. In addition, there are no recommendations for scoring those tools  that are normed by gender.29 The UGDS is static in nature, capturing elements of an individual's physical experience and not likely to capture change over time. For example, in the scale designed to be administered to those designated female at birth, the item, “I hate having a menstrual cycle,” will likely continue to receive affirmative responses from those who identify as male or a gender other than female, even if amenorrhea is achieved through medical intervention.The Utrecht Gender Dysphoria Scale (UGDS) is a short survey with questions regarding the distress persons feel when confronted in daily life with the fact that the gender they are living in that aligns with the sex designated at birth is incongruent with their own inner feelings about their gender.UGDS items that inquire about sexuality such as, “I like to behave sexually like a girl,” posit significant generalities about sexual behavior and assume that such behaviors are inherently gendered. The UGDS is binary in nature, thereby rendering it inadequate for capturing information about youth with nonbinary gender identities .30The necessity for an improved measure to capture the nuanced elements of gender dysphoria and its potential for intensification or mitigation over time has been highlighted by our transgender team members who have been on the front line with youth participating in the study. After significant deliberation, we chose to include the UGDS in this study, in the hopes that we might demonstrate its limitations in capturing the dynamic experience of youth with gender dysphoria. For similar reasons, we also included The Gender Identity/Gender Dysphoria Questionnaire for Adolescents and Adults (GIGDQAA).31 and the Gender Minority Stress and Resilience Scale32 are likely the best existing measures to collect information about both distal and proximal drivers of gender dysphoria.Over the course of the past 2 years, there were concerns articulated by participants regarding distress they experience when confronted with some of the items from both of these instruments. A decision was made to remove the UGDS and GIGDQAA from the participant assessments, except for those participants completing a substudy to obtain participant feedback regarding items included in those two scales. Our team felt that the Transgender Congruence ScaleWhile guidelines exist that provide recommendations for treatment timelines and protocols, a lack of treatment outcome data yields significant room for interpretation of such recommendations. All four gender clinics include medical professionals from the subspecialties of either endocrinology or adolescent medicine, nursing, psychiatry, psychology, and social work.The four sites in this study practice differently with respect to psychological assessment of youth before medical intervention, intake procedures, types of puberty blocking medications, hormones used for feminization, timing of initiation of medical interventions, dosing regimens, as well as medication monitoring and adjustment. No evidence outlining the comparative benefits or risks among different practice models exists, and considerable debate persists within professional communities about best practices.The differences among sites provide an opportunity to study differential effects of these care models; conversely, the differences in practices may result in variability of results that can pose challenges in drawing general conclusions about elements of care shared between all sites.n=93), GnRHa cohort parents (n=93), and GAH cohort youth (n=311).Since initiation of funding in 2015, a total of 497 participants have been enrolled in TYC across the four sites; GnRHa cohort youth  has increasingly been considered an important strategy for eliminating racial and ethnic health disparities by engaging community members as partners in research design, collaborative discourse about knowledge, and in intervention development and health policy-making.The voices of our transgender team members cannot represent the entire community, but they bring unique perspective and input to the process of identifying appropriate measures, decision-making around analysis and interpretation of measures that are scored based on gender, and provide important critiques regarding the value of certain aspects of the study to the community itself. Ongoing meetings with input from transgender team members have informed the network's consideration of CBPR methods to use as study aims evolve over the course of investigating these two developmental cohorts of youth.As our network moves toward data dissemination, our lessons learned have helped strengthen our current study, as well as inform future endeavors. Perhaps most important is the greater understanding that transgender identity and gender dysphoria are complicated, multifactorial, and extraordinarily nuanced human experiences.While some common features exist among transgender youth, the developmental trajectories and needs are unique to each individual and are impacted by innumerable factors, including physiology, psychology, temperament, genetics, familial structure, sociocultural factors, geography, religion, ideology, exposure, politics, and others. In addition, provider-related factors, including personal beliefs, religion, culture, ideology, exposure, and knowledge, will also lead to differences in how care is approached and practiced. Finally, commitment to community engagement for the ongoing investigation will infuse the work with the critical perspective of the community the study seeks ultimately to serve.