In this video Mello et al. , demonstThe surgical team carefully diagnosed and staged the ureteral tumor. A formal cystoscopy, bladder washing cytologies, retrograde pyelogram and ureteroscopy was performed to verify that this tumor was localized, single and that there were no other concomitant pathology such as carcinoma in situ of the upper collecting system.Nephroureterectomy remains the gold standard for the surgical management of upper tract transitional cell carcinoma; However, recently, minimally invasive approaches including endourological (for non invasive disease) as well as laparoscopic or robotic assisted for segmental resections of the distal ureter with ureteral reimplantation has been reported with great oncological and clinical outcomes (for non invasive and invasive disease).The authors clearly follow oncological principles of resection and anastomosis of clean ureteral margins. They demonstrate the importance of clipping the ureter above and below the tumor with a clear margin of resection. This will prevent spillage of neoplastic cells into the peritoneal cavity, preventing peritoneal carcinomatosis. In this case, the ureters were anastomosed without tension and the patient profitted from a less morbid approach to deal with a single ureteral tumor.The final pathology revealed a pT3 high grade urothelial carcinoma with associated carcinoma in situ. The prognosis of these cases is poor, but the option of local control in a patient with high medical co-morbidities and a solitary kidney is clearly evident. Finally, we congratulate the authors for such a great demonstration of a Robotic assisted segmental resection of the ureter for a clinically localized ureteral tumorAlejandro R. Rodriguez, MDDepartment of Urology and Urology Oncology,Director of Robotics and Minimally Invasive SurgerySamaritan Medical Center, Watertown, New York, USA830 Washington St, Watertown, NY 13601, USATelephone: +1 315 785-4000E-mail: armbkdd@yahoo.com


The introduction of the national health insurance scheme (NHIS) in Ghana in 2003 significantly contributed to improved health services utilization and health outcomes. However, stagnating active membership, reports of poor quality health care rendered to NHIS-insured clients and cost escalations have raised concerns on the operational and financial sustainability of the scheme. This paper reviewed peer reviewed articles and grey literature on the sustainability challenges and prospects of the NHIS in Ghana.Cochrane, PubMed, ScienceDirect and Googlescholar for initial screening. Out of this number, 31 eligible peer reviewed articles were selected for final review based on specific relevance to the Ghanaian context.Electronic search was done for literature published between 2003–2016 on the NHIS and its sustainability in Ghana. A total of 66 publications relevant to health insurance in Ghana and other developing countries were retrieved from Ability of the NHIS to continue its operations in Ghana is threatened financially and operationally by factors such as: cost escalation, possible political interference, inadequate technical capacity, spatial distribution of health facilities and health workers, inadequate monitoring mechanisms, broad benefits package, large exemption groups, inadequate client education, and limited community engagement. Moreover, poor quality care in NHIS-accredited health facilities potentially reduces clients’ trust in the scheme and consequently decreases (re)enrolment rates. These sustainability challenges were reviewed and discussed in this paper.The NHIS continues to play a critical role towards attaining universal health coverage in Ghana albeit confronted by challenges that could potentially collapse the scheme. Averting this possible predicament will largely depend on concerted efforts of key stakeholders such as health insurance managers, service providers, insurance subscribers, policy makers and political actors. Universal access to good quality health care remains a major concern of health systems globally. In view of this, countries have adopted different health financing mechanisms including National Health Insurance (NHI) to ensure universal access to quality basic health care. Ghana is one of the few countries in sub-Saharan Africa spending a relatively high percentage of its Gross Domestic Product (GDP) on health. Ghana’s total expenditure on health as a percentage of its GDP was 5.4% in 2013 compared to 3.9% in Nigeria, 4.5% in Kenya, and 4.6% in Benin [et al [et al [In the larger African context, a number of community-based health insurance systems have been implemented with varying outcomes. However, the operational and financial sustainability of these health insurance interventions has been a major challenge to many of these low and middle income countries in Africa, including Ghana. For instance, Kyomugisha et al  reportedl [et al  concludeSustainability of Ghana’s NHIS in this context refers to the scheme being able to operate successfully meeting its mandate without collapsing. A sustainable NHIS would mean the scheme overcoming challenges of limited funds for provider reimbursements and administrative activities; improving quality of health care and insurance services to enhance subscriber trust and increase utilization of NHIS services.Ghana was the first sub-Saharan African country to introduce NHIS in 2003 through an Act of parliament  and full implementation started in 2004 See . Under tThe NHIS is financed through a central National Health Insurance Fund (NHIF) which is sourced from the National Health Insurance Levy (NHIL) of 2.5% tax on selected goods and services; 2.5% of Social Security and National Insurance Trust (SSNIT) contributions, largely by formal sector workers; payment of premiums, and donor funds. Individuals who are employed in the formal sector and contribute to SSNIT are exempted from premium payment. As at 2012, over 70% of the NHIS financial inflows came from the NIL; 17.4% from SSNIT contributions and 4.5% from premium payments. Other sources of funding to the NHIF include money allocated by parliament of Ghana, grants, donations, gifts/voluntary contributions, and interests accrued from investments see . IntroduThe NHIS does not attempt to treat all diseases suffered by insured members. However, many common diseases such as malaria, upper respiratory tract infections and diarrheal diseases, are covered in the NHIS benefits list. Likewise, NHIS-insured members are entitled to medical emergency care such as road traffic accidents. Also, under the free NHIS for pregnant women policy, pregnant women are exempt from NHIS premium payment. Diseases such as cancers, which are relatively not common, are not covered by the NHIS.Overall, more than 60% of active members of the NHIS are under the premium exemption category . Critics of the Ghanaian NHIS have argued that the scheme is overly generous and financially unsustainable because of the huge percentage of NHIS members under premium exemption without co-payment  coupled with increasing cost of medical logistics/supplies and health service delivery.According to the reviewed literature, Ghana’s policy on free NHIS for pregnant women in 2008 contributed to improved maternal health care coverage ,10. DzapMensah et al  found inSome literature however argued that even though the NHIS contributed to increased utilization of formal health care services, the scheme did not necessarily improve the quality of health care in NHIS-accredited health facilities . QualityIn a Ghana Health Service (GHS) annual report , it was Besides the poor quality of health care services in NHIS-accredited health facilities, some studies have suggested client dissatisfaction with NHIS district offices in terms of delayed issuance of membership cards and inadequate education for subscribers on NHIS benefits package. According to a SEND-Ghana report in 2010, some registered NHIS members did not have access to health care services in the Upper West, Northern, Upper East and Greater Accra regions because of delayed issuance of membership cards by the NHIS district offices . In moreStatistics on the number of Ghanaians with active private health insurance or insurance by an employer were not available to the authors at the time of this review. As at 2016, only about 40% of the total Ghanaian population (estimated 26.9 million) is enrolled with the NHIS with valid membership cards See . The 40%This review explored peer reviewed articles and grey literature on the financial and operational sustainability threats to the NHIS after a decade of its implementation in Ghana. Proposed interventions towards averting a collapse of the NHIS were also reviewed.This review was based on the PRISMA guidelines for systematic reviews . ConformPubMed, ScienceDirect, GoogleScholar, Cochrane and bibliographies from publications on the NHIS in Ghana. Time frame for the search was between November, 2003 and August, 2016 since the NHIS was introduced in 2003. In addition, policy documents and reports on the NHIS were obtained from databases of the Ministry of Health (MoH), Ghana Health Service (GHS), National Health Insurance Authority (NHIA) and World Health Organization (WHO) websites.An electronic search was done with et al [The search did not include literature on health insurance in developed countries. Moreover, literature not published in English language was excluded. Articles using quantitative methodologies with sample size less than 100 were excluded because of generalizability challenges. Using the snowball approach as in Willis-Shattuck et al , the refThe reviewers first assessed grey literature on the NHIS to inform the development of the review background. To ensure that credible information is reviewed from the grey literature, a thorough exploration of only official websites was done to retrieve annual reports and relevant policy documents.The next level involved screening of abstracts and full text articles in peer reviewed journals. Specific inclusion criteria used were articles explicit on: (1) NHIS sustainability threats and prospects; (2) operational sustainability of the NHIS; (3) financial sustainability of the NHIS; (4) interventions to sustain the NHIS.All articles considered for inclusion were screened for quality by co-authors and peers who have worked substantially on the topic area. Screening for article quality was done based on rigor of methodology used in the article. Furthermore, the search process, data extraction and analysis were validated by the co-authors.et al [Data extraction was done using an Excel spreadsheet and analysis done in the narrative form since the retrieved literature is a mixture of peer reviewed articles and grey literature. This approach was used for both grey and peer reviewed literature in line with Mays et al  argumentet al [Based on Greenhalgh et al  format, Following the electronic search using relevant search engines, a total of 4,560 publications including grey literature were found on health insurance in Ghana and Africa. Out of the 4,560 articles, 2,734 were subsequently screened based on article title and abstracts. Next, 2,668 articles were excluded leaving 66 articles. The 66 remaining articles were further screened for eligibility in the final review and 35 articles excluded leaving 31. Finally, the 31 retained articles comprised of 27 peer reviewed articles and 4 systematic reviews. See The selected 31 publications included for the final review were made up of peer reviewed articles that used primary (n = 24) and secondary (n = 7) data. Out of the seven articles using secondary data, five were narrative reviews; one was a working paper, and one was a WHO policy brief. In terms of the country context, 90% of the 31 articles were solely on Ghana while the remaining 10% were comparative studies, comparing the NHIS in Ghana with health insurance systems in Nigeria, Tanzania and South Africa.In terms of methodology and study design, of the 24 articles that reported on primary data, 58% used quantitative approaches such as household surveys and exit interviews; 21% used only qualitative methods such as individual interviews and Focus Group Discussions (FGDs); 21% triangulated qualitative and quantitative techniques. Over 90% of the papers that used primary data also extensively reviewed relevant secondary data. The sample size of articles that used primary data ranged from a low of 28 personal interviews with six FGDs to a high of 120,000 respondents in a randomized control trial.As shown in Proposed interventions for a sustainable NHIS in Ghana were recommended in 71% of the 31 articles; 29% were not explicit on interventions to sustain the NHIS. Out of the 31 reviewed articles, 45% recommended supply-side or provider-centered strategies; 10% suggested mainly demand-side or client-centered approaches and the remaining 16% proposed supply and demand side interventions. The thematic areas that emerged from the 31 articles for synthesis and discussion are: perspectives on threats to NHIS sustainability in Ghana, and recommendations towards sustaining the NHIS. In all, 81% of the 31 papers included for final review reported on whether or not the NHIS in Ghana is sustainable. Fusheini et al  argued iet al [Fusheini et al  categoriet al [et al [Political threats according to Fusheini et al  include l [et al  are clieet al [Another important sustainability threat discussed in the literature is poor quality of health care services in NHIS-accredited facilities. Alhassan et al –24 alludet al  made simet al  found noet al [et al [et al [Agyepong and Nagai  attributet al  made siml [et al  and Wittl [et al  also citAs shown in Recommendations towards sustaining the NHIS and enhancing its prospects in Ghana were outlined in 71% of the 31 reviewed papers. In 45% of the papers, the recommendations were supply-side (provider-centered) while 10% of the papers proposed demand-side (client-centered) recommendations; five articles (16%) proposed both supply and demand-side interventions. Sourcing for extra funding for the NHIS through oil revenue, levies on large profitable companies and increasing Value Added Tax (VAT) levy were proposed to promote financial sustainability of the NHIS ,44.Other recommendations made were improving revenue collection, introduction of co-payment, and capitation. Apart from the capitation system, the reviewed papers also suggested the adoption of cost control mechanisms such as enforcement of the gatekeeper system; implementation of stringent monitoring mechanisms on health providers, and improving referral systems ,44. The Other proposed interventions to ensure operational sustainability of the NHIS include: improving geographical accessibility of accredited health facilities through infrastructural expansion; improving material and human resource capacity to reduce workloads in health facilities ,34. Someet al [Finally, suggestions were made on the need to de-couple politics from the routine management activities of the NHIS since political interferences could stifle the progress and sustainability of the NHIS ,28. Instet al .This review explored empirical and grey literature on sustainability of the NHIS in Ghana. Overall, the reviewed literature reported emerging challenges of the NHIS such as administrative constraints, poor quality of health care to NHIS-insured clients, political interference and limited revenue sources. These challenges were highlighted as potential sustainability threats to the NHIS.Different perspectives on the NHIS sustainability were synthesized into two broad categories: namely operational and financial sustainability. Operational sustainability was discussed in 42% of the reviewed papers. Financial sustainability was discussed in 10% of the papers see . This trEven though concerns have emerged on sustainability of the existing funding sources for the NHIS in Ghana ,42,44,45Another observation in this review is the percentage of papers (71%) that were explicit on proposed interventions to sustain the NHIS. This percentage suggests that there is sufficient scientific information available to policy makers and stakeholders of the NHIS to inform policy discussions on sustainability strategies for the NHIS. Relying on scientific data to inform policy decisions is a better alternative to anecdotal information .It was however observed that, many reviewed papers were not focused on demand-side (client-centered) sustainability strategies. Only 10% of the 31 reviewed papers see  carried m-Health initiatives in Rwanda [Client-centered interventions such as community participation in health have proved successful in mobile phone technology for health (MOTECH), and other n Rwanda , Nigerian Rwanda  and Ghann Rwanda ,52. ActiIt is recommended that, more prevalent scientific studies and reviews be conducted on client-centered quality services at various stages of implementation of the NHIS. Since there is growing recognition of the operational challenges confronting the NHIS in Ghana, adopting client-centered quality health care and health insurance services could enhance perceptions on the NHIS and increase client trust in the scheme.Key impediments towards attaining client-centered health care and health insurance services in Ghana include geographical inaccessibility to health facilities, limited information on benefits package, undue delays in accessing health services, perceived differential treatment for NHIS-insured and non-insured clients, informal fee charges, delayed issuance of membership cards and limited complaint avenues for aggrieved clients. These limitations have been discussed in detailed in the reviewed articles ,36,38,44In response to many of these challenges confronting the NHIS, the NHIA initiated the instant issuance of membership cards, biometric registration of members and establishment of Client Call Center to handle grievances of NHIS subscribers. Even though there are inadequate scientific impact evaluation studies on the effect of these interventions on quality service provision and decisions to (re)enroll in the NHIS, these interventions have improved client and health staff perceptions on the NHIS in some cases ,23,24.It is also important to emphasize the need to complement client-centered services with efforts to satisfy health care providers in the areas of early reimbursements, infrastructural improvement, improving staff numerical strengths and motivation levels, availability of drugs and capacity building for health workers. These areas have been cited as important supply-side interventions that could help ensure active stakeholder participation in the NHIS and curtail unprofessional conduct of health providers .Furthermore, to attain client-centered quality services at the levels of the NHIS and the health care providers, there is the need for strict monitoring and supervision by the national, regional and district NHIA offices. This will not only help identify genuine operational challenges but also detect illegal acts such as illegal charging of fees and other administrative lapses.All in all, the reviewed literature demonstrated that only few papers discussed client-centered interventions towards sustaining Ghana’s NHIS. In addition, even though many papers used triangulated methodologies, the emphasis was on quantitative data. Albeit large sample sizes were used in many articles, one common weakness was the lack of nationally representative sample sizes. Moreover, none of the studies did a detailed analysis of databases that explored perspectives of the three key stakeholders of the NHIS: client, provider and insurer. Discussions on the roles of the provider, client and insurer towards sustaining the NHIS were largely done in isolation, thus creating a gap that needs to be bridged in future scientific endeavors.Even though many articles examined operational sustainability of the NHIS, not much was discussed on the role of clients and communities in sustaining health insurance operations. This suggests the need to intensify efforts towards involving clients and community members in quality service monitoring and improvement. Existing platforms such as the Client Call Center instituted by the NHIA presents an opportunity to effectively involve clients in health insurance planning and implementation.Community engagement in NHIS activities is key to promoting client interest and compliance to health insurance policies. Studies by Alhassan et al ,24 foundThe current review is limited by the type of literature retrieved for review. First, the literature sources were all searched in the English language. It is possible relevant literature published in other languages were left out to the detriment of quality results and discussions.Secondly, due to limited relevant information in the topic area, grey literature was reviewed to complement findings of the peer reviewed articles. This might have inadvertently introduced reporting bias by the authors. Even though quality checks were done through credible websites and validation of findings by expert reviewers, reported information on grey literature could still be bias.Finally, the review is largely limited to the NHIS in Ghana and would be most appropriate in discussions pertaining to the Ghanaian context. Nonetheless, lessons could be drawn from the peculiar case of Ghana by other African countries which are at various stages of health insurance implementation. Future reviewers should extent their scope and do comparative reviews of health insurance sustainability in more than one African country.This paper reviewed relevant literature on sustainability challenges of the NHIS and suggested interventions to avert a collapse of the scheme. Reviewed articles generally acknowledged some sustainability threats confronting the scheme. Many of these articles discussed the administrative/operational sustainability threats while a few discussed the financial sustainability threats.Discussions in this paper are not exhaustive, instead the debate has been initiated on the need to prioritize client-centered quality service approach (i.e. customer care and client satisfaction strategies) at the health insurer and service provider levels. It is expected that complementing existing efforts with these strategies will contribute towards increasing confidence in the NHIS to promote membership participation and retention.S1 Table(DOC)Click here for additional data file.


Background: The Comprehensive Post-Acute Stroke Services (COMPASS) Study is one of the first large pragmatic randomized-controlled clinical trials using comparative effectiveness research methods, funded by the Patient-Centered Outcomes Research Institute. In the COMPASS Study, we compare the effectiveness of a patient-centered, transitional care intervention versus usual care for stroke patients discharged home from acute care. Outcomes include stroke patient post-discharge functional status and caregiver strain 90 days after discharge, and hospital readmissions. A central tenet of Patient-Centered Outcomes Research Institute-funded research is stakeholder engagement throughout the research process. However, evidence on how to successfully implement a pragmatic trial that changes systems of care in combination with robust stakeholder engagement is limited. This combination is not without challenges. Methods: We present our approach for broad-based stakeholder engagement in the context of a pragmatic trial with the participation of patients, caregivers, community stakeholders, including the North Carolina Stroke Care Collaborative hospital network, and policy makers. To maximize stakeholder engagement throughout the COMPASS Study, we employed a conceptual model with the following components: (1) Patient and Other Stakeholder Identification and Selection; (2) Patient and Other Stakeholder Involvement Across the Spectrum of Research Activities; (3) Dedicated Resources for Patient and Other Stakeholder Involvement; (4) Support for Patient and Other Stakeholder Engagement Through Organizational Processes; (5) Communication with Patients and Other Stakeholders; (6) Transparent Involvement Processes; (7) Tracking of Engagement; and (8) Evaluation of Engagement. Conclusion: In this paper, we describe how each component of the model is being implemented and how this approach addresses existing gaps in the literature on strategies for engaging stakeholders in meaningful and useful ways when conducting pragmatic trials. Engaging patients and other stakeholders in the continuum of biomedical research is a promising approach to creating more effective interventions and accelerating translation of effective interventions into practice –6. StakeIn parallel, comparative effectiveness research is the direct comparison of 2 or more interventions to determine which works best for which patients. When designed to evaluate the effectiveness of interventions in real-world practice with heterogeneous patient populations, the study is a pragmatic clinical trial .One of the first 5 large pragmatic clinical trials funded by the Patient-Centered Outcomes Research Institute (PCORI) is the Comprehensive Post-Acute Stroke Services (COMPASS) Study. COMPASS will randomize 41 hospitals (40 randomized units) and enroll a widely diverse group of 6000 patients across North Carolina (NC). Results of the trial will provide evidence to create national quality standards for post-acute stroke care delivery, which currently do not exist.The purpose of the COMPASS Study is to evaluate the comparative effectiveness of a patient-centered, transitional care model that provides both structure and processes for post-acute care of stroke patients versus usual care. Our working hypothesis is that the patient-centered model will improve functional outcomes and reduce caregiver strain and hospital readmissions.PCORI has established stakeholder engagement as a central tenet of their funded research portfolio. There is currently little guidance in the biomedical literature on how best to create and foster stakeholder engagement throughout the research process –10. The The COMPASS intervention is built on early supported discharge (ESD) studies in Canada and Europe . ESD is The COMPASS intervention provides both structure and processes to post-acute care for stroke patients. The innovative intervention combines services from a post-acute care coordinator (PAC) who is an registered nurse, and an APP (physician assistant (PA) or nurse practitioner (NP)) with linkages to community-based services to enhance continuity and coordination of stroke post-acute care. Patients are contacted at discharge, 2 days after discharge by phone, an in-person visit within 7–14 days after discharge, and phone calls at 30 and 60 days after discharge. During the clinic visit within 7–14 days of discharge, the PAC performs standardized assessments to assess the physical, social, and medical determinants of health and specifically screens for common post-stroke problems. The COMPASS intervention is unique in its focus on addressing post-discharge needs of both patients and family caregivers. An assessment of the patient’s functional status and related post-acute care needs, and the caregiver’s capacity for assisting the patient provide the basis for an individualized electronic care plan addressing specific needs; linking patients and caregivers to relevant community-based services , and follow-up care. The PA or NP reviews the results of each assessment with the PAC and—incorporating input from the patient and caregiver—creates an individualized electronic care plan for each patient. This plan is shared with home health, outpatient therapists, primary care, and community service providers. Therefore, all providers—and the patient and caregiver—understand and have reviewed the plan of care. The effectiveness of the intervention is assessed via a 90-day outcome survey call to the patient, mail survey to the caregiver, and claims data.Research reported in this manuscript was funded through a PCORI Award. PCORI had no role in the design of the study, data collection or interpretation, or writing or approving the finished manuscript.et al. [In designing our stakeholder engagement model and processes, we began with the engagement standards proposed by Curtis et al.  in their® Stroke program to improve quality of acute care. The academic centers that came together to create COMPASS had also worked with the NC Department of Health and Human Services and the Justus-Warren Heart Disease and Stroke Prevention Task Force on stroke research and improving systems of care. The current COMPASS project capitalized on extant systems and relationships around stroke systems of care in NC and expanded their focus from the acute care setting to the post-discharge setting.Our stakeholder partners are listed in While preparing the PCORI proposal, we identified stakeholder partners from among those listed above, who actively participated in the study planning and grant writing. Still more stakeholders were added after funding was awarded, primarily from the study team’s professional networks. These key stakeholders have established working relationships with the study team. Importantly, they were enthusiastic about joining the project and did not require a lengthy pre-engagement period to agree to participate. Prompt participation of stakeholders was essential to meet the rapid turnaround time between PCORI’s acceptance of the COMPASS letter of intent and the deadline for the full proposal. We later expanded the number of partners during study design phase, seeking to include all stakeholder groups who would be affected by the study and could influence its success. COMPASS now includes all project-relevant stakeholder groups whose input has shaped intervention design and implementation plans, for the purpose of maximizing effectiveness and uptake.The academic members of the COMPASS team represent 5 major universities in NC with expertise in neurology, nursing, rehabilitation, pharmacy, primary care, public health, health services research, quality improvement, epidemiology, and biostatistics. Importantly, they also have decades of experience engaging stakeholders in health services research, health system change, and policy change.We included thought leaders in each stakeholder group, including state policy makers, to support dissemination and long-term sustainability. A previous systematic review of stakeholder engagement in research indicated that engagement with patients was frequent, whereas engagement with clinicians and other stakeholders like payers, product makers, and policy makers was not . In COMPImportantly, we sought the participation of a racially diverse group of patient partners who share lived, but varied, experiences of stroke recovery. We also wanted to be guided by people with experiences in advocacy and hospital politics who understood the communities in rural eastern NC, where the risk for stroke is the highest in the nation. The lead patient partners for COMPASS are a female African-American stroke survivor known in NC as an effective advocate and patient leader in initiatives for stroke and heart disease prevention and a male Caucasian stroke survivor who, after his recovery, became involved in his local hospital and is now vice chair of a hospital board in eastern NC. These choices are important because African-Americans, women, and residents of the eastern part of the state are at greater risk of poor outcomes after a stroke and their voices needed to be incorporated into the COMPASS Study.We embedded patients and other stakeholders in the COMPASS Steering Committee and all study subgroups according to their self-reported interests and skills, so they have input and support decision making across the research process. Stakeholders experience the iterative nature of research and their input is incorporated into research decisions. Examples include educating data collectors to slowly read survey questions so stroke survivors can understand them, selecting outcome measures, and making the consent process understandable. This diffusion of stakeholder influence rests on shared leadership and facilitates management of stakeholder engagement for a trial of this size and complexity.A challenge of engaging a large and diverse number of stakeholders across sectors and across a wide geographic area is that not all stakeholders interact with all investigators. However, by embedding stakeholder partners in all subgroups, researchers and stakeholders can maintain close working relationships around the topics of their stated preferences. Stakeholders’ self-reported interests, expertise, time, and communication requirements informed who would work on which tasks. We consciously allow the level and type of engagement to vary by stakeholder according to interest, skill, time, health status, project needs, and budget constraints. We anticipate that these strategies will leverage stakeholders’ strengths and interests and foster more meaningful and productive participation.During the study planning/design phase we included stakeholders in 136 engagement activities, 18 of which occurred during proposal writing or before PCORI funding was awarded, 118 within the first 10 months of the grant. Using PCORI’s definition of engagement levels, of these 136 engagement activities, 66 (48%) were information sharing, 40 (30%) were consultation, 17 (12%) were collaboration, and 13 (10%) were stakeholder directed. Engagement also cut across levels of influence .Fig. 1StTo illustrate, multiple stakeholders from multiple groups defined intervention goals, objectives, and key messages; and developed intervention materials and activities based on the existing scientific evidence , 18, 19,Patients and caregivers also shaped our consent process for data collection. For example, our partners reviewed several iterations of the consent form to retain Institutional Review Board (IRB)-required content that will (1) foster comprehension among potential participants with a range of health literacy levels, and (2) not raise concerns among potential control group participants  because they will not receive the intervention. The result is a consent protocol for pragmatic trials that meets IRB requirements, but will be easier to understand and be received favorably by potential participants.Community-based service providers assisted with hospital and patient recruitment by leveraging their professional and personal connections to key decision makers within hospitals who made the decisions about enrolling their hospitals or hospital systems in the trial.Hospital-based stakeholders included stroke program coordinators, stroke neurologists, physician emergency department directors, quality/performance improvement coordinators, nurse managers, patient education directors, and senior hospital administrators. These stakeholders made a key contribution to the study design by advocating for a delayed start design , meaningOur statistical collaborators agreed that this design provides the best statistical control to measure the intervention’s effects, and our hospital stakeholders agreed that this design would help ensure that all communities benefit from study participation, which was essential to generate buy-in from individual hospitals and larger health systems.Although some stakeholder partners contribute to the COMPASS Study with limited financial compensation , most do not. Several patients and caregivers decline payment because they value “giving back” to their communities and being involved in a state-wide study aimed at improving the way we deliver care to stroke patients. Other stakeholders recognize value in their participation in other ways. Examples include study participation facilitating entry into hospital systems to build previously inaccessible partnerships and access to study team members, who can advise stakeholder organizations on survey design or program design unrelated to stroke care. Building such relationships is time consuming, but making them a priority holds mutual benefits and sows the seeds of long-lasting partnerships.The more stakeholders are engaged, the more feedback is received. This can lead to still more relationships, more feedback, and more management. In the COMPASS Study, a faculty “engagement officer” (25% effort) is charged with developing and executing a strategy to weave engagement through the trial and holding the study team accountable. A research coordinator (100% effort) is dedicated to engagement efforts. The oversight of engagement activities is centralized, but all faculty and staff team members (except biostatisticians) spend time leading engagement activities in study subgroups . COMPASS project managers can also direct additional administrative staff to support engagement efforts when needed. The engagement and administrative teams meet weekly to discuss administrative support for engagement.Our initial orientation of stakeholders to the COMPASS Study includes a discussion of the overall importance of engaging stakeholders in research, and the various levels of and opportunities for engagement (and compensation) in COMPASS, in particular. Our goal is not to turn stakeholders into researchers but to allow their expertise to shape the study design, implementation, and dissemination. We clarify that we are seeking partners with complementary strengths and unique perspectives on the health-care system and that understanding their needs is critical to the success of the project.During the pre-award period, the researchers developed a stakeholder management plan (“roadmap”) outlining where stakeholders could influence the study. The study team firmly believed that engagement had to be an iterative process in the post-funding phase because, if we truly partnered with stakeholders, their influence could certainly influence the study direction. Creating a stakeholder roadmap allowed us to target defined actions to engage stakeholders quickly, even before the study begun. Our stakeholder engagement plan focused on 4 key sets of activities: (1) study planning ; (2) hospital/patient recruitment and retention; (3) study implementation; and (4) translation, including interpreting study findings and disseminating results back to participating communities and the public. Within each activity, we have a series of circumscribed projects defined see .Table 2TGiven the rapid start-up phase, we wanted to be sure to capture any additional opportunities for engagement. Thus, we have designated and trained 1 researcher in each study subgroup to be the engagement advocate. This person notifies the faculty engagement officer either when engagement has occurred or when additional engagement of stakeholders will be needed as the study progresses. This information is tracked in the Research Electronic Data CAPture (REDCap) data system’s engagement tracker, which is described below.For stakeholders not continuously engaged in the COMPASS Study in the ways already noted, study updates are provided via conference calls, or by COMPASS team member updates at regional meetings that those stakeholders were already scheduled to attend. The team includes skilled facilitators with expertise in leading focus groups or group feedback sessions. In addition, the team includes experts in primary care practice and pharmacy quality improvement; physical therapy; and family caregiver needs. They engage with their respective communities for this study where their contacts and stature as opinion leaders are most effective. An important component of our stakeholder roadmap is explaining to partners how their feedback was incorporated and, if not, why not—and in a timely manner.Face-to-face contact is invaluable for building relationships, and all study team members have spent significant time traveling to meet with stakeholders. Yet, limited time and resources for state-wide travel necessitate using regular conference calls, webinars, and email contacts. When patient and caregiver stakeholders do not have email access or cell phone minutes, we call them at their homes. Openness to special arrangements is important. For example, to connect with 1 highly engaged couple (each both a stroke survivor and a caregiver) in rural NC, we email their neighbor. This person prints out emails and drives them to our partners, who then call us from their house phone when they have time to talk. We also mail documents to our partners when that seems most practical.Including patients, family caregivers, clinicians, and organizations dedicated to improving care for stroke patients in COMPASS subgroups, including the Steering Committee, allows for information exchange, keeps the patient perspective central, and fosters transparency. To make our engagement processes clear, we clarify roles, expectations, compensation, workload, timelines, deliverables, and decision-making processes.Most of our stakeholder interactions are defined by the IRB as non-research activities , but some are defined as research and require IRB approval . This distinction is made explicit to our stakeholders.Our written compensation policy has evolved to account for stakeholder feedback while meeting PCORI principles of reciprocal relationships, trust and transparency, and budget constraints. For example, we do not reimburse clinical or industry partners for their participation, but we do reimburse patient and caregiver participants. Because employees of our main community-based partner cannot accept consulting fees, we budgeted support to the organization in a subcontract. We make clear in advance when we cannot compensate individuals for their time in an upcoming activity. Some choose to participate and others decline. This flexible participation is an integral part of our model.Central to our infrastructure is a system to capture engagement work and stakeholder feedback so that we can track what actions were ultimately taken based on their input and to better delineate the impact of a given strategy on outcomes. To do this, we developed an engagement database within the REDCap software system, a secure platform for research data collection and analyses developed within the Clinical and Translational Science Award network . Our datWe will monitor our engagement process through proactive oversight of milestones and deliverables spelled out in our PCORI contract and engagement proposal.We will perform quarterly check-ins via survey to ask all patient and stakeholder partners if they feel their perspectives are valued by the study team, if they are satisfied with their level of involvement, if they are involved in the activities they want to be involved in, and if the methods used to elicit their feedback match their preferred style of communication. These data will be used to continuously improve study processes, which signals to all stakeholders the power and value of their voice.This paper describes the comprehensive methods used to incorporate stakeholders in the planning and design of a large pragmatic trial. To date, we have successfully included perspectives and input from multiple stakeholder groups in a timely fashion because our efforts were driven by a sound conceptual model and built on an existing state-wide infrastructure. This paper is intended to provide guidance to others planning or conducting stakeholder-engaged research as we collectively attempt to identify evidence-based approaches by which engagement can enhance research and close the gap from evidence to action.


Anoxia can be expressed and quantified as the anoxic factor; hypoxia, for a specific level of oxygen depletion, can be expressed as the hypoxic factor in lakes, reservoirs, and river sections. These methods summarize information of individual dissolved oxygen profiles as annual values or factors that facilitate comparison between and within lakes. Therefore, these factors are useful in the formulation and testing of hypotheses related to the dissolved oxygen status in water bodies. Methods of calculating different factors for different oxygen levels and water layers, including those applying separately to the epilimnion and hypolimnion, are presented in detail. Proven and potential applicability include: (1) the quantification of relationships with lake water quality variables and lake classification (trophic state), (2) the evaluation of restoration techniques with respect to their effects on hypolimnetic oxygen depletion, (3) the determination of internal phosphorus loading in stratified and polymictic lakes, (4) the exploration of habitat constraints due to hypoxia , (5) forecasting potential effects of climatic change on oxygen content and internal phosphorus loading, and (6) the establishment and examination of criteria and guidelines with respect to hypoxia by custom-made definitions.Hypoxia and anoxia occur frequently in freshwater systems and have biological and chemical implications.


The development of measures for assessing oral health status is essential to the evolution and maturation of a scientific knowledge based in geriatric dentistry. Their development as branches of clinical care depends in part as ability to demonstrate an effective grasp of the problems of elderly and their solutions. Any strategy for altering the health status of elderly requires a technology for first assessing that health status and then detecting increments of progress. Development of geriatric oral health assessment index (GOHAI) is a self reported measure designed to assess the oral health problems of older individuals. The goal of geriatric assessment is to collect information that will facilitate diagnosis and suggest appropriate interventions. Such a measure would facilitate estimation of prevalence of oral functional problems in older individuals, It would also increase understanding of psychological impact of oral disease and provide a means for comparing the effectiveness of various dental treatment. It also emphasizes on social well being and reflects quality of life following replacement of missing teeth. The development of measures for assessing oral health status is essential to the evolution and maturation of a scientific knowledge based in geriatric dentistry. Their development as branches of clinical care depends in part as ability to demonstrate an effective grasp of the problems of elderly and their solutions. Any strategy for altering the health status of elderly requires a technology for first assessing that health status and then detecting increments of progress.Development of geriatric oral health assessment index (GOHAI)  a self rThe geriatric oral health assessment index (GOHAI), guidelines suggest use of measure to asses the impact of oral condition on quality of life (QoL) of individual. The 12 item GOHAI was developed to evaluate three dimensions of oral health related quality of study (QoL).Oral health can be measured using patients self reports;Levels of oral health vary among patients, this variation can be measured using patients self reported measure;Predictors of oral health self reports can be identified.The purpose of this study was to test a self reported oral health assessment index for use in older adult population in kundapura taluq. GOHAI was guided by several underlying assumptions:In a study to translate and validate the Geriatric Oral Health Assessment Index  into Arabic and assess its validity and reliability for use among people in North Jordan. The study concluded that GOHAI demonstrated acceptable validity and reliability when used for people in North Jordan. It could therefore be used as a valuable instrument for measuring oral health-related quality of life for people.A study was done by Translation, reliability analysis and validation of a German version of the Geriatric/General Oral Health Assessment Index (GOHAI) The responsiveness to change in oral health status was assessed by pre- and post-treatment comparison The German version of the GOHAI had sufficient reliability, validity and responsiveness to be used as measure of oral health-related quality of life in cross-sectional and longitudinal studies of the elderly.Physical function including speech, eating and swallowing;Psychological function including worry / concern about oral health, dissatisfaction with appearance, self conscious about oral health and avoidance of social contact because of oral problems;Pain and discomfort including use of medication to relieve pain or discomfort from mouth .To evaluate three dimensions of oral health related quality of life (QoL).The study was conducted to evaluate oral health related quality of life in relation to physical functions, psychological function, pain and discomfort in older individuals residing in rural areas of kundapura taluq, udupi district, karnataka, India.All subjects were between 60 to 90 years.The present study was a questionnaire survey. The questionnaire had 16 questions, 4 on physical functions, 5 on psychological, 3 on pain and discomfort. Patient consent was obtained before administration of questionnaire. Data included age, gender, education level, perceived need for dental treatment wears a denture, perceived general health and mean GOHAI score was obtained using the same questionnaire (Annexure 1).The data collected were categorical and statistically analyzed using SPSS 15 version. The descriptive of age and mean GOHAI score was assessed. The frequencies of sex, education level, perceived need for dental treatment wears a denture, perceived general health was obtained. Chi square test was applied to see the association between mean GOHAI SCORE with age, sex, education level, perceived need for dental treatment wears a denture, and perceived general health was obtained.A total of 141 subjects were included in this study, out of 141 respondents 64 were males and 77 were females  Figure , 1c, 1d The results of the chi square test between,Mean GOHAI score and ageMean GOHAI score and sexMean GOHAI score and educationMean GOHAI score and treatment needMean GOHAI score and wears prosthesisMean GOHAI score and general healthIn a study to translate and validate the Geriatric Oral Health Assessment Index . The Cronbach’s alpha was 0.79. Mean GOHAI scores increased with more positive self-rated oral health and general health. The elderly with no perceived dental treatment need had higher mean GOHAI scores than those with perceived needs. There were slightly stronger inverse correlations between GOHAI scores and caries experience, number of teeth present, and number of pathologically mobile teeth. The measure demonstrated strong test-retest reliability. Using ANCOVA, self-rated perception of oral health and perceived need for dental treatment had the most significant impact on the GOHAI score.In a study to translate the original English version of the Geriatric Oral Health Assessment Index (GOHAI) into Arabic and assess its validity and reliability for use among people in North Jordan. After translation into Arabic and back-translation to check the translation quality, a total of 288 participants completed the Arabic version of the GOHAI questionnaire. Individual GOHAI items were recoded and summed as originally recommended. The questionnaire sought information about socio-demographic characteristics and self-reported perception of general and oral health. Clinical examination included assessment of periodontal status, and number of decayed teeth, missing teeth, filled teeth and crowned teeth. Reliability, internal consistency, and concurrent, convergent and discriminant validity of GOHAI scores were examined. Mean GOHAI score was 40.9 . The test-retest correlation coefficient for add-GOHAI scores was 0.72, indicating good stability. Add-GOHAI scores increased with poorer perceived general and oral health. The Arabic translation of the GOHAI demonstrated acceptable validity and reliability when used for people in North Jordan. It could therefore be used as a valuable instrument for measuring oral health-related quality of life for people.In a study in Assessment of changes in oral health-related quality of life among patients with complete denture before and 1 month post-insertion using Geriatric Oral Health Assessment Index As there is scanty literature available on GOHAI in the Indian population, the present study was undertaken to assess the changes in GOHAI before and 1 month after placement of dentures in completely edentulous patients reporting to a dental hospital at Indore, India. The GOHAI questionnaire was completed by the examiner who interviewed the patients (n=35) before placement of complete dentures and 1 month later. Mean, median values were calculated and the data were analysed using Wilcoxon signed-rank test. When overall mean was considered, the GOHAI scores increased from 27.48 to 30.19 .Patients reported improvement in functional changes after placement of complete dentures.


The 2014 outbreak of Ebola virus disease (EVD) in West Africa was the largest in history. Starting in September 2014, International Medical Corps (IMC) operated five Ebola treatment units (ETUs) in Sierra Leone and Liberia. This paper explores how future infectious disease outbreak facilities in resource-limited settings can be planned, organized, and managed by analyzing data collected on water, sanitation, and hygiene (WASH) and infection prevention control (IPC) protocols.We conducted a retrospective cohort study by analyzing WASH/IPC activity data routinely recorded on paper forms or white boards at ETUs during the outbreak and later merged into a database from two IMC-run ETUs in Sierra Leone between December 2014 and December 2015.The IMC WASH/IPC database contains data from over 369 days. Our results highlight parameters key to designing and maintaining an ETU. High concentration chlorine solution usage was highly correlated with both daily patient occupancy and high-risk zone staff entries; low concentration chlorine usage was less well explained by these measures. There is high demand for laundering and disinfecting of personal protective equipment (PPE) on a daily basis and approximately 1 (0–4) piece of PPE is damaged each day.Lack of standardization in the type and format of data collected at ETUs made constructing the WASH/IPC database difficult. However, the data presented here may help inform humanitarian response operations in future epidemics. The outbreak of Ebola virus disease (EVD) outbreak in West Africa that began in 2014 is the largest and most devastating since the Ebola virus was first discovered in 1976. The epidEVD is characterized by symptoms of fever, weakness and pain that may progress to internal and external bleeding, shock and death during the later stages of the infection. TransmisIn order to run an ETU, strict IPC measures and an effective and robust WASH team focused on WASH activities proved essential to protect patients and staff. Core WASH activities in an ETU setting are based on clearly defined and promulgated protocols for all activities related to IPC. These activities include: sensitization of clinical and non-clinical staff working in and around the ETU to the highly infectious nature of EVD; establishing clear protocols; training all staff on these protocols; providing critical nonclinical support to staff and patients including sanitization of facilities; ensuring safe and proper disposal of contaminated material; ensuring proper and dignified burial of deceased patients; and overseeing the logistics and procurement of appropriate materials including personal protective equipment (PPE) and chlorinated water. PPE includes the various pieces of protective clothing worn to prevent staff in the ETU from coming in contact with bodily fluids which may contain the Ebola virus, including hoods, goggles, masks, coveralls, aprons, gloves, and boots. Chlorine is used for decontamination in the ETU setting, with 0.5% chlorine used for disinfecting equipment and surfaces and 0.05% for washing hands and disinfecting skin. The design of the ETU is also essential for proper IPC. ETUs are generally divided into a high-risk zone, which includes separate wards for patients with suspected and confirmed Ebola, and a low-risk zone, where clinical staff and ancillary activities such as food preparation and laundry are based. Staff would undergo a process of donning PPE in the low-risk zone before entering the high-risk zone, where they would complete their clinical or WASH activities and then undergo a process of doffing their PPE before returning to the low-risk zone.Using data collected as part of operational procedures at two ETUs in Sierra Leone in 2014 and 2015, this study provides insight for the planning, organizing and managing of future infectious disease outbreak facilities in resource-limited settings. In particular, we utilize empirically collected data to provide estimates of chlorine, personal protective equipment (PPE), supply, and staffing needs in the context of managing an ETU in an EVD outbreak setting.de novo while the Kambia ETU utilized a previously existing health facility. As no individual patient data was collected for this study, we did not seek formal ethical approval. No additional approvals or permits were required under Sierra Leone law for this retrospective research. All data for this study has been made freely available to the public.This retrospective cohort study includes data on key WASH/IPC activities carried out in two ETUs in Sierra Leone operated by International Medical Corps (IMC) between December 2014 and December 2015 as part of its comprehensive response to the EVD epidemic in West Africa. Data was collected from the Makeni ETU, located in the Bombali District of Sierra Leone and the Kambia ETU, located in the Kambia District of Sierra Leone. The Makeni ETU had a maximum daily patient occupancy of 58 patients and maximum monthly staff levels of 300 personnel at the height of the epidemic; the smaller Kambia ETU had a maximum daily patient occupancy of 16 patients and maximum monthly staff levels of 150 personnel. Both utilized standard ETU design, as described above, with high and low-risk zones, though the Makeni ETU was built Data on all WASH/IPC activities were recorded by a WASH Officer at each ETU as part of routine WASH/IPC activities on a WASH/IPC logbook in two ways: (1) inventory and status of supplies and activities by shift and (2) “briefing-debriefing” sessions at the start of each shift.Water: quantity used in the low and high risk zones; concentration  used in the low and high risk zones; number of pumping hours; number and location of water taps and tanks; and details on any repairs and/or maintenance required for the water distribution networkChlorine: stock reports including types of chlorine and protective equipment  required to handle chlorine; quantity used to dose tanks with 0.5% (chlorine water tanks) or 0.05% (fresh water tanks) chlorine concentrations; and mixing and refilling time and contact time (at least 30 minutes for chlorine to be able to effectively kill/inactivate pathogenic organisms) of the chlorine treated waterPPE consumption: stock reports including quantity requested from warehouse to donning/dressing room; size and specification for each type of PPE used in the low and high risk zones including whether the type of PPE was disposable or re-usable; record of damages; record of disposable PPE taken for incineration; record of re-usable PPE to be disinfected, cleaned and dried; and record of disinfected, cleaned and dried PPE brought back to the donning/dressing roomLaundry: quantity brought in from and returned to donning/dressing room; and quantity of detergent and soaps used for washingWaste management: quantity (in kilograms) of waste produced in the low and high risk zones; number of waste bags used; number of sharps collected into sharps boxes and properly disposed of into sharps pit; number of buckets for wet symptoms used and contents properly disposed of into latrines; and liquid waste from laundry and wards channeled into soak pits within the high risk zoneHigh risk zone activities: number of wards disinfected; number of patient beds disinfected; number of resting areas disinfected; number of bathrooms  disinfected; and number of repairs to plumbing connectionsLow risk environmental cleaning: office cleaning; picking up trash; cleaning of toilets and bathrooms; and cleaning of drainageStaffing: number of staff assigned to each 8-hour shift; number of staff supervising (should be one officer and one shift supervisor); and number of staff in charge of waste, laundry, chlorinators, sprayers, dead body management, and high risk zone hygienistsInventory and status of supplies and activities by shift included:At the start of each shift, all staff assembled into the hygienist room for a brief presentation by the outgoing shift. In this presentation, areas of focus were highlighted, tracking forms handed over to incoming shift staff and the rigorous procedures in place for staff and patient safety while working at the ETU reinforced.Initially, data were collected on paper forms or white boards depending on ETU procedures. Data were later entered into separate electronic databases at each ETU by WASH Officers and Managers on a weekly basis for all data collected from the low risk zone and on a daily basis for all data collected from the high risk zone. Later, these data were combined into a unified database by IMC staff.The primary variables of interest for WASH/IPC activities in ETUs were 0.05% (low concentration) and 0.5% (high concentration) chlorine solution consumption, waste bags incinerated, beds/cubicles disinfected, low risk zone and high risk zone staff entries, PPE used/damaged  and daily patient occupancy.Descriptive statistics were calculated for the primary variables of interest. We conducted univariable and multivariable linear regression analyses to examine differences in amounts of chlorine used, WASH activities performed and PPE used by both daily patient occupancy and daily high-risk zone (HRZ) staff entries, in order to develop a predictive model for future usage of chlorine based on patient and staffing considerations in an ETU. The adjusted R squared statistic was used to estimate the variability in data explained by each model. In all cases, a p value less than 0.05 was considered significant. Data analyses were conducted in STATA 13 .The full International Medical Corps WASH/IPC activities database consisted of information collected from two ETUs in Sierra Leone over the course of 369 days. Approximately one-third of the data were from the Makeni ETU in Bombali District, covering the period of December 2014 to April 2015, while the rest of the data were from the Kambia ETU in Kambia District, covering the period of April 2015 to December 2015.As seen in Figs High risk zone staff entries explained about 66% of the variability in daily 0.5% chlorine solution usage, while daily patient occupancy explained about 44% of the variability in daily 0.5% chlorine usage. In linear regression analysis, 67 liters (95% CI: 62–72) of 0.5% chlorine were used on average for each high risk zone staff entry on a given day, while 133 liters (95% CI: 116–151) of 0.5% chlorine was used per admitted patient per day in the ETU. As seen in Figs Even controlling for daily patient occupancy and high risk zone staff entries, there were significant differences in chlorine usage by ETU. In multivariable linear regression, the Makeni ETU used 1982 (95% CI: 1659–2305) additional liters of 0.5% chlorine per day as compared to the Kambia ETU. There were no significant differences in the use of 0.05% chlorine per day by ETU after controlling for daily patient occupancy and high risk zone staff entries.Like other infectious disease interventions, EVD outbreak interventions require efficiently designed and operated treatment facilities in order to ensure a low risk of nosocomial infection and easy to maintain monitoring WASH/IPC practices.  Our studOur data shows that high concentration (0.5%) chlorine usage was linked to both staff high risk zone entries and patient population, but correlated much better with high risk zone staff entries. Decisions on how much high concentration chlorine should be ordered will therefore be based on how many staff will be rounding over how many rounds daily as opposed to the more unpredictable measure of a patient population. Low concentration chlorine usage was less well explained by daily patient occupancy and high risk zone entries as all staff and patients throughout the ETU–in the low-and-high risk zones–were utilizing low concentration chlorine solution for activities such as bathing, handwashing, laundry, and kitchen use (e.g. utensils). We also noted significant variability in chlorine usage between the two ETUs studied. This difference may be explained by the larger catchment area and ambulance fleet utilized by the Makeni facility, which would have required significant high concentration chlorine usage to disinfect ambulances after each trip. This is an important logistical point, which should also be taken into account during ETU operational planning.Our data demonstrated that WASH activities such as the number of cubicles cleaned and the number of waste bags incinerated daily is correlated to daily patient occupancy. This information is helpful for WASH staff planning purposes with regards to incineration needs within ETUs and allows for an incineration schedule based on waste volume and replenishment of essential waste bags both in warehouse and in the high risk zone.Laundering and disinfecting PPE on a daily basis is extremely important to estimating the number of items needed for laundering. The number of PPE damaged gives a sense of how often each type of PPE is damaged and how often we can expect to replace items on a daily basis. Damaged sprayers cannot be overlooked as they are vital to IPC/WASH protocols and on average, 2–3 are damaged daily. Timely replacement is vital, which can be difficult due to overwhelmed local markets, which might not normally supply this high quality product.One of the greatest challenges in building our database was the lack of standardization in the data collected across different ETUs. Despite being managed by the same organization, the various ETUs collected different types of WASH/IPC data in variable formats and in some cases the types of data collected changed over time. This was due to a variety of factors, including the emergent nature of the epidemic, the lack of time to agree upon and disseminate standardized data collection forms and the lack of prior empiric evidence on which data elements were most important to collect in the context of operating an ETU.The severe logistical constraints related to collecting data in a treatment facility tailored to a highly contagious and virulent disease such as Ebola cannot be overemphasized. The majority of IPC/WASH data were collected in the ETU’s high risk zone. Therefore, staff collecting the data were either dressed in full PPE, which limited both their movements and the time they could spend collecting and recording information, or they recorded information by recall on a whiteboard after exit from the high risk zone, doffing and rest/rehydration. This may have led to recall bias. In the future, other solutions such as electronic databases access through hand held tablets could be considered for more efficient means of collecting data. Even for organizations and individuals with significant humanitarian logistics and supply chain experience, the unique factors involved in managing an ETU during an EVD outbreak require special consideration. The key findings from this study, as well as lessons learned with regards to data collection, will inform the planning, organizing, and managing of ETUs in future Ebola or other infectious disease outbreak. In particular, this research provides estimates on the amount of chlorine and personal protective equipment required to manage an ETU during a future Ebola epidemic, based on the anticipated size and staffing of the ETU. The manuscript also provides recommendations for improving operational data collection in future similar humanitarian emergencies, in order to contribute to continuous learning and improvement.S1 Fig(XLSX)Click here for additional data file.


Mage = 14.09, SD = 1.40) who participated in two waves of surveys across six months. Among the participants, 227 were left-behind children with two parents migrating, 176 were with one parent migrating, and 485 were comparison children. Results showed that, (1) left-behind children reported lower levels of life satisfaction, school satisfaction, and happiness, as well as a higher level of loneliness in both waves; (2) After controlling for several demographics and characteristics of parental migration among left-behind children, future orientation significantly predicted life satisfaction, school satisfaction, and happiness in both cross-sectional and longitudinal regression models, as well as loneliness in the longitudinal regression analysis. Social support predicted immediate life satisfaction, school satisfaction, and happiness, as well as subsequent school satisfaction. Similar to left-behind children, comparison children who reported higher scores in future orientation, especially future expectation, were likely to have higher scores in most indicators of psychological adjustment measured at the same time and subsequently. However, social support seemed not exhibit as important in the immediate status of psychological adjustment of comparison children as that of left-behind children. Findings, implications, and limitations of the present study were discussed.Existing research has found that parental migration may negatively impact the psychological adjustment of left-behind children. However, limited longitudinal research has examined if and how future orientation  and social support  are associated with the indicators of psychological adjustment  of left-behind children. In the current longitudinal study, we examined the differences in psychological adjustment between left-behind children and non-left behind children (comparison children) in rural areas, and explored the protective roles of future orientation and social support on the immediate  and subsequent (lagged effects) status of psychological adjustment for both groups of children, respectively. The sample included 897 rural children ( The rapid urbanization in China has led to an unprecedented growth of economically driven rural-to-urban migration, with millions of rural residents migrating and seeking employment in cities  to describe, explain, and optimize the development of children who are residing in challenging contexts. According to the Five Cs model of PYD Lerner, , all youAs one of the most significant individual factors in child development, future orientation is associated with children's psychological adjustment and health risk behaviors  Does the psychological adjustment of rural children differ by the characteristics and patterns of parental migration? (2) After controlling for key demographics and parental migration characteristics, how are future orientation and social support associated with the immediate and subsequent status of psychological adjustment among left-behind children? Do future orientation and social support play different roles in the psychological adjustment of left-behind children compared to non-left-behind children?M = 14.09, SD = 1.40) who were available in both surveys for the analysis. Among these participants, 403 were left-behind children (227 children with two parents migrating and 176 children with one parent migrating) and 485 were comparison children.A longitudinal study was conducted in two rural counties of Yulin City, Guangxi province in 2010. In the Wave 1 survey, participants were 1164 rural children. After 6 months, the Wave 2 data were collected from 978 of the Wave 1 participants (84% follow-up rate). Children who had substantial missing data  in either Wave 1 or Wave 2 were excluded, resulting in 897 participants  between 10 and 17 years of age  provided participants with a detailed description of the study design and consent procedure, and then invited them to participate in the study. The eligibility criteria for left-behind children included the following: (1) they were born and raised in the countryside; and (2) one or both of their parents migrated to cities for jobs. Similar numbers of non-left-behind children were recruited from the same schools for comparison. Eligible students provided written informed consent for their participation. We did not obtain written parental consent for participants for the following reasons: (1) Parental consent process was not required by the ethics review board for the study of this nature at the time when we submitted the research protocol; (2) The survey did not include questions that would solicit sensitive personal information regarding the participants, their parents, or relationships within their family; (3) When we explained the consent process to the youth, we mentioned that they can stop taking the survey or skip any questions whenever they do not feel comfortable answering. Participants who provided written consent were asked to complete a self-administered questionnaire. For younger students in lower grades, interviewers provided assistance by explaining survey items using the local dialect. All study procedures including the consent process were approved by the IRB at Beijing Normal University.All participants were asked to provide demographics, including gender, age, grade, parental education levels, and family economic level. A composite family socio-economic status (SES) score was then created by indexing parents' (father and mother) education level and family economic status. The composite SES score ranged from 0 to 3, with a higher score indicating better family SES. Children whose parents (father and mother) had not received any education and whose family economic status was very poor compared with their neighbors were coded 0 in their SES, indicating that they had a low family SES. Children whose parents (father and mother) had higher than primary school education and whose family economic status was at least average compared with their neighbors were scored 3 in their SES, meaning that they had good family SES.Participants were asked to answer two questions about the whereabouts of their father and mother. There were three options for these two questions: “1 = at home ; 2 = working in another province; 3 = working in another city in the same province.” Patterns of parental migration were created based on the answers for these two questions: children with one parent migrating (either mother or father was migrating to another city out of their hometown), children with two parents migrating (both parents were migrating to another city out of their hometown), and comparison children (both parents were at home). We did not create subcategories  for one parent migrating because we did not have sufficient sample of children under the category of maternal migrating.Left-behind children answered the following questions regarding their experience with parental migration: “How old were you when your father/mother migrated?” “How long has your father/mother migrated?” ; “How frequently does your father/mother contact you?” ; “Who is your current caregiver?” ; “What is the educational level of your current caregiver?”; and perceived caregivers' attitude toward left-behind children . In the present study, current caregiver refers to adults  who were physically taking care of the left-behind children.Three scales were used to measure the children's future orientation, including Children's Future Expectation Scale . The questions were: “In general, how satisfied are you with your life?” and “How satisfied are you with your schooling”? A higher score indicated a higher level of satisfaction with life or schooling.Happiness was measured using Oxford Happiness Questionnaire (OHQ) short scale  or ANOVA (for continuous variables).Second, to answer the first research question, group differences  in terms of satisfaction, loneliness, and happiness in both waves 1 and 2 were examined using ANOVA.Finally, to answer the second research question, we used linear regression analysis to examine the cross-sectional and lagged effects of future orientation and social support on psychological adjustment after controlling for several key demographics and characteristics of parental migration that have significant associations with psychological adjustment in Wave 1. These analyses involved a two-step process. First, cross-sectional analysis was conducted to investigate the associations of future orientation and social support with the indicators of psychological adjustment in Wave 1 for left-behind children and comparison children, respectively. Second, longitudinal data were analyzed by examining the prediction of future orientation and social support in Wave 1 for psychological adjustment in Wave 2 for these two groups of children, separately. Although, the separate regression analyses for left-behind children and comparison children do not allow us to compare the prediction effects of social support and future orientation between the two groups of children, we would still be able to examine which factors are common protective factors for both groups of children, and which factors may play unique protective roles in the psychological adjustment of a particular group. All statistical analyses were performed using SPSS for Windows 22.0.The attrition analysis showed no significant differences between the longitudinal sample (897 children) and attrition sample (186 children) with respect to Wave 1 demographic characteristics, including gender, grade, parental education level, and family economic status. Demographics analysis  and hopefulness about future  significantly predicted life satisfaction; social support  and control over the future  significantly predicted school satisfaction; future expectation , control over the future  and social support  significantly predicted happiness. Multiple linear regressions for comparison children showed that, future expectation and hopefulness about future significantly predicted life satisfaction  and school satisfaction ; future expectation  and social support  significantly predicted happiness. Loneliness was not predicted by any of covariate or independent variables among either group of children.In the cross-sectional analysis Table , multiplp < 0.001, η2 = 0.04), school satisfaction , happiness , and negatively predicted loneliness ; Wave 1 hopefulness about future was negatively associated with Wave 2 loneliness ; Wave 1 social support significantly predicted Wave 2 school satisfaction . In the longitudinal analyses for comparison children, after controlling for key demographic characteristics, Wave 1 future expectation positively predicted Wave 2 life satisfaction , school satisfaction , happiness , and negatively predicted loneliness ; Wave 1 hopefulness about future was negatively associated with Wave 2 loneliness ; Wave 1 social support significantly predicted Wave 2 happiness .In the longitudinal analyses Table , after cThe current study presented both cross-sectional and longitudinal evidence to suggest that, left-behind children are disadvantaged in terms of unfavorable status of psychological adjustment compared with children living with both of their parents in the same rural communities in Guangxi province. In addition, we found that future orientation and social support can serve as protective factors for some indicators of immediate and subsequent status of psychological adjustment among left-behind children and comparison children. Future orientation plays similar roles in the psychological adjustment of both left-behind children and comparison children. Compared to non-left-behind children in rural areas, social support appears to be more important for the immediate status of psychological development of left-behind children.Our analyses revealed that left-behind children reported lower levels of life satisfaction, school satisfaction and happiness, as well as a higher level of loneliness in both waves, compared to rural children living with both of their parents. The finding is consistent with previous studies, which indicated that parental migration had a significantly negative effect on psychological adjustment of left-behind children, especially children with two migrating parents (Fan and Sang, Another important feature of the present study is the testing of the protective effects of future orientation and social support on the immediate and subsequent status of psychological adjustment among left-behind children and comparison children. Both cross-sectional and lagged effects analyses showed that one or more sub-measures of future orientation served as important protective factors for indicators of psychological adjustment of all rural children, including left-behind children and non-left-behind children. Specifically, for both groups of children, higher scores in hopefulness about future was related to higher levels of life satisfaction, and higher scores of future expectation was associated with higher levels of happiness measured at Wave 1. Left-behind children who reported higher scores in control over the future were likely to exhibit higher levels of school satisfaction and happiness measured at the same time. Comparison children who reported higher scores of future expectation were likely to have higher levels of school satisfaction measured at the same time. For both groups of children, future expectation is beneficial to all indicators of psychological adjustment measured subsequently among both groups of children; hopefulness about future is also an important protective factor for loneliness measured subsequently. These findings further highlight the protective effects of future orientation on children's psychological adjustment and elevate our understanding of the specific effects of each indicator of future orientation. It should be noted that, rural children's higher levels of hopefulness about future and future expectation may relate to lower vulnerability to loneliness, a typical emotional problem among rural children, especially left-behind children (Su et al., In addition, we found that social support is also an important protective factor for the psychological adjustment of rural children, especially for left-behind children. For comparison children, higher scores in social support were related to higher levels of happiness measured at the same time, and a better status of school satisfaction measured subsequently. Social support was associated with life satisfaction, school satisfaction, and happiness among left-behind children at the cross-sectional level, as well as school satisfaction reported 6 months later. Cumulative evidence from a variety of settings have showed that a social support network can help disadvantaged, troubled, or vulnerable youth to cope with difficulties and, as well, facilitate their psychological development (Cohen and Wills, Several potential limitations of the present study are worth noting. First, despite efforts to ensure sample representation, the participants were only chosen from two rural counties in China. Thus, the findings may not be generalized to left-behind children in other areas of the country. Second, although the current two-wave longitudinal study has deepened our understanding of the associations between positive factors and psychological adjustment among left-behind children, future longitudinal studies using data from multiple time points are still needed to further explore how may the coactions of protective individual and contextual factors mitigate the adverse effect of parental migration and shape the development of left-behind children. Third, the present study solely relied on self-assessment of key psychological adjustment variables. A better measurement scheme should be utilized to gather information from left-behind children, as well as from other sources, including reports from parents, current caregivers, teachers, and peers. Finally, although the separate regression analyses allowed us to preliminary compare the roles of future orientation and social support in the psychological adjustment of left-behind children and comparison children respectively, we were not able to compare the effects of these protective factors between these two groups of children because they were not sharing the same sets of covariates in the present study. Future research may adopt appropriate approach to compare the effects of these protective factors between the two groups of children.Despite these potential limitations, the findings of the present study have some important implications for psychological health promotion and future research. First, the findings highlight the need for governments and local communities to recognize the adverse effect of parental migration on the psychological adjustment of left-behind children. As such, effective psychological adjustment promotion and intervention efforts should be provided to vulnerable rural children, especially for children with two migrating parents. Second, the government and concerned community organizations may explore effective ways to help rural children build a positive perspective toward their future and enhance the social support network for them. Governments and local communities may seek effective intervention strategies from some successful programs designed for other vulnerable Chinese children. For example, in the resilience-based intervention research for children affected by parental HIV (Li et al., DL is the primary investigator of the study and provided comments and ideas, and revised this paper. SS did data analysis, developed and revised the manuscript. XL provided comments and ideas, and helped revised the manuscript. MZ helped conduct the study, including developing survey, sampling, data analysis, and proof reading the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


For nearly five decades long-term studies in rodents have been the accepted benchmark for assessing chronic long-term toxic effects, particularly carcinogenicity, of chemicals. The European Food Safety Authority (EFSA) and the World Health Organization (WHO) have pointed out that the current set of internationally utilized test methods capture only some of the potential adverse effects associated with exposures to these agents over the lifetime.In this paper, we propose the adaption of the carcinogenicity bioassay to integrate additional protocols for comprehensive long-term toxicity assessment that includes developmental exposures and long-term outcomes, capable of generating information on a broad spectrum of different end points.An integrated study design based on a stepwise process is described that includes the priority end points of the Economic Co-operation and Development and the National Toxicology Program guidelines on carcinogenicity and chronic toxicity and developmental and reproductive toxicity. Integrating a comprehensive set of relevant toxicological end points in a single protocol represents an opportunity to optimize animal use in accordance with the 3Rs . This strategy has the potential to provide sufficient data on multiple windows of susceptibility of specific interest for risk assessments and public health decision-making by including prenatal, lactational, neonatal exposures and evaluating outcomes over the lifespan.This integrated study design is efficient in that the same generational cohort of rats used for evaluating long-term outcomes can be monitored in satellite parallel experiments to measure biomarkers and other parameters related to system-specific responses including metabolic alterations and endocrine disturbances.http://dx.doi.org/10.1289/EHP419Manservisi F, Babot Marquillas C, Buscaroli A, Huff J, Lauriola M, Mandrioli D, Manservigi M, Panzacchi S, Silbergeld EK, Belpoggi F. 2017. An integrated experimental design for the assessment of multiple toxicological end points in rat bioassays. Environ Health Perspect 125:289–295;  Synthetic chemicals have been used individually and as mixtures in consumer products for over a century, gaining intense momentum beginning after World War II. Naturally occurring elements and compounds have been used for millennia. The first bioassays for identifying chemicals posing a greater and immediate danger for carcinogenicity to individuals were first developed about 100 years ago . The cheStudy designs and outcomes investigated by current guidelines and our proposed protocol on carcinogenicity and chronic toxicity and reproductive and developmental toxicity are summarized in The NTP reference guideline for reproductive and developmental toxicity, the NTP’s Modified One-Generation (MOG) Reproduction Study , employsThe OECD TG 443 and the NTP MOG were introduced only recently, and there is still no published data comparing studies with the same substance according to the two guidelines. We cannot exclude the possibility that authorities such as the U.S. Environmental Protection Agency (EPA), the U.S. Food and Drug Administration (FDA), the European Chemical Agency (ECHA), and the European Food Safety Agency (EFSA) could require  the repetition of the tests with both guidelines considering the need for empirical evidence supporting the use of one of the two. It is our opinion that regularly studying the same substance with both the NTP MOG and OECD TG 443 represents an unnecessary repetition. The NTP’s MOG is able to generate large and robust data sets that include early-life exposure and teratogenicity, but requires a larger number of animals than the OECD TG 443 .Starting from the 1990s, the Cesare Maltoni Cancer Research Centre (CMCRC) of the Ramazzini Institute (RI) performed carcinogenicity studies on low doses of chemical or physical agents that may expose millions or even billions of people to potential carcinogenic risks, such as radiations and food additives , 2008, uTo satisfy the need to consider multiple effects  across multiple life stages and to reduce the overall number of animals required for separate studies of these end points, we propose the following experimental design that integrates traditional cancer guidelines with more recent proposals of OECD and NTP for studying reproductive and developmental toxicity. This new integrated experimental design aims to maximize the end points measured for each animal, thus reducing the overall number of animals produced and utilized, in accordance with the 3Rs  .The central aim of the methodology proposed in the Integrated Long-Term Toxicity and Carcinogenicity Study is to maximize the breadth of outcomes assessed and to increase the sensitivity of testing beyond that in commonly used protocols to give more reliable and inclusive information on many important end points .The integrated experimental design proposed by the CMCRC/RI is outlined in Carcinogenicity and chronic toxicity study. Animals are treated from fetal life  until 104 weeks of age, then observed  until 130 weeks of age (30 months). Interim kills are included to provide information on progression of non-neoplastic or neoplastic changes and mechanistic information . Animals included for interim evaluation are also exposed from fetal life  until 26, 52, 78, and 104 weeks of age following OECD guidelines  related to reproductive and developmental and other noncancer effects are studied. The possible adverse effects of the substances are studied in prenatal, neonatal, prepubertal, pubertal, and adult parous and nulliparous WOS and compared among them, or with the possible long-term carcinogenicity effect.The laboratory rat has served as the traditional animal model of choice for research and regulatory developmental and reproductive toxicity testing conducted to support human health hazard identification and risk assessment. The rat has been used extensively for developmental and reproductive physiology and endocrinology research and has been more thoroughly characterized in these research fields than other species, likewise for identifying likely human carcinogens .Our proposal to use Sprague-Dawley (SD) rats is based on the evidence that they are adequately sensitive, have a long history of being used in research studies, and are also recommended by the There are known limits for this animal model for individual cancer end points. For example, SD rats represent an optimal model for breast cancer research , while tThere is widespread agreement that the relatively small numbers of animals used in most standard toxicity tests is a serious issue in terms of sensitivity and reliability. On the other hand, there are social and ethical concerns about the number of animals used in these tests . InadequIn our proposal, breeders  of about 10–15 weeks of age are matched in a single outbred mating, in a number adequate to obtain sufficient animals for the study. The objective of breeding is to generate animals in order to have no more than one sister and one brother for each control and exposed group (two sisters and two brothers in the carcinogenicity arm) in order to avoid any bias due to familial relationship.Studying at least three exposure groups plus controls, the number for a comprehensive human equivalent hazard identification study is 1,720 animals . A higheIn compliance with the 3Rs, we suggest, whenever possible, to avoid the use of culling and to use all the pups generated during the experiment, avoiding unnecessary sacrifice of animals. It is our opinion that avoiding culling also would permit generally a more rigorous measure of litter mortality and simulate a human equivalent scenario, with more genetic variability and avoiding possible selection bias . Nevertheless, the use of culling might be appropriate for studying suspect endocrine disruptor substances, as litter size can impact the weights and the growth rate of the pups, which can affect the timing of puberty. Puberty timing regulates other end points, so that the change in body weight from not equalizing litter size early on might have an inadvertent impact on the study.Under current testing procedures , when toa priori statistical power to detect noncancer effects using a relatively small number of animals, although remarkable exceptions exist particularly for endocrine effects  prenatal (F1) animals treated during embryonic life and sacrificed at postnatal day (PND) 21; b) postnatal (F1) animals treated through lactation, starting from birth (PND 1) and sacrificed at PND 21; c) prepubertal (F1) animals treated from PND 21 to PND 42; d) pubertal (F1) animals treated from PND 42 to PND 63; and e) adult parous and nulliparous (F1) female animals treated from PND 1 through lactation, until PND 181  and organs, together with paraffin-embedded tissues, are stored for histopathology and molecular biology studies, EDCs effects, neurotoxicity, biochemical and biohematological changes (metabolism), and toxic and preneoplastic lesions.The aim of our integrated experimental design was to investigate all or a majority of possible health effects related to exposure to the studied agent and to minimizing the unnecessary use of experimental animals. Our design also avoids wasted time when doing sequential end point studies. End points assessed in traditional toxicology and carcinogenicity testing are food and water consumption, chemical exposure, weight loss and gain, clinical pathology, survival and mortality, changes in organ weight, preneoplastic and neoplastic diseases with histopathological analyses. However, many examined chemicals have shown to also cause complex effects in animals, affecting organ development and functional and behavioral changes . To bestIn our proposed lifetime experimental design, we assess a range of adverse outcomes of interest using a relatively large population of animals (sufficient power), born at the same time after mating of outbred breeders, randomized and studied for dose-related effects, with the lowest possible risk of bias . Typically, for studying all the previously mentioned parameters , approximately 10–20 studies are performed, using more animals, in different laboratories, with different procedures. Our experimental model and design overcomes these deficiencies and permits more information to be gathered on toxic, mechanistic, and biological parameters, using the same but fewer overall animals in a large but unique experiment. In fact, in our experimental design, rats from the same generation are used for studying chronic toxicity and carcinogenicity outcomes and distributed in satellite parallel experiments (WOS), thus minimizing variables between different arms of the multi-end point investigation, for detecting also reproductive/developmental toxicity.Our integrated experimental protocol requires 1,720 animals, with a reduction up to 53% in animal use as compared to using separate test protocols , represein vivo biophase should be the responsibility of one laboratory in order to favor consistency of methods and quality of long-term animal studies  for a more comprehensive analysis of potential effects of chemical exposures and outcome assessment; enabling interim analyses and other strategies to examine specific outcomes over the lifespan. For increased efficiency, results of these tests can be shared among laboratories. Ideally the  studies . After t studies .This protocol represents a proposal to regulatory scientists and the scientific community in general.A priori establishment of criteria and consensus on relevant end points of interest is also a good starting point for evidence-based evaluations and following systematic review of obtained results  PDFClick here for additional data file.