An informatics system consisting of a mobile health application and business intelligence software was used for collecting and analyzing Ebola contact tracing data. This system offered potential to improve data access and quality to support evidence-based decision making for the Ebola response in Guinea. Implementation challenges included software limitations, technical literacy of users, coordination among partners, government capacity for data utilization, and data privacy concerns. An informatics system consisting of a mobile health application and business intelligence software was used for collecting and analyzing Ebola contact tracing data. This system offered potential to improve data access and quality to support evidence-based decision making for the Ebola response in Guinea. Implementation challenges included software limitations, technical literacy of users, coordination among partners, government capacity for data utilization, and data privacy concerns. Challenges in data availability and quality have contributed to the longest and deadliest Ebola epidemic in history that began in December 2013. Accurate surveillance data, in particular, has been difficult to access, as it is often collected in remote communities. We describe the design, implementation, and challenges of implementing a smartphone-based contact tracing system that is linked to analytics and data visualization software as part of the Ebola response in Guinea. The system, built on the mobile application CommCare and business intelligence software Tableau, allows for real-time identification of contacts who have not been visited and strong accountability of contact tracers through timestamps and collection of GPS points with their surveillance data. Deployment of this system began in November 2014 in Conakry, Guinea, and was expanded to a total of 5 prefectures by April 2015. To date, the mobile system has not replaced the paper-based system in the 5 prefectures where the program is active. However, as of April 30, 2015, 210 contact tracers in the 5 prefectures were actively using the mobile system to collectively monitor 9,162 contacts. With proper training, some investment in technical hardware, and adequate managerial oversight, there is opportunity to improve access to surveillance data from difficult-to-reach communities in order to inform epidemic control strategies while strengthening health systems to reduce risk of future disease outbreaks. The longest and deadliest Ebola epidemic in history has prompted a historic global response in Guinea, Liberia, and Sierra Leone, the 3 countries affected most by Ebola. The epidemic began in December 2013 and has spread to 7 countries, causing, as of May 2015, more than 25,000 infections and almost 11,000 deaths.Controlling the epidemic requires a series of interventions: (1) community engagement; (2) identification of contacts; (3) contact monitoring for symptoms; (4) rapid lab confirmation of cases; (5) isolation and treatment of new cases; and (6) safe and dignified burials. Each activity is intrinsically complex, yet all need to be synchronized to stop transmission and control the outbreak.Real-time data in emergency situations are indispensable for guiding strategy, coordinating interventions, and troubleshooting problems.--Lessons learned from the mobile health  community demonstrate a possible opportunity to address some of these gaps. mHealth tools have been used in low-resource settings for a variety of applications, including strengthening access to data, managing health records, and improving the quality of heath interventions.We established a real-time information system for contact tracing activities in Guinea. This system comprises a smartphone application that supports contact tracer workflow and enables rapid submission of surveillance data, which is linked to a business analytics and visualization software. Dashboards built on this platform facilitate access to the data by government response managers based in both prefecture-level offices and within the National Ebola Coordination Unit (NECU). The system allows for: (1) improved quality of work through built-in algorithms for contact tracers to follow; (2) real-time identification of contacts who have not been visited, allowing same-day intervention; and (3) stronger accountability of contact tracers through timestamps and collection of GPS points with their surveillance data. In this article, we describe the opportunities and challenges of deploying an mHealth system in response to an epidemic based on our experiences in Guinea during the Ebola outbreak.An mHealth application for contact tracing in Guinea was developed to facilitate rapid surveillance of and response to the Ebola epidemic.Contact tracing is the linchpin of epidemic control. It involves identifying all people who were exposed to someone with confirmed infection (termed “contact identification”) and monitoring them for the incubation period (21 days in the instance of Ebola). The purpose of the monitoring period is to diagnose and isolate contacts immediately if they develop symptoms (termed “contact monitoring”).In the current Ebola epidemic, contact tracing has been uniquely challenging due to the sheer numbers of cases and, thus, contacts; at one point, there were almost 4,000 contacts being monitored in Guinea spread over half of the country’s 33 prefectures.Contact tracing efforts in Guinea are conducted by contact tracers who are recruited locally and trained to monitor contacts in their community. Each contact tracer is required to visit their assigned contacts on a daily basis for 21 days to assess for symptoms of illness while also serving as a community resource for Ebola-related information. Contact tracers are managed by “close supervisors” who coordinate their activities and monitor whether contact tracers visit their assigned contacts through random household spot checks.The current paper-based system proceeds as follows: The contact tracers use paper forms, based on internationally recognized templates,This paper-based system and the time-intensive effort required for entering, collating, and cleaning data pose several limitations . First, The paper-based system for contact tracing requires significant time and effort to enter, collate, and clean data, creating lag times in response.Collectively, these limitations undermine the ability of response managers to accurately understand which contacts are going unmonitored, to evaluate the performance of different contact tracers, to pinpoint where and why lapses are occurring, and to troubleshoot breakdowns. For example, if a contact tracer missed a contact visit or a contact could not be located, response managers may not find out until a few days later. Furthermore, given the urgent nature of the epidemic, many contact tracers were rapidly deployed and inconsistently trained, potentially leaving gaps in performance. Given how pivotal effective contact tracing is for reducing transmission, these delays and problems have significant consequences for controlling the epidemic whereby new chains of transmission may go unrecognized.Implementation of the mobile contact tracing program involved 3 major stages:PreparationDeploymentAdaptationSome steps within each stage ran concurrently. The length of time and key partners involved in each step are summarized in We chose to use the Open Data Kit (ODK)-based software CommCare from Dimagi as the data collection tool for contact tracing activities. Although there were other mobile health options that were being considered for use in Guinea, there were several features of CommCare that guided our decision. CommCare is one of the few mobile health applications that allows longitudinal tracking of individual people over time—a feature critical for a surveillance system that required daily visits over a set period of time—and can also accommodate the reassignment of contacts and their history from one user to another if a contact tracer needs to be switched. Furthermore, with its algorithm-based decision support features, the software has been used to reinforce maternal and child health tracking in other projects managed by the implementation team since 2013. The implementation team’s familiarity with the software’s strengths and weaknesses was also a factor that contributed to the selection of the software, as it enabled rapid implementation and minimal software iteration. Key functions of CommCare that were leveraged for the program are summarized in the Box.Goal 1: Reduce Time Delay Between Data Entry and Data ConsumptionData transmits to the server upon Internet access . If a user is not in an area with network coverage, the data are saved to the phone and automatically sent upon entering an area with network coverage.All data collected in the system are available for export to CSV or Excel files. Additionally, the data are automatically linked to dashboards with predefined analytical charts.All users with the appropriate permissions and with Internet connection can access the data as soon as the data are synchronized to the server. Permissions can be set based on location and data management needs. In areas of low bandwidth, reports can be set up to be sent automatically via email.Goal 2: Strengthen Quality of Work of the Contact Tracers and Quality of DataPreset contact tracing algorithms with built-in skip logic can guide a contact tracer through visits by prompting key questions based on previously inputted answers.Real-time data on performance  can inform supervisors for monitoring and supporting the contact tracers.Multimedia files including image, audio, and video can assist contact tracers in sensitization activities and can be used for refresher training as needed.CommCare is one of the few mHealth applications that allows longitudinal tracking of individual people over time.,We created the contact tracing application using protocols that were publicly available from the US Centers for Disease Control and Prevention (CDC) and the World Health Organization (WHO),Register contact details upon first visit by the contact tracerFollow-up with contacts daily for 21 days to monitor symptomsClose the contact from the system once the 21-day period has been completed or for other reasons, e.g., the contact has been confirmed to have Ebola or has moved permanently to another areaThis application structure thus enables users to follow contacts for the duration of the disease’s 21-day incubation period. Additionally, the application recommends action steps based on data entered in the form. For example, if a contact develops symptoms, the application reminds the contact tracer that the contact should be immediately referred for testing and treatment (if positive). A contact tracer would then follow-up with the family and/or a supervisor to confirm the test results. If confirmed for Ebola, the diagnosis would inform the closing of the contact from the system; otherwise, the contact would only be closed after completing the 21-day monitoring period upon returning home. Audio and visual content translated into local languages was also added to support contact tracers for Ebola sensitization activities. During the initial deployment of the application, publicly available audio and visual sensitization resources from WHO, Doctors Without Borders, and the CDC were included. The application was later updated with multimedia content tailored to the Guinean context, developed by headquarters in New York in consultation with the local team, over the course of 3 months. Lastly, a training video module was included to provide contact tracers with ongoing assistance if they needed refresher information on how to use the system.To facilitate interpretation of the data collected in CommCare, the business intelligence software Tableau was integrated with the CommCare server. This software was recommended by Dimagi as a powerful analytic tool that could be integrated with CommCare. Once the key fields in the application were established, we began designing draft dashboards to visualize the data. Two sets of dashboards were developed by volunteer Tableau experts over the course of 1 month:In one set, 3 dashboards depict aggregate indicators that are directly collected in each of the 3 forms, such as the relationship between the source case and the contact and symptoms presented by the contacts.The other set of 2 dashboards presents information on the performance of the contact tracers, including the number of visits conducted by each contact tracer per day .We also designed a Key Performance Indicator (KPI) dashboard with the Tableau experts, which displays a summary of data for the past 7 days . FiltersThe link between the CommCare and Tableau software made by Dimagi’s engineers and the Tableau experts allows contact tracing data to be updated on the dashboards automatically every hour. Users can view the visualizations by logging into the secure online Tableau server. The underlying specifics of the data, i.e., the personally identifiable information for each contact registered in the system, are also viewable in Tableau for authorized users and are linked to the aggregate dashboards. Thus, if users with the appropriate permissions notice that many instances of resistance were reported in the past week, they can navigate from the aggregate graph directly to the details page, where the resistant contacts will be listed and can be identified for follow-up. Access to the Tableau server was granted to all government health staff managing surveillance activities at the prefectural and regional levels, to members of the NECU, and to any workers from partner organizations and agencies as requested by the NECU.Data analytic dashboards are updated automatically every hour, enabling surveillance managers to quickly identify trends and gaps.The system’s features thus enable surveillance managers to directly interact with and use the data to quickly identify trends and gaps. This rapid data intake and ease of interaction allows for faster response. This is in contrast with the paper system’s outputs that include: (1) daily, aggregate, static visuals for the NECU, and (2) inconsistently updated Excel files that capture paper-form entries, which require additional manual analysis to identify trends.Ericsson donated an initial 1,000 Sony Xperia E1 phones, and the UN Mission for Ebola Emergency Response (UNMEER) then donated an additional 580 Samsung Galaxy S3 Neo and Motorola ATRIX phones. Solektra International donated Sunking Pro2 solar chargers to facilitate phone charging, as well as 33 solar panels to enable regular access to power in the government offices where CommCare data are being used. The United Nations Population Fund (UNFPA) helped facilitate the purchase of SIM cards with a closed user group and 500 MB of data per month from Orange. Our implementation team also provided computers and USB modems to government staff as necessary, to ensure regular access to data.Trainers were selected by the government and local UN agency partners from a pool of governmental health staff who supervise contact tracing activities. The first training of trainers (ToT) was conducted in Conakry by the local program staff in November 2014 and included approximately 17 trainers. The first half of the ToT included a didactic introduction to the phones and CommCare generally, while the second half of the ToT used blended learning techniques,Upon completion of the ToT, trainers were divided into pairs based on geographic area of supervision to conduct 2-day trainings with contact tracers and close supervisors. Because contact tracers had already received training on basic contact tracing activities, this training focused on the mobile health application. The training started with a brief review of the importance of contact tracing, safety, Ebola danger signs, transmission, and prevention, as well as tips for respectfully engaging with households. Trainers then provided an overview of the purpose and structure of the CommCare application, the types of data collected in CommCare, how to use the application, and how the data would be used. The second day of the training served as a practicum during which trial phones were distributed for practice sessions. Following initial training, periodic refreshers have been conducted as needed, especially with the objective of engaging supervisors in troubleshooting and monitoring data quality issues reported by local staff. At the end of the contact tracer training, participants were given unique usernames and passwords for CommCare, phones, SIM cards, and phone accessories, and they were asked to sign proper phone use agreements. Contact tracers who were already monitoring contacts were prompted to begin using CommCare to register their contacts and record information from daily contact monitoring visits in addition to the paper forms, which have remained the official reporting medium throughout the crisis. In certain prefectures, the training ended with the contact tracers registering all current contacts as a group. Contact tracers were also instructed through the application to obtain consent with each visit. To date, contact tracers in 5 prefectures—Boffa, Conakry, Coyah, Dubréka, and Forécariah—are using the CommCare system to monitor contacts of Ebola cases. As of May 2015, 210 contact tracers from the 5 prefectures were actively using CommCare to monitor more than 9,000 contacts collectively .Once the application was in use, the prefecture-based surveillance teams and our implementation team members began using the Tableau dashboards to monitor contact tracer activities. Initially, these surveillance teams mostly used the dashboards that focused on contact tracer performance, which functioned as a quick mechanism to see which contacts had not received daily visits. As the numbers of contacts has diminished and familiarity with the system has grown, prefecture-based teams are increasingly using the system to identify trends, for example, to investigate areas where resistance is high or zones where re-exposure to Ebola is frequent.To test the accuracy of the CommCare system, validation exercises compared CommCare raw data with paper-based contact tracing databases from the government for a given week for each commune or prefecture. When contacts could not be matched by first name, last name, age, and village, we would confer with the contact tracers and supervisors to understand the reasons for discrepancies.During the first round of data validation for the 5 communes of Conakry, there was 78% agreement between the 2 databases. In response, we made programmatic adjustments to reinforce supervision and improve communication with partner organizations. For our second round of validation in Conakry, we reached 86% agreement. In general, missing data were attributed to 4 main deficiencies:Training shortcomings/misunderstanding of the softwareMiscommunication with partners who manage contact tracersMisuse of phonesErrors in the reference database from the paper systemTo address the first 3 deficiency categories, we changed hardware and SIM card management protocols and conducted targeted refresher trainings. Challenges from the fourth category highlighted potential weaknesses in the original paper system in which data entry errors, including duplicates, created misconceptions about contact numbers.We refined the design of the application and the accompanying dashboards as the epidemic evolved, largely due to the lack of standardized contact tracing protocols and metrics agreed upon by all partners. Clearer standards from the outset could have accelerated the design process. Instead, the results of the validation exercises and feedback collected from the field were used to inform at least 3 software adaptations every 2 months for application content, logic, and prompts to better align with on-the-ground practices. For example, new options and alerts were added to the forms to reflect events in the community, such as a contact moving to a new area or being transferred to a health facility for testing. Rollout of these application updates was accompanied by refresher trainings by the local implementation team. We also iteratively designed the dashboards based on evolving end-user needs. For example, as frequent resistance from and movement of contacts continued to be a key challenge facing the Ebola response in Guinea, a new dashboard was developed to allow local surveillance teams to quickly identify contacts who were lost to follow-up .Centralized management of the mobile contact tracing system was not sufficient to ensure quality data collection by the contact tracers and routine data use by government staff and partners. In response to these capacity issues, information officers with experience in data analysis and community health programs were hired, trained, and deployed to local governmental health offices where Ebola surveillance activities are coordinated. Their initial training included an overview of the mobile contact tracing system and how to use the Tableau dashboards for monitoring data submissions by the contact tracers. The information officers’ responsibilities include basic troubleshooting of the mobile phones, daily review of data submissions, and communication of key trends to local response teams. They also train and support government office-based staff in using Tableau to manage the Ebola response.Information officers were hired to ensure quality data collection and facilitate routine data use by government staff and partners.Implementation of technology during an emergency outbreak is not without its challenges. The current iteration of the program is the result of a series of trials and corrective measures put in place to address the most common obstacles, detailed below.While the CommCare software incorporates data submission and workflow features meant to be used in low-resource health contexts, it was not designed explicitly for emergency use. Additionally, the structure of the software created limitations in capturing nuances of complex contact tracing protocols. Furthermore, messy data from the field that required additional editing in the backend of CommCare was not easily accommodated in Tableau—which is more commonly used for commercial activities in developed settings. Tableau experts and Dimagi’s engineers performed considerable troubleshooting remotely to refine and improve the connection between the 2 software programs. Finally, time limitations were also a challenge as the dashboards were built by Tableau experts who were volunteer consultants. Commissioning full-time Tableau consultants could have helped to shorten the time required to build the dashboards; however, they may not have had as much experience as the volunteer experts suggested by Tableau Foundation.The availability of local staff to support phone configuration and hardware management was limited. Adjusting phone settings for language, date, and time and addition of application locking software and widgets for battery saving required between 10–15 minutes per phone. During the expansion phase, recruitment of local tech-savvy youth volunteers helped to accelerate the configuration of hundreds of phones.After equipment was distributed, there were also limitations in implementing management policies for lost phones, data plan abuse, and phone storage. Because the contact tracers are co-managed by multiple entities including UN agencies, NGOs, and the government, coordination was necessary to create consistent policies that were agreed to and adopted by all local managers.Finally, although the CommCare software could be used offline for data collection, access to the mobile network necessary for data submission was limited in the border regions of Forécariah prefecture, which affected 21 contact tracers. In these areas, we employed workarounds such as having the contact tracer submit data every few days when in locations with better network coverage or having the information officers directly request the data from the contact tracer via phone to enter themselves when necessary. An additional option, which has been used in previous projects but not in Guinea, is the manual transfer of data from the phone to an SD card, which could then be inserted to another phone for submission in areas with better network coverage.The quality of data submissions by contact tracers was an issue at the outset. There were occasional instances where users—often older generation—were unfamiliar with smartphones and therefore could not use the software effectively. For example, some contact tracers submitted data under test user accounts that were used for training, or they made spelling errors. To address this, older-generation contact tracers would sometimes be paired with younger, more tech-savvy contact tracers or would receive more one-on-one support from implementation team members.In some cases, contact tracers used paper forms to first record information at the household, which was then entered into the phones after the visit. Although we did not systematically collect data on the frequency of this practice, comments from the field indicated that this would occur when contact tracers were concerned that the introduction of foreign technology could affect their trust with community members, who may have found the act of data collection, especially through unfamiliar equipment, strange or threatening.There were also a few cases of contact tracers misusing data plans for personal purposes. Eventually, a solution was developed in which a fund for personal phone use was added, and if contact tracers exhausted their monthly data allotment, additional data would be supplied using their personal accounts to ensure continued CommCare data submission. A monthly auto-recharge system was also set up to ensure uninterrupted submission of data.Coordination among partners was consistently a challenge that created difficulty in the identification of both contacts and contact tracers, access to paper-based data for validation, release of funds, integration of mobile contact tracing data with other Ebola response data, and cooperation with local supervisors who were managed by other organizations. International partners, including NGOs and UN agencies, expressed concern about the introduction of mobile phone technology for Ebola response purposes, citing poor literacy or possible network gaps as challenges. These apprehensions created significant delay in implementation and expansion despite having support and directive from the government lead of the Surveillance Pillar of the NECU. Validation exercises were conducted to address issues of data quality; however, presentations with partners were often delayed or canceled. Furthermore, the lack of clear leadership and coordination among partners, as well as the monthly rotation of leadership within key response organizations, created miscommunication and inconsistent messaging. To facilitate collaboration with other partners, our approach has been advocacy and transparency, offering to demonstrate the system and granting aggregate-level access to all agencies as requested by the government. Nevertheless, to date, the mobile system has not replaced the paper-based system in the 5 prefectures where the program is active, and many contact tracers still use both systems.While government staff members were especially enthusiastic about the wealth of data that could be used for supervision of contact tracers, actual use of the data was limited. Local government managers often had many competing interests and responsibilities, and thus relied instead on supervisors to report issues to them rather than exploring the data directly. While the corrective action of hiring information officers to be paired with government staff helped to increase the use of performance data, utilization of CommCare data to inform transmission trends and contact risk patterns has still been limited. It is important to recognize that the contact monitoring data being collected by this program are only one aspect of the response. Because each aspect of the response and the associated data are managed by different partners , it is difficult to interpret broader trends from the contact monitoring data alone.As the data collected by CommCare contains protected health information as well as personally identifiable information, there were considerable concerns around data confidentiality. Although the Dimagi cloud server is compliant with the Health Insurance Portability and Accountability Act (HIPAA) and care was taken to ensure that data were shared in aggregate form, the national laws in Guinea around data privacy are still unclear. For this reason, datasets shared with partners have been mostly limited to aggregate indicators while government staff members have access to contact-specific details. Level of access has been determined by members of the NECU and facilitated by the implementing organization, which currently holds the end user license agreement with Dimagi per the government’s preference for ease of implementation. Formal transfer of data ownership to the government is in process and undergoing review by the NECU. There were also concerns from the government about data not being housed on a machine in country; for this reason a secondary local server was set up with a partner organization.Use of a smartphone-based mHealth application to assist with Ebola contact tracing activities has demonstrated the potential to improve access to surveillance data for informing response strategy. There is also opportunity to extend this type of methodology to other pillars of the Ebola response to improve rapid access to data and integration of key indicators in the response chain. While there are inherent challenges in introducing technology in complex emergencies, the benefits of innovation to disrupt the status quo may be integral to controlling long-standing epidemics. This being said, careful consideration must be given to whether it is feasible and beneficial to implement new technology during an ongoing outbreak. When the decision is made to implement technology, it is critical to accompany the deployment with close managerial oversight to quickly correct data inconsistencies and to address challenges.A formal government digital health information system currently does not exist in Guinea to allow for rapid health data collection and analysis, and thus the mobile contact tracing program cannot directly be absorbed into another system. However, the government and UN agencies in Guinea have expressed interest in using digital technology for longer-term surveillance, especially if embedded in a strong primary health care system. Certainly, the operational costs needed to bring a mobile health program to national scale would be substantial and have to be formally considered. Many of the resources and time required to set up this system were donated by partner agencies for Ebola-specific work. This initial investment in human resources, in-country technical expertise, and hardware for the program can certainly be leveraged and expanded to strengthen health systems.With proper technical capacity and oversight, mobile technology has the potential to expand access to critical data from remote communities needed to control an outbreak. As we have learned from this Ebola epidemic, it is essential to acknowledge and plan for stronger information systems that can detect and report health trends and aberrations. Accurate and readily accessible data can help health surveillance officers quickly identify and contain outbreaks. Furthermore, a strong information system can not only serve to prevent emerging epidemics but also be adapted and leveraged for activities such as contact tracing in the event of a disease outbreak. Thus, investments in strong community-based information systems should be considered as integral for strengthening health systems and emergency preparedness to prevent future epidemics as tragic as the 2014–2015 West Africa Ebola outbreak.


Aim: Interprofessional teaching and learning is gaining significance in the health professions. At the same time, the development and implementation of such educational courses is demanding. Focusing on factors critical to success, the aim of this paper is to evaluate the experience gathered by eight grant projects in which interprofessional courses were designed. Emphasis is placed on the level of cooperation between the participating educational institutions, course content, the operative implementation of the course units and their permanent integration into curricula.Method: Data was collected in semi-structured, guideline-based interviews with project leaders and team members (n=43). University and vocational students who had attended the evaluated courses were also included in the survey (n=7) as a means to triangulate data. Analysis was carried out based on qualitative content analysis.Results: A participatory, dialogue-centered model of cooperation appears to be most suited for developing and implementing courses. Belonging to the factors critical to success are the time when courses are offered, the conditions for attendance, the different teaching and learning cultures of the professions involved, preparation and deployment of instructors, and the role played by project coordination. Permanently integrating interprofessional units into medical curricula revealed itself to be difficult.Conclusion: While the development and realization of interprofessional courses can be achieved easily enough in projects, curricular integration of the new course units is challenging. In respect to the latter, not only a large amount of staffing resources and time are required, but also the creation of the necessary system-level structures, not just within the educational institutions  but also in the frameworks governing the professions. The Robert Bosch Foundation is currently funding eight projects involved in developing interprofessional education in the German cities of Berlin (n=2), Bochum, Greifswald, Jena, Lübeck, Mannheim and Munich. The professional socialization of future occupational therapists, midwives, speech therapists, medical doctors, nurses, and physical therapists is being addressed in a targeted manner through the joint development of interprofessional courses at universities and vocational schools. Doing this enables future health care professionals to experience collaboration in a multiprofessional team early on and to acquire the necessary skills and competencies. The project results also serve to anchor interprofessionalism in the standard curricula to sustainably establish the topic in each educational system.https://gesellschaft-medizinische-ausbildung.org/aktivitaeten/ausschuesse/interprofessionelle-ausbildung.html accessed on 5 August 2015]. Considering the institutional, structural and legal constitution of the health care system, interlinking the health professions appears very challenging. First of all, there are not only historically based asymmetries between the professions, but also legal ones , , .The eight model projects, to which the evaluation results refer, all differ in their focus, organization of the interprofessional courses, constellation of the target group, and project architecture. What they do share is a basic understanding of interprofessional learning as a process in which “two or more professions learn with, from and about each other to improve collaboration and the quality of care” , , , on the The methodological approach followed a mixed method design combining qualitative and quantitative methods of social research. The empirical approach first called for document analysis in which all available texts (primarily grant applications and concept papers) were assessed in terms of structure and content for each project. In a second step between February and March, 2015, guideline-based expert interviews (n=43) were conducted with project leaders and team members , 19]. T. T19]. TThe most significant evaluative findings are presented and discussed in the following section. These refer to the organization of the cooperation between the affected educational institutions (3.1), content design (3.2), permanent curricular integration (3.3), and the conducive and restricting factors concerning implementation of the interprofessional courses.The development and implementation of interprofessional courses does not simply mean bringing together students of different professions, but rather bringing together, in equal measure, the ideas and approaches of the various people and institutions involved in a project. In terms of the latter, this entails the people and members of a specific profession who are also employees of a particular institution ; as such, these people usually come to the project with two distinct agendas as experts in their fields with their interpersonal skills and as representatives of larger bodies with limited resources, institutional restrictions and overriding interests. Independent of organizational membership is the professional membership of those involved with their specific occupational perspectives, methods of working and interacting which are of importance when confronting challenges such as the ones posed by the projects under evaluation here.The implied potential for conflict and any requisite need for mediation at the cooperative level were perceived very differently by the project groups. In the following, two typical forms of ideal cooperation emerged to varying degrees in the projects and are compared here:Participatory discursive partnership: In six of eight cases there was equal cooperation between the participating institutions in which participation by all represented professions/institutions was observed as a guiding principle for all tasks and decision-making. All project phases  were accompanied by negotiations between the affected stakeholders. If key persons could be identified in this mode of cooperation, it did not result in predefinition or assignment of roles and functions to the other partners. Rather, the delegation of tasks and the assumption or assignment of responsibility was discursively coordinated within the context of the particular spheres of authority, resources and restrictions. Coordination points were mutually agreed on or jointly delegated.Instrumental partnership: The starting point in this case is a situation in which the form and content of the project was already defined to a large extent by a main stakeholder prior to the cooperative phase. The instrumental character of this form of cooperation basically lies in the fact that the main stakeholder needs a partner primarily as a means to implement his own ideas, which are not up for discussion. The instrumental nature remains in effect for assigning roles and delegating tasks and responsibilities. In the projects affected by this form of cooperation, instrumental partnerships mostly led to the main stakeholders taking on the tasks relevant to development, while the other partners were primarily occupied with the logistics of recruiting participants from specific professions. It cannot be stated here which form of cooperation is better. The evaluation showed that none of the eight projects were completely without problems; both modes have advantages and disadvantages. Ideally, cooperation based on dialogue may be for the most part coordinated and result in sustainable decisions, but it can also become a paralyzing model of consensus in the face of limited resources and a lack of pressure to act, or lead to a diffusion of responsibility. Rational decision-making and pragmatism fall under the functional aspect of efficiency as advantages of instrumental cooperation, while the extent to which joint goals can take on a generally binding character in this particular mode of cooperation remains open. Still, it is possible to discuss several critical aspects that coincide with a more or less pronounced form of instrumental cooperation:Difficulties in offering courses: Cancellations due to problems with recruiting students occurred almost exclusively in the projects with instrumental cooperation. It is possible that dialogue-centered cooperation is not only better able to yield an adequate recruiting strategy, but also to acquire, in equal measure, champions and advocates at the participating organizations thereby having a greater effect and generating a higher level of loyalty in the promotion of the courses. Likewise, it can be assumed that a discursive approach is more suitable for doing justice to the diverse organizational features of the participating institutions  in the early phase of project planning.Conformance with traditional role patterns: With the interaction of the various institutions in mind, it can be stated that the “medical dominance” of the health care system criticized in the discourse surrounding interprofessionalism was not perpetuated in many of the projects, but rather new forms of cooperation, even one on equal footing, were seen. However, there were examples, particularly in connection with more instrumental partnerships, of how a personal determination to accomplish something took a back seat to the determination of a profession to do the same and how the usual hierarchies reasserted themselves in the project’s structure. The descriptions of those surveyed do not paint a picture in which the physicians actively claimed leadership status for themselves. But rather, this was left to the medical schools, whereby the partners from non-medical professions repeatedly cited the reasons for this being the experience and ability of medical doctors to deal with third-party funding and project administration.Conflicts regarding project management and distribution of resources: Negative developments on the cooperative level were seen in exceptional cases, starting with the participating medical schools. This mainly involved the distribution of resources and opportunities to exercise influence on project management. Such conflicts endangered not only the coming together of the project in the application phase, but also affected the cooperative relationship and made operations difficult.Despite the wide variance in course content, meaning the response to precisely how interprofessional learning can be realized or interprofessionalism addressed, two general approaches can be identified; these are outlined in the following along with their implications .Thematic intersections as vehicles for interprofessionalism: What is meant here is the teaching of interprofessionalism using a subject of equal relevance to all the participating professional groups. Ideally, not only knowledge and decision-making skills should be acquired, but learning situations should also be created in which the students can also learn from, about and with each other. In respect to formal integration, this variant provides opportunities for linking the topics of existing curricular units (module combinations) across different study and training programs and the potential for longitudinal integration of multiple units at different points in the curricular programs. This becomes more and more difficult as the number of professions involved increases. In addition, “harmonizing” any asymmetrical knowledge among the professional groups is a challenge. Interprofessionalism as an explicit teaching topic: In this case, interprofessional collaboration itself is the main focus. To address it, completely new courses were developed, rather than combining existing modules or using common topics covered by different subjects. Naturally, expert input was important to design new courses, the core elements being team conflicts, dealing with stereotypes, becoming familiar with the responsibilities, skills and capabilities, limits and shared goals of the different health professions, communication techniques, informal interaction, etc. This implementation strategy simplified the participation of many professions. In respect to the potential for permanent inclusion or longitudinal integration, however, a sufficient number of interfaces are currently lacking in existing curricula.Regardless of the strategy chosen, care should be taken to avoid having the content relevant to one profession dominate; an “interprofessional balance” should be reflected in the content of the courses.A general problem seen in curricular integration is that not only medical degree programs, but also those of the other health professions, cover an extensive amount of material with individual subjects building off of and recurrently referring to each other over the entire course of study. Restrictions also arise from the rules and regulations governing each institution, such as those affecting training, study, testing, accreditation criteria, and enrollment capacities. Within this context, only one of the eight projects was successful in offering a required course that was mandatory for all of the participating professions. In three additional cases attendance was completely voluntary, whereas hybrid forms were seen in the rest of the projects. Almost all of those surveyed felt integration in the form of a required course was the ideal solution, but this was considered impossible under the existing circumstances by the medical schools. The reasoning behind this rejection most often involved the statute-based enrollment capacities and the much higher numbers of medical students in comparison to the smaller cohorts in the other professions that precludes any feasible interprofessional mix in the courses. Furthermore, scheduling and coordinating suitable periods of time within the various curricula proved to be a great barrier. Ultimately, those surveyed at the medical schools saw the introduction of a required interprofessional module as involving the impossibility of making compromises or cutting other curricular content, thus potentially affecting satisfaction of the medical licensing requirements.The experts interviewed were unable to say much about the continuation of the interprofessional courses at the time the interviews were conducted. This is due in part to the fact the projects are relatively new and the need for intensive coordination and staffing. Above all, the issue of continuation is constrained by legal restrictions , organizational and logistical challenges , and many other system-level barriers. At the administrative level, at least, long-term inclusion of interprofessional courses appears to be less of a challenge for the participating vocational schools where, despite state-defined curricula and legislation governing vocational training, administrators have both sufficient authority and the necessary leeway to make decisions about implementing changes at their institutions. The medical schools and university teaching hospitals differ from vocational schools in that they are much larger organizations with broader and deeper hierarchies of management. As academic institutions they also have a highly differentiated system of departments, are subdivided into numerous smaller units, and have complex decision-making processes in contrast to other post-secondary educational institutions.In the overall consideration of the projects, several factors can be identified as hindering or facilitating planning, implementation and offering the courses.Course attendance: Over the course of several projects it became clear that interprofessional courses can be negatively impacted if the joint learning sessions are offered on a voluntary basis for medical students but are mandatory for nursing students. The negative influence of differing attendance rules may especially play a role if the participating groups have not been informed about the particular context for this.Time point of the courses: When interprofessional courses can best be offered during training or education cannot be answered conclusively by the data collected for this paper. However, it was clear that scheduling courses in close proximity to exams was disadvantageous in terms of recruiting participants if the interprofessional courses did not cover exam-relevant content. Scheduling courses early or late in the day can also be viewed as a hindering factor, especially if participants have long distances to travel in order to attend.Different “learning cultures”: Different cultures of learning appear to play a considerable role in connection with the willingness to attend extra-curricular courses. This appears to have less to do with specific professions than with the socialization within an academic versus non-academic educational system. While vocational students find themselves in a highly defined and tightly focused educational system with alternating practical units, university students are more strongly required (and as a consequence much more used) to taking charge of their own education.Teaching staff: As direct conveyors of interprofessional learning content, instructors are the most important point of transmission between the conceptual levels of the project and its operative goals. Depending on the context, instructors were placed more in the role of moderator or more in that of classic teacher; they participated to varying degrees in designing the content of the units including involvement in the contextual inception of the project. In the expert interviews interprofessional teaching was repeatedly appraised as very demanding and the importance of extensive preparation prior to teaching was emphasized.Scheduling flexibility of the institutions: Finding suitable time slots within the curricula of the participating institutions revealed itself to be the single largest challenge for all of the projects. The degree of complexity increased in proportion to the number of professions involved. In connection with these coordination challenges, flexibility regarding scheduling is a crucial aspect for enabling organization.Project coordination: In all of the projects, coordinators fulfilled a central function connecting the operative and strategic levels and served as a contact point for the interfaces at the participating institutions. The administrative efforts associated with joint collaboratives between institutions  require appropriate staffing.The results of international studies confirm our findings and indicate the presence of further influential factors, primarily affecting learning outcome. Focus is on the importance of small-group work, a clearly articulated teaching strategy, an unthreatening learning environment, and enabling reflection on practical experiences .Considered as a whole, these eight projects clearly demonstrate the commitment and wealth of ideas with which interprofessional learning can be realized in a diversity of teaching formats. At the same time, the evaluation showed the effort connected with interprofessional teaching in the health professions and which organizational, institutional and legal barriers make its implementation so difficult, particularly in terms of longitudinal curricular integration. As a result, it was only possible in individual cases to implement curricular courses that addressed all students of the particular participating professions with the same degree of commitment.Without a doubt, the post-secondary vocational institutions and very specifically the university medical schools play a key role in the permanent implementation of interprofessional courses , 17], , . ApproprOverall, sustained integration of interprofessional courses does seem to depend solely on the effort of the project leaders and coordinators, but is just as dependent on higher hierarchical levels and decision-making bodies  that were not involved within the scope of these grant projects. At the political level frameworks must be negotiated that not only compel educational institutions to permanently implement interprofessional courses into curricula, but also ensure the provision of the resources and authority necessary to accomplish this. Within the context of the evaluation results seen here this appears to represent the greatest need for action in medical curriculum development, specifically granting interprofessional learning a lasting place in medical education.http://www.nklm.de] represent positive political signals for establishing structures encouraging innovation in and development of interprofessional teaching in the health professions.Approval of the position paper of the GMA Committee on Interprofessional Education in the Health Professions  and the We wish to express our gratitude to all the project participants for their willingness to share their experiences in the interviews.The author declares that he has no competing interests.


Mortality data are often unavailable, incomplete, and difficult to access for research and other purposes. Gaps in mortality data reports, particularly those detailing place of death, deprive healthcare professionals, decision-makers, and many others of the information that is needed to plan, implement, and evaluate interventions designed for purposes such as to assist people in achieving their preferred death place or reduce hospital utilization. Alternative methods of collecting reliable and valid data on death place may be needed. Our study primarily compared mortality data from a conventional information system  with mortality data collected using an unconventional system  over a 6-year period (2005-10) for the canton of Geneva Switzerland. Only a small average difference (4.8%) in death incidence was found. Death place data comparisons were also useful. This study suggests that the unconventional data from funeral homes provides reasonably reliable, valid, timely, and useful mortality data. Death place is increasingly being recognized around the world as having major public health importance . In manyA rapidly growing body of research on death place is evident, however. This research covers various fields of interest, such as a comparison of actual and preferred place of death , trends Despite this surge of interest in death place, there has been little research examining the quality of death information systems or mortality databases, and the usefulness of these data collection and management systems. Of note is one relatively recent study, conducted by This study was designed to compare mortality data  from the demographic information system of the Federal Statistical Office and an unconventional system  over a 6-year period (2005-10) in the canton (i.e. state) of Geneva Switzerland. The unconventional system was initiated after the 2003 heat wave when it became evident that a rapid means of detecting excess deaths was needed. Death place data comparisons were also undertaken, using available 1970-1986 Federal Statistics Office data (with 1970 and 1985 data reported), 2000 Cantonal Statistics Office data, and 2010 Geneva Observatory death data . The study was aimed at testing the reliability of this new unconventional system, and identifying the strengths and weaknesses of it, along with conditions for success.Data from the conventional sources were derived from state registries maintained by the Federal Statistical Office. Post-mortem procedure requires a death report from the physician or the police, the death certificate from the attending physician, the death certificate’s transfer to the family and funeral home, the encoding of data relating to various information on the certificate in an electronic file at the district level, and the transfer of these data to a centralized computerized national registry (i.e. Infostar) in Berne Switzerland . This procedure, illustrated in Data from the unconventional sources were obtained from an ad hoc newly created mortality observatory of the Geneva canton. This observatory was implemented in the aftermath of the 2003 heat wave, with contributors being the Institute of Social and Preventive Medicine , the Geriatric Policlinic (Policlinique de gériatrie), the Health Directorate (Direction de la santé), and local funeral homes. In the Geneva canton, three companies provide funeral services. The data collected were based on the death certificate issued by the attending physician or police officer in charge, a copy of which was delivered to the funeral home. By mutual agreement, each funeral home sent daily faxes with detailed information on these registered deaths to the observatory officer (the identities of the deceased were not disclosed). The officer in charge of registration at the observatory entered the data in an Excel file recording the following parameters, date and hour of death, name of the funeral home in charge of the funeral arrangements, year of birth, gender, place of residence, and place of death. The officer then entered the data into the mortality database of the Health Directorate . On a weekly basis, a senior officer at the Health Directorate checked the Excel file using the faxes’data as control. The entire procedure is illustrated in (a)a reduction in the proportion of deaths occurring at home,(b)an increase in the proportion of deaths occurring in medico-social institutions such as nursing homes, and(c)relatively minor changes overall in the proportion of deaths from 1985 to 2010 occurring in hospitals.Our study results show that the mortality data, and specifically the number or incidence of deaths, collected from the ad hoc mortality observatory  was relatively similar to the conventional information. The minor discrepancies between data sources in terms of death incidences can be accounted for by the following observations, (a) some funerals will not be handled by the three major funeral homes in Geneva, (b) data entry errors, and (c) some Geneva canton inhabitants die outside of the Geneva canton, either elsewhere in Switzerland or abroad. For example, it is known that during the last 6 months of 2004, approximately 23 deaths out of a mean monthly total of 245 (9.3%) that were recorded in Geneva canton actually occurred outside the canton . As suchThe comparative data on place of death were also revealing. The 2010 data collected by the Observatory from funeral homes was in line with what could be expected from conventional data sources, although with death location trends over time needing to be taken into account. This suggests that the ad hoc mortality Observatory is a good alternative method to collect place of death data, a matter of considerable relevance as the Swiss Federal Public Health Office no longer collects such information.The concordance of our data with the data collected from conventional information sources allows us to draw two primary conclusions on the representativeness and sensitivity of data collected using the unconventional information system. The data representativeness is largely assured because the three funeral home companies  have almost exclusive control of the Geneva canton funeral market . Data seIt should also be noted that other studies have confirmed the validity of mortality data based on funeral homes, coffin sales, and religious ceremonies in Malawi and Ethiopia - where official statistics are lacking—for the monitoring of mortality rates due to HIV/AIDS .The question of data reliability must be addressed by considering the competence of the involved institutions and the mechanisms implemented for data collection, transmission, and analysis of importance to consider. The following indicate considerable data reliability through the unconventional information system:(a)legal and technical competence, as the system was implemented on the initiative of the Cantonal Health Directorate in collaboration with the Institute of Preventive and Social Medicine at the Medical Faculty of the Geneva University, and the funeral homes;(b)scientific competence, as assured by the Institute of Preventive and Social Medicine, which was primarily in charge of designing, testing, and the critical analysis of the collected data; and(c)data coverage and representativeness was assured by the funeral homes, since these have information on almost all deaths occurring on cantonal territory.As such, one could indicate that the collected data were reasonably reliable. This raises the key question of the usefulness of such a system; particularly in times of health crises or emergency situations. Any system’s usefulness is largely dependent on its reactivity and time-saving quality. Whereas Swiss mortality data from conventional systems is only available on an annual basis and does not take into account place of death now, mortality numbers through the Observatory is available within 4 to 5 days, and with the place of death data also available for additional consideration. This time period could be shortened further in crisis situations.Also worthy of highlighting is the unconventional information system’s simplicity. This simplicity, and thus the relatively low cost to set up and maintain it, is largely attributable to the fact that the data collection, centralization, and diffusion circuits are the shortest and easiest possible. More specifically, the implemented procedures did not require major human, computer, or other logistic resources. The data were easily obtained by fax and easily entered in an Excel sheet, and were limited to a small number of variables . Where the conventional system relies on six steps, many of which involve formal or legislated processes, the Observatory system requires on only four in total, starting with the attending physician’s completion of the death certificate. These factors maximally limit the loss of data, data reporting lags, and data inaccuracies. The Observatory system could also be described as comprehensible and accessible to all . Also woIt should also be stressed that the unconventional system is cost-effective, as it is based on the voluntary and gratuitous participation of all involved participants, including the funeral home operators who kindly agreed to participate without any financial compensation. However, this lack of financial compensation must be considered as one of the system’s weaknesses; as any party, in the absence of any legal framework or well-understood incentive for participation, could delay or block the data collection. These difficulties are well-known in the field of collaborative work, as However, this unconventional system was readily implemented in this Swiss canton. The contributing factors for this success are notable. All involved actors, at the project’s initiation, mutually perceived a need for a tool using mortality data so that the data would function as an alert system. Recommendations highlighting this need were made at a 2004 conference designed to reflect upon the consequences of the 2003 European heat wave. One such recommendation was the implementation of a simple mechanism to collect mortality data in the Geneva canton . A seconRegardless, one major limitation of this specific unconventional data system is the limited number of mortality variables provided and thus recorded. As cause of death data were not obtained, this reduces the Observatory’s usefulness for epidemiological and other studies. This limitation was due to regulatory constraints  over technical issues, such as the system’s inability to collect such data.Another limitation is that this study suffered from a short observation period (2005-2010). This period was too brief to test the observatory system in vivo - such as in a real humanitarian crisis situation. In addition, it missed official data on places of death for 2010 comparative purposes. This lack of official standard reference data restricted us from conducting a more useful comparison between the two systems regarding place of death data accuracy and completeness.However, many studies on place of death lack methodological rigor . The quaOur study was designed in part to emphasize alternative models for collecting mortality data, particularly when the conventional information model has information gaps or hazards such that it reduces the ability to respond quickly in times of crisis. The unconventional information system  was shown to provide reasonably reliable, valid, and potentially useful mortality data. Although unconventional information systems cannot make the same claims of completeness as conventional systems, their “real-time” data should render them useful to decision-makers and opinion-leaders in periods of emergency, crisis, or disaster. They may also fill gaps in the collection, analysis, and publication of mortality data from conventional systems, such as information gaps on place of death. Having data readily available would address the considerable concern over the normal long delays in the publication or sharing of mortality data reports. Data delays deprive public health professionals, decision-makers, and opinion leaders of the information they need in order to manage public opinion, plan interventions, and implement short- to longer-term health action plans. Our study and these issues highlight the need for additional research on alternative unconventional information sources that would provide decision-makers and others with timely access to the information they need. Timely information is needed to enable them to understand the magnitude of a crisis, problem, or trend; assess and then determine what responses are the best under the circumstances, and then undertake necessary actions.


The use of mobile apps in health care is growing. Current and future practitioners must be equipped with the skills to navigate and utilize apps in patient care, yet few strategies exist for training health care professional students on the usage of apps.To characterize first-year pharmacy student use of medical apps, evaluate first-year pharmacy student's perception of skills in finding, evaluating, and using medical apps before and after a focused learning experience, and assess student satisfaction and areas for improvement regarding the learning experience.Students listened to a recorded, Web-based lecture on finding, evaluating, and using mobile apps in patient care. A 2-hour, interactive workshop was conducted during which students were led by an instructor through a discussion on strategies for finding and using apps in health care. The students practiced evaluating 6 different health care–related apps. Surveys were conducted before and after the focused learning experience to assess students' perceptions of medical apps and current use and perspectives on satisfaction with the learning experience and role of technology in health care.This educational intervention is the first described formal, interactive method to educate student pharmacists on medical apps. With a 99% response rate, surveys conducted before and after the learning experience displayed perceived improvement in student skills related to finding , evaluating , and using medical apps in patient care  and the health sciences classroom . Students described satisfaction with the educational experience and agreed that it should be repeated in subsequent years . Most students surveyed possessed portable electronic devices  and agreed with the concept of medical apps being an important part of the health care profession in the future .Student pharmacists recognize the key role technology plays in the future of health care. A medical apps workshop was successful in improving student pharmacists' perceptions of ability to find, evaluate, and use medical apps. With the worldwide popularity of mobile devices , mobile apps are increasingly being used by health care professionals in a variety of settings. A mobile device is defined as “a portable, wireless computing device that is small enough to be used while held in the hand” . In the Clinicians have access to mobile medical apps that serve as drug references, clinical calculators, disease references, and clinical decision-processing aides ,11. PharThe Food and Drug Administration (FDA) has recently released guidelines on which subset of medical apps it will review and regulate . WhereasFuture health care practitioners will likely be the generation to formally adopt mobile technology into their workflow and clinical practice. Studies have identified that mobile devices and medical apps are increasingly being incorporated into formal medical education -25. HoweFaculty from The Ohio State University College of Pharmacy (OSU COP) and Massachusetts College of Pharmacy and Health Sciences (MCPHS) University School of Pharmacy collaborated to create a novel learning experience for pharmacy students that focused on finding, evaluating, and using medical apps. The project was approved as exempt research by The Ohio State University Institutional Review Board. At OSU COP, foundational concepts related to drug information, including literature assessment, are taught in the spring semester of the PharmD curriculum as a distinct module that is part of an introductory pharmacy practice course series. During this module, students are engaged with didactic, workshop, and project-based experiences focused on finding, evaluating, and using drug information in patient care. This medical app educational experience was integrated into this course. This two-part learning experience involved a prerecorded, Web-based lecture that students were required to view before arriving for a 2-hour, small group workshop of approximately 30 students to apply concepts learned in the lecture. Available for 1 week before the workshop, the 38-minute lecture was created by the faculty member from MCPHS, video recorded via QuickTime, and uploaded onto a private view YouTube profile made available to OSU COP students. The lecture content was based on the role of mobile medical apps in clinical practice, issues and opportunities with utilization of medical apps, and how to review and assess medical apps. The lecture demonstrated to students how to evaluate mobile medical apps with several examples provided. In addition to the lecture, a tool for evaluating medical apps was shared with students; they were required to bring a copy of this tool with them to the workshop. This tool was developed by one of the coinvestigators and previously published incorporating previous strategies published ,27.Prior to the workshop, students were sent a list of medical apps that could be used free of charge and were available for download to portable electronic devices with iOS or Android operating systems, such as mobile phones or tablets. Students were asked to download these apps to a device, if available, before the workshop. Medical apps were selected by faculty and then reviewed before the start of the workshop. Identification of apps was conducted via previously identified means , with anWhen students arrived to class, a discussion was led by the instructor to review key concepts covered in the previously posted Web-based lecture, including how to use the medical app evaluation tool. The discussion also involved a facilitated conversation about where students find apps and students' experience with apps that had been useful or not useful. Students were split into groups of 4 to 5 students each  to evaluate up to 6 medical apps. Each group presented their experience using the evaluation tool to the class and discussed what the individual groups had found with a focus on challenges in evaluating medical apps as well as how to determine what apps are useful in practice.Metrics were collected from YouTube Analytics on the number of views and visits received before the start of the workshop. To evaluate the effect of these medical apps on learning experience, prospective surveys were conducted before and after the involvement of first-year pharmacy students in the learning experience see  and . ThBoth surveys were conducted during the workshop as paper surveys. The first survey was conducted on the first day of workshop for the semester. The second survey was administered during the workshop 2 weeks after the medical apps workshop experience.The first-year class includes a total of 120 students. There were 96 visits to the YouTube video before the start of the workshop. From the first-year pharmacy class of 120 students, 119 students completed the first survey whereas 120 students completed the second survey. Differences in response total were due to 1 student being absent from class the day the first survey was conducted. Demographics of students gathered are shown in On the basis of the results of the first survey, most students learned about apps via word of mouth, including obtaining information from classmates , social media such as Facebook, Twitter, and blogs , pharmacy staff at work , and family  of students indicated currently having a drug information app installed on a portable electronic device .Before the workshop, most students indicated that medical or pharmacy-related apps are beneficial to pharmacy practice  and that mobile technology will influence pharmacy practice in the future . However, less than 44% of students agreed or strongly agreed that they knew how to find , evaluate , or use medical or pharmacy-related apps . Approximately 44% of trainees indicated apps for mobile phones as being very useful in clinical practice . Our phaAlthough our study demonstrated that students felt more comfortable with evaluating mobile medical apps, our results indicated that they still felt they would benefit from greater knowledge on when it is appropriate to use the technology in practice. Although this was not a focus of our study, our results suggest possible benefit from addressing the topic of e-professionalism and the integration of mobile devices into practice in pharmacy and possibly other health sciences curricula.Limitations of this project relate to the narrow scope of the educational intervention. This intervention occurred in one course at one pharmacy school with first-year pharmacy students who have not yet experienced patient care at the level of more advanced students and may not be ready to apply the experience to real-life medication management in practice. Although the faculty attempted to choose a variety of medical apps to evaluate in one workshop session, we were able to accommodate up to 6 total apps. A greater variety may have allowed for a more in-depth discussion through application to core concepts. Approximately 80% of the class viewed the preparatory Web-based training module before the workshop, though it cannot be determined if each video view was conducted by each student alone or in groups. Despite this, data were analyzed in aggregate and showed overall improved perceptions by the majority of the class. Another limitation to considering the effect of this experience is that faculty evaluated student perception of skills with no formal assessment of student abilities. Data on students' general use of digital technology, including social media, were evaluated as demographic information to describe the population and not assessed in the second survey; thus, investigators are unable to identify any effects this learning experience may have had on this digital technology usage. Additionally, unique identifiers were not collected to match each student's responses for the first and second data collection. Consequently, no formal hypothesis testing for the changes before and after the learning experience could be performed and analyses of the data were limited to descriptive summaries.Medical apps will be an inevitable component of health care in the 21st century. Pharmacy and other health care professionals must be equipped with the skills to navigate this new open-access world to provide safe and effective recommendations and care of patients. An important element of the portable technology world is medical apps. This paper describes a model for engaging pharmacy students in an active learning experience in finding, evaluating, and using medical apps. This model is transferable to other colleges of pharmacy as well as other health care professional training programs aimed at both current and future practitioners.


To identify risk factors associated with the duration of hot flashes and the time of peak hot flash severity in mid-life women.A cohort of 647 women reporting hot flashes were followed for 1–7 years, with survey data and hormone measurements. Survival analysis determined the association of risk factors with the duration of hot flashes. Linear regression determined the association of risk factors with the time of peak severity. Final models were determined through stepwise model selection.Average hot flash duration was 2.5 years (range: 1–33), with peak severity on average at 2.96 years (range: 1–20). Duration of hot flashes was associated with race, education, menopause status, smoking history, BMI, alcohol consumption, leisure activity levels, and levels of estradiol and progesterone. In the final model, only race, alcohol consumption, leisure activity, and menopause were retained. White women had significantly shorter hot flash durations than non-white women. Women consuming at least 12 alcoholic drinks in the previous year had a significantly shorter duration of hot flashes with a smaller effect of hot flash duration on increasing in time to peak severity compared to those who consumed less than 12 alcoholic drinks in that year. Higher serum progesterone levels were associated with later peak severity if the duration of the hot flashes was less than 2 years and an earlier peak severity otherwise.These results suggest that some behaviors  are associated with shorter durations of hot flashes, and that progesterone was associated with the dynamics of hot flash severity. Approximately 75% of perimenopausal women experience hot flashes; this perThe existing knowledge on the risk factors of hot flash dynamics is limited. Most previous studies of hot flashes have been cross-sectional–10, and The purpose of this study was to examine the duration of hot flash symptoms in US women in mid-life, and to identify risk factors for longer durations. A secondary purpose was to examine the time to peak severity of hot flash symptoms and identify factors affecting that time. By studying the relationship between risk factors and changes in symptoms, we will be able to identify actions that can be taken before and during the menopause transition to improve clinical outcome.All participants gave written informed consent according to procedures approved by the University of Illinois and Johns Hopkins University Institutional Review Boards. University of Illinois and Johns Hopkins University Institutional Review Boards approved this research. The study design for the parent study is described in detail elsewhere. Briefly,During the baseline survey, women were asked if they had ever had hot flashes, with a description of hot flashes provided. They were also asked how old they were when they first experienced hot flashes, how old they were when the hot flashes were most severe, and the severity of the majority of their hot flashes. During annual follow-up visits, women were asked if they had hot flashes since their last visit, and were again asked the severity of the majority of their hot flashes. In terms of severity, each woman was asked at each visit to describe her hot flashes as: mild (sensation of heat without sweating), moderate (sensation of heat with sweating), or severe .Serum samples extracted from the collected blood samples were used to measure estradiol, and progesterone levels in each sample using commercially available, previously validated enzyme-linked immunosorbent assay (ELISA) kits .–19 The mOnly women who reported hot flashes at any point during the study were included in this analysis. Duration of hot flashes was determined by the difference in the age at first hot flashes (self-reported) and the age at which the woman first reported no longer experiencing hot flashes. Specifically, the end of hot flashes was assumed to occur in the year prior to the first year a woman reported no hot flashes. If women later reported a resumption of hot flashes, the duration of hot flashes was calculated to include the full length of time, including the years in which no hot flashes were reported. Women who reported an end to hot flashes were interval censored; women who did not report an end to hot flashes during the study were right-censored. Parametric accelerated failure time models were fit based on Weibull and exponential distributions, with the best-fitting model  used as a base for all other analyses.Risk factors considered were education level, race, history of pregnancy, menopause status, smoking history, BMI, alcohol use , relative activity level during leisure , relative activity level during work , estradiol levels, testosterone levels, and progesterone levels. A combined variable for total exercise, being the sum of the leisure and work activity levels, was also considered; for women not reporting work activity levels, the leisure activity level was doubled as it was assumed that the women spent twice the amount of time at leisure. Race, education, and history of pregnancy were considered constant variables, and were determined by the first survey. All other variables were considered time-dependent, and were allowed to vary between years as determined by follow-up survey responses. Continuous variables were tested for normality and transformed if normality assumptions failed to hold. The counting method, with a frailty term for each woman to account for non-independence of responses within subjects, was used to account for time-dependency.Univariate models were fit for all risk factors, and those with p≤0.1 were included for consideration in the multivariate model. BIC-based stepwise model selection was applied, in which for each step the single change (addition or subtraction) that produced the model with the lowest BIC value  was accepted. All main effects and interactions considered. All models were fit with the survival package in R, accessedTime to peak severity was taken as the difference between the age at which hot flashes were most severe and the age at which hot flashes were first experienced. Peak severity was assumed to be the age at which hot flashes were reported to be most severe during the baseline visit, unless a woman later reported a higher severity than reported at the baseline visit; in that case, the age at which the woman first reported the higher severity was considered to be the peak. Peak severity varied in intensity, with some women having only mild or moderate hot flashes during their peak. All risk factors considered  consisted of their baseline values, with no time-dependent variables. The duration of the hot flashes, as calculated above, was also included as a risk factor. Linear regression models for the time to peak severity, in years, were fit with the glm function in R, with model selection procedures as described above.A total of 238 women reporting hot flashes also reported the end of those hot flashes during the study, with 482 reporting the time of their first hot flashes, out of 647 women reporting hot flashes in the total cohort of 780 women. The number of women enrolled in the study and participating in each year is demonstrated ; at timeProgesterone and estradiol levels were log-transformed to meet normality assumptions. Race, education level, menopause status, smoking history, BMI, alcohol use, relative activity during leisure, exercise level, progesterone levels, and estradiol levels were all significantly related with the duration of hot flashes . For alcIn the final multivariate model, no interactions were included; smoking, BMI, exercise level, and progesterone and estradiol levels were removed during the stepwise selection process. The results of the final model are shown in The effect of menopause status on hot flash duration  is an artifact of the counting method; 39% of premenopausal data points are censored (35% experiencing the end of hot flashes in postmenopause and the remaining experiencing the end of hot flashes in perimenopause), compared to 17% of perimenopausal and 10% of postmenopausal see , and posIn the study, 15 women experienced hot flashes for more than 20 years. A subset analysis of the data excluding those women produced similar results for race, alcohol use, menopause status, and leisure activity. The final model also included progesterone and an interaction between progesterone and menopause status; increases in progesterone decreased duration in premenopausal women and increased duration in peri- and post-menopausal women.Hot flash duration, BMI, alcohol use, race, and progesterone levels were all significantly related with the time to peak hot flash severity. In the final multivariate model , BMI andWomen who consumed at least 12 alcoholic drinks in the previous year had later peaks in hot flash severity than those who did not if the duration of their hot flashes was less than 3 years; women experiencing hot flashes for longer than 3 years had earlier peak severity if they reported ≥12 drinks consumed in the previous year.In the subset analysis excluding women who experienced hot flashes for more than 20 years, the relationship of time to peak severity with hot flash duration and alcohol use was similar. Progesterone levels were no longer included in the final model. Race was included in the final model, with white women having significantly earlier peak severity.This study has found that the majority of women do not experience hot flashes for long periods of time and that certain risk factors are associated with longer hot flash duration that have not been previously identified. Race was associated with the duration of hot flashes, as it was with their likelihood, frequency, and severity:,23 Non-wBesides the demographic risk factors, there were lifestyle choices associated with the duration of hot flashes, and consideration of these factors may help identify interventions or protective measures to be studied in perimenopausal women. We found that some amount of alcohol consumption  resulted in significantly shorter hot flash duration in women. Another study found that high alcohol consumption (>16 drinks per week) was associated with increased vasomotor symptoms, but this was compared to <10 drinks per week, and so was comparing heavy consumption with lower consumption, while inAlthough smoking was not retained in the final model, in the univariate analysis current and former smokers both suffered from longer periods of hot flashes than non-smokers, but former smokers were numerically (if not significantly) better off than current smokers. Therefore, quitting smoking was associated with shorter duration of hot flashes as well as less frequency and severity. In additLeisure activity was significantly correlated with a number of other variables , and so may have been reflecting some of the effect of a number of these. Exercise has been recommended as a method for alleviating hot flashes, but the evidence has been mixed. Our resuThe model for time to peak severity found that a longer duration of hot flashes delayed that peak. However, alcohol consumption was associated with a smaller effect of the duration of hot flashes on the time to the peak severity. Both effects (shorter duration and earlier peak) may be caused by a similar process, although again the biological mechanism is unclear. Higher serum progesterone levels were associated with decreased time to peak severity in women with longer hot flash durations; low progesterone levels have been found to be associated with risk and severity of hot flashes, possiblyOne limitation to the ability of this study to understand dynamics is the periodic nature of the surveys. The duration of hot flashes is short in most women, with a highly right-skewed distribution: the median of the upper end of the duration interval in this study was 2, and the mode was 1. As such, fine differences in durations were not detectable. However, the long tail of the distribution enabled us to determine risks for the larger differences which may be more clinically relevant. The use of interval censoring improved that ability, as well, by accounting for our inability to determine the exact event time. The study is also limited by the self-reporting of severity, as a woman may change her definition of “moderate” or “severe” hot flashes as her symptoms continue. It is also possible that the study suffered from some amount of selection bias, as women undergoing treatment for their hot flashes (who might be assumed to be suffering from more severe symptoms) were excluded from the study; however, many women in the study reported severe and frequent hot flashes, so this bias is not evident. Recall bias could also affect the results of this study, as many women were suffering from hot flashes prior to the study onset and had to remember the year in which their hot flashes started; this bias is unlikely to be systematic and so should not affect the results. Despite the limitations, this data set has allowed for the most detailed analysis of hot flash dynamics to date.One other possible limitation to these data was the high rate of censoring for the duration analysis. To determine the potential bias of the high censoring observed, sensitivity analysis was performed by repeating the analysis under 2 assumptions: that all women experienced an end to hot flashes before their last visit, or that all women not reporting an end to hot flashes experienced the maximum duration of hot flashes. Under these analyses, little change was seen in either univariate or multivariate models; therefore, we may conclude that the high rate of censoring did not bias our estimate.This study has found that there are both demographic and behavioral risk factors associated with longer duration of hot flashes in perimenopausal women and later peaks in severity. This information may be useful to clinicians advising women on what to expect with hot flashes and potential interventions to limit their duration and severity.


This study assesses the effects of maternal healthcare on child survival by using nonrandomized data from a cross-sectional survey in Azerbaijan. Using 2SLS and simultaneous equation bivariate probit models, we estimate the effects of delivering in healthcare facility on probability of child survival taking into account self-selection into the treatment. For women who delivered at healthcare facilities, the probability of child survival increases by approximately 18%. Furthermore, if every woman had the opportunity to deliver in healthcare facility, then the probability of child survival in Azerbaijan as a whole would have increased by approximately 16%. Poor child outcomes are usually associated with underutilization of maternal healthcare –3. GivenThe contribution of the presented study is that it attempts to fill the gap in the existing literature by quantifying the direct effect of delivery in healthcare facility on probability of child survival. The robust evaluation of program effect on population usually involves randomized control trials (RCT). In many cases, including evaluation of maternal healthcare, conducting a RCT is not possible from an ethical perspective, withholding vital service, and from technical perspective, lack of money and time required to conduct a countrywide RCT. To overcome these difficulties, we assess the effect of healthcare and homecare on child survival by using quasiexperimental evaluation of nonrandomized data from a cross-sectional survey. In this way, this study contributes to the recent discussion on appropriate methods for the evaluation of effect of healthcare programs when RCT is not feasible –12.Azerbaijan, a low-income transitional country on Caucasus, is an interesting setting for examining the above-mentioned issues for several reasons. First, Azerbaijan has the highest infant mortality rate and one of the highest proportions of child deliveries outside of healthcare facilities even compared with other transitional countries in the region . Second,We are guided by Mosley and Chen's  framewor biprobit command in Stata software package [We are interested in estimating effect of treatment, having child delivery at a healthcare facility, on the outcome, probability of child survival. Thus, we face a problem of self-selection—the sampled individuals who receive the treatment are different from those who do not receive it in unobservable ways which are also simultaneously correlated with outcome , 12. To  package . After b package . The val package  and Wool package .This study uses data from the 2006 Azerbaijan Demographic and Health Survey (the AZDHS). The AZDHS is conducted by the national statistical authority, the State Statistical Committee of Azerbaijan, with technical assistance of Macro International, USA, and with financial support from USAID and UNICEF . The AZDThe outcome variable of this study is child survival defined as probability to survive during 60 months or 5 years. This variable is binomial; it takes the value of 1 if the child survives 60 months and takes the value of 0 if otherwise. There are two endogenous instrumented variables of interests which denote treatment and serve to gauge healthcare input. The instrumented treatment variable is “delivery in a healthcare facility” that takes the value of 1 if the child was delivered in a healthcare facility and takes the value of 0 if otherwise. The healthcare facility is defined as a government or private hospital, maternity home, polyclinic, woman's consultation, and primary healthcare posts. Overall, from the sample of 2,285 women who answered the questions about place of delivery in the AZDHS, approximately 79% delivered babies in a healthcare facility.The instrumental variables used to estimate the endogenous treatment variables are taken from a previous study that used instrumental variables to estimate the effect of prenatal healthcare utilization on child birthweight in Azerbaijan . There aThe exogenous variables used to explain child survival are taken from the previous studies on the determinants of child mortality conducted in the countries of the region of Caucasus and Central Asia , 21, 22.J statistic is employed to evaluate overidentifying restrictions. If Hansen J statistics cannot reject the null hypothesis, then the selected instrumental variables are exogenous. In addition, Kleibergen-Paap LM statistic is used to test the adequacy of the instruments. If the test rejects the null hypothesis, the instruments are adequate to identify the equation. Lastly, we conduct Durbin-Wu-Hausman test for potential endogeneity. The significance of the test confirms the presence of endogeneity and suggests that estimation of equations without taking into account endogeneity will lead to biased results. All the above-described tests have been passed in all estimated models.We commence with 2SLS model because the tests for overidentifying restrictions and the adequacy of the instruments are readily available for the 2SLS but not for biprobit , 12, 32.Next we estimate biprobit which is more relevant model due to the binary nature of outcome and treatment variables. A straightforward Wald test of endogeneity is available in biprobit. If result of the test is significantly different from zero, then biprobit should be estimated due to the presence of endogeneity. In all estimated models, the Wald tests have confirmed endogeneity. After biprobit model estimation, ATE and ATT are computed and reported.The results are reported in ATE and ATT are reported in the last two rows of the table below biprobit estimations. ATE suggests that delivering in a healthcare facility improves the chances of child survival for a woman randomly drawn from the population by approximately 16%. In comparison, ATT suggests that delivering in a healthcare facility improves the chances of child survival for women who actually participated in the program by about 18%.In this study, we identified and then attempted to fill the important gap in the literature regarding the effectiveness of maternal healthcare in reducing under-five child mortality in the region of the Central Asia and the Caucasus. We assessed the effects of delivering in a healthcare facility on child survival by using a quasiexperimental evaluation based on nonrandomized data from a cross-sectional survey in Azerbaijan, a low-income country in transition. The empirical evidence presented in this paper allows for drawing several conclusions. raison d'être for maternal healthcare programs, such a funding could be expected. However, we were able to confirm that the effect of delivering at a healthcare facility on child survival is statistically significant on the national level. We also quantified the positive effect of such treatment. For women who delivered at healthcare facilities the probability of child survival increases by approximately 18%. Furthermore, if every woman in Azerbaijan had the opportunity to deliver in a healthcare facility, then the probability of child survival in the country would have increased by approximately 16%. These findings suggest that utilization of maternal services in transitional countries should be encouraged and promoted in spite of the limitations and deficiencies in the current maternal healthcare system.First, delivering children in healthcare facilities increases the probability of survival. Since reducing child mortality isSecond, our study demonstrates that the wealth gradient is an important barrier for utilization and hence influences the child outcomes. Since maternal healthcare is officially free, the prior studies explained the effect of wealth gradient by high level of unofficial out-of-pocket expenditures for healthcare personnel, supplies, and medication , 33, 34.Third, our study demonstrates that the risk of not delivering at a healthcare facility increased for less educated women. Women with higher education are strongly associated with delivering in medical settings and hence with higher chances of child survival. Habibov  reportedFinally, the population based nationally representative surveys such as the Demographic and Health Surveys by Macro International and Living Standards Measurement Surveys by the World Bank became an important tool for measuring policy effect on health outcomes in many transitional and developing countries , 33, 34.