The National Cancer Institute (NCI) was among the first federal agencies to recognize the potential of the Internet for disseminating health-related information. The evolution and refinement of NCI's online cancer information has been substantially “user driven”—from the launch of CancerNet in 1995 to the recent redesign of its award-winning successor, the NCI website. This article presents an overview of NCI's multi-pronged approach to gathering input about its online information products, including stakeholder meetings, focus groups, standard and customized online user surveys, usability testing, heuristic reviews, and search log analysis. Also highlighted are some of the many enhancements that have been made to NCI's online cancer information products based on user input. The National Cancer Institute (NCI) was among the first federal agencies to recognize the potential of the Internet for disseminating health-related information, and it launched its CancerNet website in 1995. This site was a natural extension of NCI's information dissemination efforts, which have been carried out in response to mandates from Congress in the National Cancer Act of 1971 [®) cancer information database, which contains information summaries on numerous cancer-related topics and a cancer clinical trials registry. PDQ was available to medical librarians, physicians, oncology nurses, and other professionals through the National Library of Medicine's online information system [A large part of NCI's pre-1995 information dissemination efforts was targeted at health professionals through the Physician Data Query  that the NCI website avoid segmenting information pathways according to type of user ; (2) that information be customized to provide varying degrees of technical detail, complexity, and reading level; and (3) that users be able to easily move between these levels. It was also recommended that the NCI website integrate clinical trials information with the full spectrum of cancer information; include information about clinical trials, patient rights, and the informed consent process; and include a feature covering news topics related to clinical trials.One of the outcomes of this meeting was the development of a new NCI website, cancerTrials, to provide an educational context for the PDQ clinical trials registry that was offered on CancerNet. The cancerTrials website was launched in 1999. In addition to guidance on how to search the PDQ registry, visitors to the new site were offered original articles explaining what cancer clinical trials were, how they worked, and where to find them. They also were offered brief summaries of recently announced cancer trial results and other timely news related to the US clinical trials system.The subsequent redesign of CancerNet in 1999  carried Information specialists at NCI's Cancer Information Service (CIS) are the front line of NCI's interactions with the cancer community, particularly the public . ThroughA critical factor in achieving continuous improvement of NCI's Web resources is soliciting user feedback to learn what works, what doesn't, and where gaps in information or functionality exist. In preparation for the 1999 redesign of CancerNet, an online survey asked users to identify the information they were seeking , difficu                    Feedback from the online survey, along with input from the Chantilly meeting, guided the redesign of CancerNet in 1999. On the redesigned site, users could start with the “Types of Cancer” page , which e                    In 2002, NCI's overarching website was redesigned, and the CancerNet and cancerTrials websites became the Cancer Information and Clinical Trials portal areas of the redesigned site. In 2004, the NCI site underwent another redesign, once again guided by extensive evaluation and user input.NCI's early decision to provide information tailored for patients and their families continues to be supported by surveys conducted during the past five years. Data from 1999 showed that 44% of visitors to the site described themselves as cancer patients or family members or friends of a cancer patient. Data from the American Customer Satisfaction Index (ACSI) survey posted on the NCI website in 2004 showed that more than 50% of respondents identified themselves as cancer patients or family or friends. The next largest audience in 2004 was health care providers, about 13%. NCI continues to keep the patient at the center of many of its online resources—PDQ's cancer information summaries and clinical trial abstracts, clinical trial results summaries, fact sheets and other information products, and the website's dictionary are all written for lay audiences.Both before and after the 2004 redesign, the website displayed the ACSI survey . This suThe ACSI methodology provides continuous online feedback and is a uniform, national, cross-industry measure of customer satisfaction. A core set of ACSI questions measures overall satisfaction, and customized questions can be added regarding individual websites or pages.Data from the survey are helpful in supporting or dispelling impressions of who uses a site and what their information needs are. For example, data from the 2004 ACSI survey showed that approximately 57% of NCI's website visitors are first-time users of the site. This underscores the need for intuitive site structure and navigation tools that can be easily grasped by users with no prior knowledge of the site. Multiple paths to core information, such as cancer-specific home pages and clinical trial search tools, were created in 2004 to help new users easily find the most sought-after information. While we cannot make a direct correlation between these enhancements and increased customer satisfaction, the ACSI survey results published in December 2004 named the NCI website the “best in customer satisfaction” in the portal/department main site category . OverallUsability testing helps ensure that products and services address the needs and interests of website visitors ,12. In lSearch log analysis played an important role in the 2004 redesign of the NCI website. Each year, users enter approximately 2.5 million free-text searches in the basic search box on the site. More than 50% of searches are for types of cancer or specific body systems or locations. To give visitors immediate access to information on the most common cancers, prominent links for each of these cancers were added to the site's home page, along with multiple links to an A to Z list of cancers to enable easy information retrieval . The sam                    When the NCI website was redesigned in 2002, the site's search tool was supplemented with a “Best Bets” feature that gives users a concise list of editorially selected NCI sites and pages that are displayed above the full set of search results. Whereas the full set of search results, which are generated by a free-text search of NCI's Web content, can number thousands of documents for a given search term, the Best Bets offer an average of two links, with a range of one to 18. There are currently 677 Best Bets categories  with selected Spanish-language categories included.To populate Best Bets initially, a team of information experts identified cancer-related information categories, selected the most relevant NCI sites and pages for each, and created a table of related terms for each category name. When a search term is entered in the search box on the site and a category name or related term matches the term or any part of the term, the associated list of Best Bets is displayed.Search log analysis after the launch of the Best Bets feature in 2002 validated the choice of category names and related terms, the majority of which proved to be among the more popular search terms. Since 2002, the Best Bets database has been edited by NCI staff as needed, on the basis of periodic analysis of search logs and knowledge of new and changing NCI Web content. Log analysis has prompted a considerable expansion of the Best Bets database by suggesting new category names and related terms. In addition, there have been a few instances in which the large number of searches on a topic indicated the need for new content. These findings have already led to the creation of two important pieces of content , namely a fact sheet about cancer staging and a substantial resource on the NCI website called the “Tobacco and Cancer” home page. Best Bets categories, related terms, and links have also been added in response to comments from users.In the summer and fall of 2002, NCI initiated a multi-pronged review of the Clinical Trials portal of its website  to deter                    The 2002 evaluation used six qualitative and quantitative methods:key informant interviews with NCI staff integral to the development of the portal.The initial phase of the evaluation involved diary activity was conducted to capture feedback from users who were representative of three of the portal's target audiences, including patient advocates, oncology nurses, and CIS information specialists. Participants were asked to complete a written, formatted diary entry for each visit they made to the portal in the course of their regular activities over a period of one month.A Focus groups and in-depth interviews were later conducted to gain more feedback.                                online surveys were posted in the Clinical Trials portal of NCI's website. A general survey was presented to each user who visited any page of the portal except for news summaries. A news summary survey was presented to users who visited news summaries. Session cookies were used to recognize possible repeat visitors and to serve up the survey once per visitor during a 30-minute time period (to minimize both the burden on the public and duplicate responses). Once duplicate responses were eliminated, the adjusted survey sample contained 1589 general survey responses and 207 news summary survey responses. Two Usability testing was conducted to determine whether users could easily find and understand the news summaries. Perceived usefulness of the news summaries was also explored in usability testing with six participants.                                log analysis software to collect the following usage statistics: unique visitors, visitors who visited once, visitors who visited more than once, sessions, median visit length, page views, and visits from referring sites.Server log file entries were analyzed using WebTrends Several key findings emerged from these evaluation methods . The topWith regard to the Clinical Trials portal, most users found the information they needed, were able to understand it, and found it useful. However, they had difficulty finding their way to the Clinical Trial Results news summaries, even though this type of information was among the top three categories of information desired. When directed there (or when identified as having been there via the pop-up exit survey), users found the summaries to be useful, understandable, and well organized.These findings were used to inform changes to the content and design of the Clinical Trials portal over the course of 2003 and again during the 2004 redesign of the NCI website . For exa                        To further help visitors locate specific trials in which they might be interested, a new section was created called Featured Clinical Trials . This seIn addition, links to the Clinical Trial Results pages organized by type of cancer are more prominently displayed on the Clinical Trials portal home page, and teasers (brief description and link) for the two most recently posted Results articles are prominently displayed on the Clinical Trials portal home page.The PDQ clinical trials registry has been a key component of NCI's online cancer information services from its inception in the 1980s . OriginaSince its appearance on the Web on CancerNet, the complexity of the clinical trials search form has been a topic of discussion within NCI. The PDQ clinical trials registry began as part of a DOS-based, menu-driven system used almost exclusively by health professionals, medical librarians, and cancer information specialists. Developers were wary of transplanting the sophisticated search functionality of this system to NCI's website because many of the site's visitors had little familiarity with clinical trials, cancer staging, treatment choices, and other elements in the original system. Initial methods of clinical trial searching on NCI's CancerNet website included a form with limited search options and clinical trial descriptions written in technical language, a legacy from the original system. Simply written, patient-friendly descriptions of clinical trials were introduced in 1997.The second-generation search form that was launched in 1999 was based on recommendations from the Chantilly meeting, data from an online feedback form on the website, analysis of the search form, and personal interviews. In addition, a prototype of the form was developed through iterative rounds of usability testing. A two-step search form was designed to allow users to search by common search parameters, such as type of cancer, type of trial, and geographic location. Users could then review their search results or choose to narrow their search with other parameters, such as stage of cancer, drug (including brand and generic names), type of treatment, and trial sponsor. Users were also given the option of viewing two descriptions of each clinical trial, one for patients and one for health professionals.Other changes to the search form based on usability testing included a user's guide for less experienced users, annotated labels for search parameters with links to more detailed explanations, and explanations of how to select multiple items per field .In the 2002 redesign of the NCI website, the clinical trials search form was included in the Clinical Trials portal of the site, giving users a more integrated information pathway that grouped information such as patient safety, informed consent, and insurance issues with the listing of clinical trials.A major consideration in this redesign was the addition of a specific new group of users—information specialists from the CIS, whose duties include assisting patients, their families, and health professionals in identifying clinical trials of interest. Information specialists had previously used the DOS-based, menu-driven PDQ search system that allowed them to perform complex searches, review results, refine as needed, and then prepare an “information packet” that could be emailed or mailed to callers. Web designers visited a CIS regional office to understand the needs of this group of users and did extensive usability testing with them prior to launching the revised form.Given the diversity of users, it became clear that a single search form was not ideal. Some users found the detailed choices on the form confusing and beyond what they needed. An interactive search form that guided users through the search process was considered, but such a form would require JavaScript, which does not meet Web accessibility requirements for federal government websites. It was determined that the best approach was to develop two search forms with different levels of complexity. Web accessibility requirements could be satisfied by creating one form without JavaScript, and a more complex, interactive form could be created with JavaScript.The basic search form, designed for the patient, caregiver, or busy health professional, provided three search options—cancer type, type of trial, and zip code proximity. The results were also presented in a format more suited to the casual Web user, who was accustomed to clicking on a search result link to go to a page that contained more information. Usability testing had also indicated that users did not normally click on the check boxes that were provided with the search results in order to prepare a “package” for viewing or printing as a batch.The advanced search form  was JavaIn addition, for the CIS users, the search results display was developed to enable information specialists to read a preliminary result set, so they could identify the most appropriate trials for their callers and prepare an “information packet.”The 2004 redesign of the NCI website saw further changes in clinical trials searching. Based on user input, the ability to narrow a search to subtype or stage of cancer was added to the basic search form. User feedback also indicated that physicians preferred trials to be listed by phase rather than by title, so the default display of search results was changed to a listing by phase, with phase IV and phase III trials appearing before phase II and phase I trials. The most substantial change, as a result of the Clinical Trials portal review, was adding the basic search form to the top of the Clinical Trials portal home page to give more ready access to the form . With coIn the summer of 1996, NCI collaborated with the National Alliance of Breast Cancer Organizations (NABCO) to develop patient-oriented abstracts (summaries) of clinical trial protocols for breast cancer trials. By October 1996, these clinical trial abstracts were available on the NABCO and CancerNet websites in a one-paragraph format. After seeking input from many advocacy organizations, the patient-oriented clinical trial abstract format was redesigned, writing guidelines were developed, and the project was expanded to include all cancer types. By September 1998, patient-oriented abstracts for all active clinical trials were available on CancerNet. Since that time, clinical trial abstracts have been written according to the original guidelines.In November 2001, selected patient-oriented and corresponding health professional clinical trial abstracts were evaluated. As a result of this evaluation, several problems were identified in the guidelines for writing the patient-oriented abstracts, including a lack of specificity in some respects and inconsistent application and interpretation of the guidelines. These findings led to the recommendation that the guidelines be redefined and expanded. Consequently, a qualitative and quantitative evaluation of the needs and preferences of users of the patient-oriented abstracts was undertaken. This evaluation included the following two elements: (1) a written survey of advocacy organizations, members of NCI's Consumer Advocates in Research and Related Activities (CARRA) Program, members of the NCI Director's Consumer Liaison Group (DCLG), comprehensive cancer center directors and administrators, cancer cooperative group chairs/administrators, and oncology nurses; and (2) in-depth interviews with CIS information specialists.A 10-question survey was mailed to nearly 400 organizations and individuals, with a 43% return rate. A key question focused on whether or not users could understand and act on the information provided in the clinical trial abstracts for patients. Results showed that 82% of users could explain the rationale or purpose of the clinical trial, 93% could determine if basic eligibility requirements were met, and 73% could understand the treatment plan.The organization and layout of the clinical trial abstracts were rated “excellent” or “good” by 72% of the respondents. Three samples of text written at 5th-, 8th-, and 12th-grade reading levels were included with the survey. The different reading levels were preferred by 37%, 42%, and 20% of the respondents, respectively.The results of the survey were better understood when viewed in the context of comments from individual respondents. Taken as a whole, the respondents' comments were varied and, at times, contradictory. Several themes, however, emerged related to language and readability, access to other resources, and pursuing participation in a clinical trial. Although 27% of the respondents indicated they could not understand the treatment plan, few specific suggestions were offered for improvement.Structured interviews were conducted with staff in six CIS offices in different geographic areas of the United States in order to obtain their perceptions of users' needs, preferences, and comprehension of the standard elements  of the patient-oriented clinical trial abstracts. The CIS information specialists interact directly with users of the abstracts by answering their questions and by guiding their use of the abstracts online during a phone call or through LiveHelp. The information specialists emphasized the need to use consumer-oriented language and the fact that users “skip” disclaimer-type information.Based on these findings and on published principles , improveThe PDQ cancer information summaries are descriptions of the latest cancer information on treatment, supportive care, screening, prevention, genetics, and complementary and alternative medicine that are reviewed and updated monthly by cancer experts. Most of the summaries are available in two versions: one written for health professionals and a corresponding patient version written in lay language.  In 2000, in response to the Chantilly meeting, work was initiated to reformat the patient-oriented information summaries. The goal was to present the information in a format and style of language that was easier to read and understand, to provide more detailed information, and to take advantage of features afforded by new Web technology.Based on design concepts that enhance readability, as well as on strategies used in information mapping, the process of reformatting and reorganizing the patient-oriented summaries was begun. “Key Points” boxes that highlighted critical concepts and linked to explanatory information in the body text were added. Links to pop-up definitions from the website's dictionary and to clinical trials information were included. For users who wished to print documents, a printer-friendly version was added that included dictionary terms and their definitions as an appended glossary.Usability testing was done to assess the ease of learning, efficiency in information gathering, and recall of information from the online documents. Based on testing results and Web design and usability guidelines , the temNCI's website is a leading resource for cancer information on the Web, consistently appearing high on the list of retrievals using search engines such as Google, Yahoo, MSN, and AltaVista. It has been awarded the Freddie Award in the website category of the 2004 International Health and Medical Media Awards, and it placed first or as an honorable mention in seven out of eight categories in the 2005 Medicine on the Net Web Excellence Awards. Its success can be at least partly attributed to NCI's efforts to make the site highly responsive to the needs of its users.The large volume of traffic that the site receives offers tremendous opportunities to study user patterns, gather feedback, and test new ideas and designs. Online surveys are an efficient way to solicit opinions from users, and analysis of website logs provides insight into user needs. NCI's relationships with members of the cancer research and advocacy communities also facilitate the gathering of advice, suggestions, and other feedback related to NCI information products. The growing body of Web-design literature and advice from usability experts are important to the development of new Web features, but input from the site's wide range of users promises to have the greatest impact on shaping online information from the National Cancer Institute.


The globalization of medical science carries for doctors worldwide a correlative duty to deepen their understanding of patients' cultural contexts and religious backgrounds, in order to satisfy each as a unique individual. To become better informed, practitioners may turn to MedLine, but it is unclear whether the information found there is an accurate representation of culture and religion. To test MedLine's representation of this field, we chose the topic of death and dying in the three major monotheistic religions.We searched MedLine using PubMed in order to retrieve and thematically analyze full-length scholarly journal papers or case reports dealing with religious traditions and end-of-life care. Our search consisted of a string of words that included the most common denominations of the three religions, the standard heading terms used by the National Reference Center for Bioethics Literature (NRCBL), and the Medical Subject Headings (MeSH) used by the National Library of Medicine. Eligible articles were limited to English-language papers with an abstract.We found that while a bibliographic search in MedLine on this topic produced instant results and some valuable literature, the aggregate reflected a selection bias. American writers were over-represented given the global prevalence of these religious traditions. Denominationally affiliated authors predominated in representing the Christian traditions. The Islamic tradition was under-represented.MedLine's capability to identify the most current, reliable and accurate information about purely scientific topics should not be assumed to be the same case when considering the interface of religion, culture and end-of-life care. With the globalization of medical science, there is a concomitant need to better understand cross-cultural differences . This etTo answer these questions we have queried MedLine to assess the scope and quality of information that those conducting a search might obtain using common search phrases that working practitioners might employ. Our objectives are twofold. First, to parse out MedLine's representation of reality from wider scholarly treatment concerned with the interface of religion, medicine and death and dying. Second, to stress that MedLine's reliability is related not only to what is indexed in it, but also to how it is accessed. Our topic is more cultural than scientific: death and dying in the three major monotheistic religions. This type of issue is particularly pertinent to healthcare providers treating a heterogeneous population, since culture and religion play a central role in shaping everything from patients' notions of health and disease causation, -10 to thCompiling the literature on religion and medicine raises questions about the bibliographic strengths of informatic catalogues in general  and MedLWe searched MedLine using PubMed in order to retrieve and thematically analyze full-length scholarly journal papers or case reports dealing with religious traditions and end-of-life care. Eligible papers were limited to English-language articles that included an abstract, to ascertain the articles' content prior retrieval. Editorials, book reviews, letters-to-the-editor and brief comments were excluded, because generally they are not peer-reviewed.To assess the relative magnitude of the broader literature, a search without language or genre limitations was also run. The search was limited to articles published between January 1993 and June 30, 2004, with the actual search completed on July 2, 2004.We defined our search as a string of words, connected by the appropriate syntax and Boolean operators . The sea• christianity OR christian• islam OR islamic OR muslims• judaism OR jewishThe second part included the sub-strings "death and dying" and "terminal care". The former is the heading term used by the National Reference Center for Bioethics Literature (NRCBL)  as the h• (islam OR islamic OR muslims) AND • (christianity OR christian) AND • (judaism OR jewish) AND Eligible articles were assessed for publication characteristics including site of publication and country of origin as well as authorship and any institutional or denominational affiliation. Professional backgrounds of authors were also recorded from the papers' institutional websites or their home pages.Articles that were the product of a logical – but nonsensical – Boolean hit, such as the retrieval of articles by authors with a surname of "Christian" or "Islam" or writers who worked at a "Jewish Hospital", were removed from consideration if they had nothing to do with death and dying. All other articles retrieved were studied in order to avoid the introduction of a selection bias.The initial general searches for religion, death and dying and denomination are noted in Table The search retrieved a total of 75 references. After eliminating duplicative results (four) and nonsensical retrievals (five), 66 articles were available for analysis. These articles were published in 36 different journals from seven countries or regions, but U.S.-edited journals accounted for the majority, with 27 publications, followed by the United Kingdom with three. Other English-speaking and European countries were represented with one or two journals at most. There was one Israeli journal and none from the Arab League or other predominantly Muslim country.The New England Journal of Medicine, The British Medical Journal, The Journal of the American Medical Association, The Lancet or Annals of Internal Medicine, were represented in our sample. Only two were from denomination affiliated journals: Health Progress, which is the official organ of The Catholic Health Association of the United States, and Christian Bioethics, an interdenominational, non-ecumenical publication [Health Progress. More than a third of articles were authored by physicians, with another 16% written by nurses, thus indicating that health professionals accounted for almost two-thirds of retrieved articles. The remaining articles were written by authors working in ethics, the humanities, social work, theology or the law. Nine authors were identified as clergy. Eight were noted as Christian clerics (six Catholics and two Orthodox), including one nun. One author was noted to be a rabbi. There were no Islamic clerics identified amongst the writers.The majority of articles were in non-denominational journals, although no major medical journal such as lication . These tMost authors (74%) were affiliated with hospitals or universities, and more than half (56%) were based in the U.S. The rest were evenly distributed among Europe, Canada and other English-speaking countries. Only one author was based in a Muslim country (Pakistan). Most authors  belonged to non-denominational institutions, while 17 were affiliated with Christian institutions, predominantly Catholic , evenly distributed among Catholic universities and hospitals. Another 12 authors belonged to Jewish institutions, eight from Jewish hospitals or hospices and another four from universities.With this data we then tried to assess what practices would be prohibited, obligatory and permitted according to the three monotheistic traditions as represented in the retrieved literature. Nevertheless, our efforts led to identifying papers that failed to provide more than broad categoricals within the traditions. This obscures the rich debate that occurs in theological and scholarly circles over diverse questions such as pain management and the potential hastening of death; whether suffering is redemptive; artificial nutrition and hydration; medical futility; and the role of quality-of-life considerations, to name but a few contentious topics that involve a broader scholarly community than those whose work is cited on MedLine.Our inquiry is limited by the search methods we employed as well as the MedLine database queried. However, our objective was, precisely, to identify such limitations in order to demonstrate the need for caution when excessively relying on database searches for topics which transcend the purely scientific.Having noted this caveat, our study was limited by circumscribing eligible articles to those with abstracts, but this was necessary to identify articles that merited additional analysis. Our findings are only applicable to articles in English, the predominant language of MedLine, and cannot be generalized to other religious traditions or different thematic areas.Finally, to avoid the introduction of our own biases about what would constitute an appropriate article for study, we included articles for analysis that would otherwise only marginally shed light on our designated areas of inquiry. This illustrates the limitations of relying purely on retrieved articles without an additional level of scholarly discrimination.While MedLine is an invaluable source of information, our study indicates the potential limits of the scholarly convenience afforded by informatics. Our data suggest that users of MedLine will gain only a partial view of the range of scholarship related to death and dying and the three major monotheistic traditions. Although a bibliographic search in MedLine on death and dying in the teachings of the three major monotheistic traditions will produce instant results and some valuable literature, the price of such convenience may be a somewhat biased and partial view of the issue.Although religion is present as a topic among the MedLine indexed literature, articles retrieved under "ethics" are three times more numerous than "religion". Despite the relatively large number of citations concerning religion, our study indicates that there was a general imbalance between the number of articles and followers of each of the traditions ,21. ThisWhatever the tradition, the vast majority of articles came from journals based in the U.S. A few articles came from developed European countries, with one from Israel and another from South Africa. It is notable that, despite the presence of 400 medical journals in the WHO-designated Eastern Mediterranean region , which iThis preponderance of U.S. journals and authors can alter the scholarly landscape and distort the application of religious teachings when articles are read outside of the North American context. Because these journals have global reach and influence, most articles they carry are firmly rooted in the American context, and cannot necessarily be extrapolated to the local environment. That may limit the literature's usefulness to those practicing in other parts of the world. And when practitioners respect uncritically these texts the way they do the scientific papers found in MedLine, this may lead to a misrepresentation of local cultures and traditions.For example, in predominantly Catholic Latin America, the question of physician-assisted suicide or euthanasia – frequently addressed in these articles – may be little more than an intellectual curiosity, the practice of which is nearly inconceivable in a Latin context. No North American Catholic author wrote about death and dying in the context of poverty, which gives rise to the most difficult ethical quandaries from the Rio Grande to Antarctica . AlthougThe fact that most articles were written by doctors and nurses may also contribute to a rather narrow focus on clinical practices at the end of life, ignoring broader public health and societal context factors that may influence the care provided to dying patients and their families. Papers featuring practical guidelines for the care of the dying Catholic ,27 or Jeobjective reality of science, our observations indicate that MedLine is an incomplete source of information for the complex interplay of death, dying and religion.Although MedLine is an excellent source of the more These errors of omission reflect the orientation of the biomedical paradigm MedLine was meant to serve. Unfortunately, it has been unable to transcend its scientific origins.The author(s) declare that they have no competing interests.Both authors contributed equally to this work.The pre-publication history for this paper can be accessed here:


More than 72% of health expenditure in India is financed by individual households at the time of illness through out-of-pocket payments. This is a highly regressive way of financing health care and sometimes leads to impoverishment. Health insurance is recommended as a measure to protect households from such catastrophic health expenditure (CHE). We studied two Indian community health insurance (CHI) schemes, ACCORD and SEWA, to determine whether insured households are protected from CHE.ACCORD provides health insurance cover for the indigenous population, living in Gudalur, Tamil Nadu. SEWA provides insurance cover for self employed women in the state of Gujarat. Both cover hospitalisation expenses, but only upto a maximum limit of US$23 and US$45, respectively.annual hospital expenditure greater than 10% of annual income," to identify those who experienced CHE.We reviewed the insurance claims registers in both schemes and identified patients who were hospitalised during the period 01/04/2003 to 31/03/2004. Details of their diagnoses, places and costs of treatment and self-reported annual incomes were obtained. There is no single definition of CHE and none of these have been validated. For this research, we used the following definition; "There were a total of 683 and 3152 hospital admissions at ACCORD and SEWA, respectively. In the absence of the CHI scheme, all of the patients at ACCORD and SEWA would have had to pay OOP for their hospitalisation. With the CHI scheme, 67% and 34% of patients did not have to make any out-of-pocket (OOP) payment for their hospital expenses at ACCORD and SEWA, respectively. Both CHI schemes halved the number of households that would have experienced CHE by covering hospital costs. However, despite this, 4% and 23% of households with admissions still experienced CHE at ACCORD and SEWA, respectively. This was related to the following conditions: low annual income, benefit packages with low maximum limits, exclusion of some conditions from the benefit package, and use of the private sector for admissions.CHI appears to be effective at halving the incidence of CHE among hospitalised patients. This protection could be further enhanced by improving the design of the CHI schemes, especially by increasing the upper limits of benefit packages, minimising exclusions and controlling costs. Out-of-pocket payments by individual households are the main source of health care financing in India. National health accounts show that 72% of all health expenditure is made by individual households  which iscatastrophic health expenditures (CHE)".Contrary to most other consumption expenses, medical expenditure is largely unpredictable both in timing and quantity. Households, especially in low income countries, cope either by divesting their savings, borrowing, mortgaging or selling assets, or by forgoing treatment -10. ManyHealth insurance is put forward as a measure to protect against CHE ,9. While1. CHI schemes protect households from catastrophic health expenditure2. The household income, the depth of the benefit package and the cost of health care determine the incidence and intensity of CHE.Finally, based on our findings, we explore how these schemes' protection against CHE can be enhanced.This is a comparative study, measuring the effect of two CHI schemes on CHE. For this study, we purposively selected two schemes that had different design features – ACCORD and SEWA (as documented below).ACCORD, a non-governmental organisation (NGO) in Tamil Nadu, south India, works for the overall development of the indigenous people of the Gudalur sub-district. This population also called 'adivasis' have traditionally been hunters and food gatherers. However, with progressive deforestation over the past few decades, most of them have shifted to wage labour. As per the 2001 census, there were 215,269 inhabitants in Gudalur, of which 14,149 were adivasis . ACCORD ACCORD's health programme (ASHWINI) is a three tier health system, with village health workers, health centres and a 20 bed hospital. Other than the ASHWINI hospital, there are four other NGO hospitals with a total of 75 beds, three government hospital (160 beds) and one private hospital (10 beds) in Gudalur sub district. There are only four specialists in Gudalur, two in the ASHWINI hospital and two in the government hospitals.Part of the ACCORD health service is financed by a CHI scheme initiated in 1992. All AMSThe Self Employed Women's Association (SEWA) is a union of women employed in the informal sector. While its main area of operation is Gujarat, it also has presence in other states of India. Among its diverse activities, it provides an integrated insurance package – including life, asset and medical insurance – on a voluntary basis, for its members and their husbands . In 2003In Gujarat, like in most Indian states, there is a network of public and private providers, operating independent of each other. While the government hospitals are based on population norms: a community health centre with 30 beds for 100,000 population, a sub-district hospital with 100 beds for 500,000 population and a district hospital with 200 beds for a million population, the private sector is more diverse and disparate. Most of the private sector hospitals are located at the district level have just 6 – 30 beds and limited facilities .Both CHI schemes, in principle target the poor and vulnerable sections of Indian society. While both are organised by NGOs and insure against hospital expenditure, there were some differences. First, at the ACCORD CHI, all members of the AMS and their family were eligible to enrol. At SEWA, only the woman member and her spouse were eligible to enrol in 2003. Secondly, the ACCORD CHI scheme was a cashless one, so the patient does not have to pay anything as long as the hospital bill was less than US$23. At SEWA, the patient had to pay the bills to the provider and was reimbursed later up to a maximum of US$45. Furthermore, the ACCORD CHI scheme recognised only one provider, the ASHWINI hospital, which is a not-for-profit institution. Patients enrolled in the SEWA scheme sought care at both private and government hospitals. Lastly, while the ACCORD CHI scheme had minimal exclusions, SEWA usually excluded pre-existing illnesses from reimbursement, especially during the member's first year as a policy-holder. Both schemes were supported by external donors to a certain extent.At both ACCORD and SEWA, we reviewed the insurance claims registers and identified all the scheme members who were hospitalized between the period April 2003 to March 2004, and who registered this hospitalization with the scheme. For these patients, we collected details on the age and sex of the patient, diagnosis, total bill amount and amount paid out-of-pocket by the patient. At SEWA, we also documented the amount reimbursed subsequently and the annual household income as reported by the patient from the insurance claim register. At ACCORD, we used the median annual household expenditure obtained from an independent survey  to estimate the annual income of the households with patients.if a household expenditure for hospitalisation exceeded 10% of the total annual household income". We used this definition because, according to literature, 10% of total expenditure is considered as an approximate threshold at which the poor household is forced to sacrifice other basic needs, sell productive assets or incur debt [There is still a lot of debate on the definition of CHE. Most authors agree that health expenditure is catastrophic if it forces households to significantly lower their standard of living now or in the future. While some have defined CHE if the total health expenditure is more than 10% of annual income ,12,15, ocur debt . FurtherUsing this definition, we calculated the incidence of out-of-pocket (OOP) payments and CHE. To calculate the OOP before insurance, we used the gross hospitalisation expenditure at both ACCORD and SEWA. We then expressed this expenditure as a proportion of the reported annual income. If this was more than 10%, then that household was considered to have experienced CHE. OOP payment after insurance was calculated by subtracting the total hospital expenditure from the amount reimbursed. In the case of ACCORD it was the amount that the patient had to pay at discharge . CHE was then calculated by computing this OOP payment (post insurance) as a proportion of the reported annual income. If this was more than 10%, then that household was considered to have experienced CHE. The difference gives us an idea of the effect of the CHI scheme on CHE among these poor households. We compared the medians and used the 95% confidence intervals to analyse any differences between the two schemes. To compare the incidence of CHE by specific characteristics, we used the risk ratio and the 95% confidence interval. As we did not have the individual household income at ACCORD, we limited the analysis of determinants to the SEWA scheme.Ethical clearance for this study was provided by the ethical board of the Sri Chitra Tirunal Institute of Medical Sciences and Technology, Trivandrum, Kerala, India.In April 2003, a total of 4,268 individuals out of 12,226 AMS members (35%) enrolled in the ACCORD CHI scheme. In the same period, at SEWA, 101,809 women and men out of about 500,000 SEWA members (~20%) enrolled in the CHI scheme  in the absence of an insurance scheme. However, because of the CHI scheme, this proportion was reduced to 3.5%. At SEWA, 49% of the households would have been catastrophically affected by the admission costs if they were not insured. The CHI scheme at SEWA has been successful in reducing the incidence of CHE in insured households to 23%. Not only has the incidence of CHE been halved in both the schemes, the magnitude of CHE has also been significantly reduced in both schemes. This is graphically represented in Figure There are many causes for the phenomenon of CHE. One important factor is high medical bills. The medical bills in these two schemes were very different, nearly four times higher at SEWA. This difference was further exaggerated when one disaggregates the data by providers. At SEWA, the median bill with a private provider was US$48, while it was only US$28 with a government provider. One possibility for this difference in price between the two schemes could be different case mixes. However, even though these schemes are more than 1000 km apart and the populations are different, we note that the case mix is relatively uniform in both the schemes; the cases are a mixture of communicable and non-communicable conditions (Table Our results also indicate that at SEWA low annual household income predisposes a household towards CHE Table . The SEWProtecting households against catastrophic health expenditure is a health policy goal. There is documented evidence to show that health expenditure can impoverish households; make thHowever, this protection is only partial and some households still experience CHE, especially the poorest ones. The incidence of CHE was nearly six times higher among the poorest quartile than among the richest quartile. This is understandable as the poor have low incomes and so any health expenditure can easily exceed 10% of their incomes. This is more so in a scheme like SEWA, where bills are reimbursed after a month or more. Patients have to mobilise resources to pay the bills. Sometimes this is in the form of short term loans from moneylenders who charge high interest rates. However, it is positive to note that the poorer households also benefited the most from the health insurance mechanism. The risk of CHE reduced by more than 50% after insurance in the poorest two quartiles, while in the richest two quartiles, it was less than 50%. This indicates that such schemes are still pro-poor in their financial protection function.One of the limitations of our study is that it is facility based. For estimating CHE, one should consider all health expenditure during a full year. However, here we have assessed the household expenses only for hospitalisation; therefore, the real incidence and intensity of CHE is underestimated in our study. However, the high bills associated with a hospitalisation generally has a much larger impact than expenses for ambulatory or preventive care that are lower in magnitude and spread over time. So, calculating CHE based on hospitalisation expenses gives us a fair idea of the economic shock experienced by a household. We do however recognise that that for some households, even expenditure on medicines can be catastrophic.At SEWA, patients who had crossed the upper limit in the first admission may not have submitted a second claim, knowing that they would not be reimbursed. This further compounds our underestimate. As we did not have information on the individual household incomes of the patients at ACCORD, we had to use survey data as a proxy for calculating the CHE. So our calculation of CHE in ACCORD is actually the proportion of the median household income. This naturally would result in an underestimation of the incidence of CHE in the poor, while there would be an overestimation among the better off. This disparity would depend a lot on the variation of the income within the community. But, as most adivasi households at ACCORD are relatively homogenous in their poverty, we feel that this factor should not affect the results significantly.As mentioned earlier, there is considerable debate on the definition of CHE. While some authors put the threshold at 10% of the annual income, others use the indicator based on disposable income. Which is more valid? Is it relevant to have an uniform threshold, irrespective of the economic status? It may be presumed that in a poor household, even a small proportion spent on health care may be catastrophic. On the other hand a better off household may be able to absorb the shock of a higher proportion of its income spent on health care. Hence to put uniform thresholds for all households is somewhat arbitrary and misleading. Similarly can this indicator be equally applied to subsistence farmers? These households have very little cash transaction and hence even a small expenditure may be catastrophic, especially if it means that they have to sell their future food supply. These and other questions beg answers, which need detailed validation studies. It was not our purpose to validate any of these indicators, but to show that even in the presence of insurance, households do still continue to experience CHE.The fact that health expenditure continues to lead to CHE despite insurance coverage is a concern and is a consequence of various factors. One reason is the low maximum limit in both schemes. One way of increasing the protective effect of CHI schemes would be to expand the maximum limit of the benefit package to cover common surgical and medical conditions. However, any increase in the benefit package would be associated with a rise in the premium. This may adversely affect enrolment. One possible solution is for government or donors to subsidise the premiums, especially for the poorer households. This would allow the poor to enrol in the scheme as well as protect them from CHE.It was surprising to see that though SEWA has a higher upper limit, more patients pay OOP and more households experience CHE. One important reason could be the higher bills at SEWA. Another measure to reduce the risk of CHE would thus be to reduce the costs of health care. From our data we note that the prices of hospital services for similar conditions were systematically higher at SEWA compared to ACCORD. This could be because of the predominant use of the private sector in SEWA. In India, CHI schemes may be forced to use the private health services if they want to provide their community with choices; but in this case, they would need to introduce certain cost containment measures. One measure could be to pay providers on a case basis, rather than on a fee-for-service basis . This inThis tendency for health insurance to increase the total health care expenditure of households is described by Bogg et al in their study in China . They shExclusions in a health insurance scheme are detrimental for various reasons. From the patient's perspective, it adds to the uncertainty at the time of care. From a public health perspective, it does not provide protection for those patients who are the most vulnerable i.e. patients with chronic illness and who require regular medications . FinallyCHE is a major cause of impoverishment and patients need to be protected from it . Some ofIndian CHI schemes are able to protect their members against CHE, but only to a limited level. However, this protection can be further enhanced if some design changes are incorporated. To begin with, the upper limit of the benefit package needs to be raised. To keep the premiums affordable, donors or the government would need to directly subsidise the premium, especially for the poorer sections of society. Exclusions need to be minimised to protect vulnerable populations. And finally, scheme managers need to negotiate costs with providers from the start  to ensurNB: Subsequent to our study, SEWA has made significant changes in its design, including expansion of eligibility to include all members of the family and piloting of a third party payment mechanism. ACCORD has increased its upper limit from US$ 23 to US$ 69.ACCORD Action for Community Organisation, Rehabilitation and DevelopmentAMS Adivasi Munnetra SangamASHWINI Association for Health Welfare in the NilgirisSEWA Self Employed Women's AssociationNGO Non-governmental organisationCHE Catastrophic health expenditureCHI Community health insuranceOOP Out-of-pocketSCTIMST Sree Chitra Tirunal Institute for Medical Sciences and TechnologyThis is certify that Dr. N. Devadasan, was the founder of the ACCORD CHI. He is however, not directly involved with the project anymore and is currently a visiting faculty at SCTIMST and pursuing a PhD with the ITM, Antwerp.Dr MK Ranson was involved with SEWA while pursuing his PhD and post doctoral research.Dr. Wim van Damme, Dr Bart Criel and Dr Patrick Van der Stuyft have not been associated either with ACCORD or with SEWA.The authors declare no financial competing interest.ND conceptualised the study and participated in it by collecting the data at ACCORD, analysing and interpreting the data and drafting the manuscript. WVD conceptualised the study, assisted in interpretation of the data and revised the manuscript critically. KR participated in the study by collecting the data at SEWA and revised the manuscript critically. BC assisted in interpretation of the data and revised the manuscript critically for substantial intellectual content. PVDS conceptualised the study and assisted in interpretation of the data and revised the manuscript critically for substantial intellectual content.All the authors confirm that they had access to all the data, have read and approved the final manuscript.The pre-publication history for this paper can be accessed here:


The main aims of this paper are to describe the setting and design of a Minimal Intervention in general practice for Stress-related mental disorders in patients on Sick leave (MISS), as well as to ascertain the study complies with the requirements for a cluster randomised controlled trial (RCT). The potential adverse consequences of sick leave due to Stress-related Mental Disorders (SMDs) are extensive, but often not recognised. Since most people having SMDs with sick leave consult their general practitioner (GP) at an early stage, a tailored intervention given by GPs is justified. We provide a detailed description of the MISS; that is more accurate assessment, education, advice and monitoring to treat SMDs in patients on sick leave. Our hypothesis is that the MISS will be more effective compared to the usual care, in reducing days of sick leave of these patients.The design is a pragmatic RCT. Randomisation is at the level of GPs. They received the MISS-training versus no training, in order to compare the MISS vs. usual care at patient level. Enrolment of patients took place after screening in the source population, that comprised 20–60 year old primary care attendees. Inclusion criteria were: moderately elevated distress levels, having a paid job and sick leave for no longer than three months. There is a one year follow up. The primary outcome measure is lasting full return to work. Reduction of SMD- symptoms is one of the secondary outcome measures. Forty-six GPs and 433 patients agreed to participate.In our study design, attention is given to the practical application of the requirements for a pragmatic trial. The results of this cluster RCT will add to the evidence about treatment options in general practice for SMDs in patients on sick leave, and might contribute to a new and appropriate guideline. These results will be available at the end of 2006. Stress results from an imbalance between demands and resources . It is aThe societal and financial costs of dysfunction in terms of (long term) sick leave due to SMDs are extensive. In the Netherlands, almost one million workers are entitled to disability benefits (9 percent of the working population), this prevalence is high compared with other countries . About oNearly every employee contacts the GP at the beginning of the sick leave. Most patients having SMDs are managed in primary care, and not referred to specialised secondary care. And despite the fact that mental health problems are common in primary care, GP's may still find it difficult to diagnose and treat them, unless they have a high index of suspicion . Due to Terluin and Van der Klink  have outThe MISS is a prototypical intervention for SMDs with sick leave in general practices, aiming specifically at successful rehabilitation and preventing long-lasting sick leave. By using specific communication and the minimal amount of time necessary, the GP helps the patient on the accurate and time-contingent course. The MISS should take no more than 3 consultations of 10–20 minutes, and consists basically of 5 elements: assessment, education, advice, monitoring and, if necessary, referral.Assessment implies in the first place to identify patients having a SMD and help them to acknowledge their distress. Second, the GP detects significant depression and anxiety, and propose management steps for these problems separately. The Four-Dimensional Symptom Questionnaire  is usede  one or two times a day for 30–45 minutes. The patient is also advised to schedule necessary activities of daily living, such as children's care and housekeeping. Furthermore the patient is recommended to visit his or her occupational physician without delay, and to explore ways after which it is possible to partially return to work, even when not all symptoms have disappeared.Monitoring is about ensuring the patient is moving towards the accurate course. This implies a focus on the problems (and possible solutions) in stead of symptoms. The patient has to investigate the specific problems and consider different ways to cope. The switch in the patient's focus has often already occurred before the second consultation; however it should in any case have occurred after four weeks of sick leave.Referral to specialised care comes into play when there is no evidence of progression after four weeks of sick leave. The GP should realize that the patient is at risk for prolonged sick leave and ultimately loss of employment. A more specialised treatment is necessary since the patient is not likely to benefit from more time off. The GP may refer the patient to a counsellor, social worker or a  psychotherapist, ideally after consulting with the occupational physician.The central aim of this pragmatic cluster randomised controlled trial (RCT) is to investigate the effectiveness of the MISS in general practice. Our hypothesis is that GPs who carry out the MISS will be more effective than GPs who perform usual care in reducing the number of days of sick leave, as well as in reducing symptoms of SMDs in patients. Usual care in general practice contains the guidelines on depression  and anxiThe recruitment of GPs was split up in four different rounds. We approached GPs in two different districts where the Department of General Practice of the VU University Medical Centre has some type of network positioned. A total of 46 GPs signed informed consent.In order to recruit enough eligible patients, we made use of the computerised patient record system and approached the source population of patients (n = 22.740) by mail. The source population consisted of all primary care attendees (20–60 years) who visited consulting hours of the participating GPs. GP's excluded only patients with very severe psychiatric disorders (mania or psychosis), patients with terminal illness or an inadequate command of the Dutch language. The source population of attendees was asked only to respond when they met our criteria: moderately elevated distress level , havingEvery one or two weeks we approached the source population, until enrolment of a sufficient number of patients from a particular GP was realised. Final recruitment took place by phone survey of the patients who returned the questionnaire and met the criteria. A total of 433 patients 1,9%) who could and also wanted to participate were enrolled , and an occupational physician. This training existed of two times 3,5 hours and 2 times follow-up of 2 hours . Skills needed for successful treatment were accentuated. The participating GPs own experiences were evaluated; they were expected to provide case histories and to practice with the different parts of the intervention during the training. Screening of eligible patients started after the MISS group got the second training. When a patient had signed the informed consent , the GP was informed of the participation. Actual application of the elements of the MISS and steps in usual care are measured with a questionnaire.During the baseline interview all patients were asked whether they had planned another visit to their GP. If not they were advised to consider this, in order to give the GPs the opportunity to start with an intervention. Even though, it should be noted that neither the GPs were obliged to apply the MISS or any other intervention for mental disorders, nor were the patients obliged to go visit their GPs. No intervention was done on the actual completion or successfulness of the application of the MISS, the present method comprises real clinical practice in primary care.The primary outcome measure is defined as: duration of sick leave in calendar days from the first day of sick leave to full return to work, lasting at least 4 weeks without  relapse. Variables of the rehabilitation process itself, like the time to first  return to work, total days of sick leave in the whole year, and  return to work rates after 2, 6 and 12 months, are secondary outcome measures. Sick leave in the past year is considered to be a prognostic factor for our primary outcome measure, as are job content data ,22.An important secondary outcome measure is reduction in symptoms of depression, anxiety, somatization and distress, measured with the 4DSQ . Life-evCost effectiveness will be evaluated from societal perspective  by using the Tic-P . The empAt baseline, patients who entered the study were measured by a phone survey and received a questionnaire by mail. Follow-up measurements, again a phone survey followed by a questionnaire, were scheduled at 2, 6 and 12 months after baseline. All outcomes were measured at the level of the patient, except for the GP interventions and skills. These were reported by the GPs two months after baseline. Moreover, data on received health care were extracted from the medical records after the completion of the one year follow-up.Proportions used to determine the sample size needed, were adopted from a related study completed in the occupational health care setting . In thatRandomisation took place at the level of GPs, after each of the four recruitment moments and after the GPs signed informed consent. Because balance between groups in size and characteristics was presumed all four times GPs were randomised, no blocking or stratification was used. We developed the following procedure to conceal allocation. The names of the GP's (and dummy in an uneven group) were put on a list of which the order was subsequently randomised by one person (IMB). Another person (BT) independently drew up a randomised list of codes (1 = MISS group and 2 = control group) with an equal number of '1' and '2' codes up to the number of GPs being randomised. Finally, these two lists were brought together and the first GP on the list was allocated to the group indicated by the first code; and so on. As a result, 24 GPs were allocated to the intervention (MISS) group and 22 GPs were allocated to the control  group.After assigning the GPs and training was given to the MISS group, patients were enrolled by screening the source population (see participants). The general practice team gave entrance to data on the source population, the source population was given the inclusion criteria through a screening questionnaire. The research assistance team was responsible for the final recruitment. They gave information in a phone survey and asked the patient informed consent to participate.Patients were kept unaware that two different interventions were studied; both groups were given exactly the same information and questionnaires. The patients, as well as the external interviewers who carried out the phone surveys, were told that the study was about stress and sick leave. Finally, the internal research assistance team responsible for the process of data collection knew that the study involved a training of half of the GPs; nevertheless the internal research team had no information on which GPs were allocated to what conditions.First of all, baseline similarity between the MISS and usual care groups will be examined, and baseline characteristics of drop outs and completers will be compared. Cox regression analyses will be used to investigate the intervention effect, by analysing differences in outcome with survival analysis of the primary outcome measure between the MISS and usual care group. To correct for misclassification of patients and severity of complaints (inclusion is only by level of distress with sick leave), and as a consequence to avoid bias in the effect, baseline measures of psychological symptoms , costs A cost-effectiveness analysis will be performed to assess the incremental costs per unit improvement on the 4DSQ . BootstrAs regards the prognostic measures, univariate analyses will be used to select relevant factors, with a focus on identifying prognostic factors for our primary outcome measure. Subsequently, Cox regression analyses and logistic regression analysis will be performed on these relevant factors.This project is developed for the primary care setting with its' typical case load of stress-related mental disorders, and not for specialised care in which patients have more clearly defined mental disorders. We provide an intervention that is aimed at better recognition, good communication, and a time-contingent framed recovery process. Our approach of SMDs, with the need to identify specific psychiatric disorders where they exist and also to respect the roles of daily life in determining and shaping psychological disability, is an example of specified stepped care in general practice. Potential risk factors for chronicity are pointed out in our training, and early recognition and treatment is the main goal of the MISS. The role of more specialised care is well acknowledged in the intervention.Particular strength of our study protocol is the method of recruiting patients. By using the computerised patient record system, we screened the whole population of general practice attendees and thereby determined who entered the study, instead of chartering the GPs to select their patients. Therefore, similarity in both groups is assured and we avoided selection bias. Also, we did not have difficulties in including enough eligible patients, which is often a problem in primary care trials. Furthermore, the current method allows us to consider the pragmatic effectiveness and to avoid interference with daily practice of consulting hours. In this trial, follow-up is on patients having SMDs with sick leave, instead of patients who actually get an intervention for their complaints. Good external validity  is accomplished by this rather heterogeneous, and therefore highly representative, group of patients with SMDs on sick leave. To assure as much internal validity as possible in this pragmatic trial, we randomised on the level of GPs to avoid contamination. In addition, research assistants who collected the data and also the patients were blinded.Notably, effectiveness instead of efficacy is studied. We are evaluating what is possible in real clinical practice, rather than under ideal circumstances. As a consequence, mental health state will vary between the participants. Through subgroup analysis on severity of complaints and levels of distress (measured with the PRIME-MD and 4DSQ), we can classify possible high or low risk groups for prolonged disability within this heterogeneous group. Identification of a high risk group for non-recovery may lead to better suited guidelines on stepped care and treatment. We cannot assure that everyone in the MISS group has received the intervention; the GPs were given total freedom in actually delivering the MISS. Nevertheless, the number of visits, diagnoses, recommendations, treatments and proceedings will give us information about the compliance of the GPs and their influence on the effect of the MISS. In this way, daily practice is measured in stead of ideal circumstances. To avoid social desirable answers from the GPs on their advises and treatments, we will also check the medical records of the patients.Finally, many requirements for a high quality trial are being met. Results of this cluster RCT will contribute to treatment options for patients having SMDs with sick leave in general practice, and might contribute to new and better suited guidelines and stepped care. Results will be available in the end of 2006.The author(s) declare that they have no competing interests.BT developed the intervention. BT and HvM participated in the design and co-ordination of the trial, and in writing the article. JS advised on the content of the design of the trial. CG advised on the content of statistics. WS and WvM advised on the content of the study design and article. IMB conducted the research and wrote the article. All authors provided comments on the drafts and have read and approved the final version.The pre-publication history for this paper can be accessed here:


The fields of expertise of natural disasters and complex emergencies (CEs) are quite distinct, with different tools for mitigation and response as well as different types of competent organizations and qualified professionals who respond. However, natural disasters and CEs can occur concurrently in the same geographic location, and epidemics can occur during or following either event. The occurrence and overlap of these three types of events have not been well studied.All natural disasters, CEs and epidemics occurring within the past decade (1995–2004) that met the inclusion criteria were included. The largest 30 events in each category were based on the total number of deaths recorded. The main databases used were the Emergency Events Database for natural disasters, the Uppsala Conflict Database Program for CEs and the World Health Organization outbreaks archive for epidemics.During the past decade, 63% of the largest CEs had ≥1 epidemic compared with 23% of the largest natural disasters. Twenty-seven percent of the largest natural disasters occurred in areas with ≥1 ongoing CE while 87% of the largest CEs had ≥1 natural disaster.Epidemics commonly occur during CEs. The data presented in this article do not support the often-repeated assertion that epidemics, especially large-scale epidemics, commonly occur following large-scale natural disasters. This observation has important policy and programmatic implications when preparing and responding to epidemics. There is an important and previously unrecognized overlap between natural disasters and CEs. Training and tools are needed to help bridge the gap between the different type of organizations and professionals who respond to natural disasters and CEs to ensure an integrated and coordinated response. The causes of disasters are not always clear and often overlap. For example, Sen argues that famines are usually caused by a lack of purchasing power or entitlements and not necessarily due to drought and consequent food shortage, which can be exacerbating factors . An epidThere are few articles and data that examine the frequency of occurrence and overlap among natural disasters, complex emergencies and epidemics. These data have important implications for disaster planning and response. Do large-scale epidemics commonly occur following large natural disasters, as was recently loudly claimed by the World Health Organization (WHO) and widely repeated in the media worldwide following the recent Asian tsunami ,3? If soThe fields of expertise of natural disasters and CEs are quite distinct with different tools for mitigation and response as well as different types of competent organizations and qualified professionals who respond. However, natural disasters and CEs can occur concurrently in the same geographic location and epidemics can occur during or after either event. For example, in the recent Asian tsunami, affected areas in Sri Lanka and Aceh province, Indonesia, have rebel insurgencies, and Somalia has been in civil war for decades . In the The objectives of this article are twofold: (1) to identify large-scale natural disasters, CEs and epidemics over the past decade (1995–2004); and (2) to document, for each of the large-scale events in the above three categories, the occurrence in the same location and relevant timeframe of the other two types of events, regardless of their magnitude.The data sources consisted of using the Center for Research on the Epidemiology of Disasters' (CRED) Emergency Events Database (EM-DAT), a database containing essential core data on the occurrence and effects of over 12,800 mass disasters in the world from 1900 to present , for natlarge-scale events. Thirty events were considered to be sufficient to meet the objectives of the article as well as to allow the authors to clarify and resolve conflicts in the data and to match timeframes and geographic location. However, other concurrent events that met the inclusion criteria with each major event category were included regardless of the magnitude of mortality. These other events were recorded as occurring within each major event if they occurred in a specific timeframe and in the same geographical location but not necessarily among the same populations affected by the events; the data did not allow for such a distinction. The same geographical location refers to the same state or similar type of entity (e.g. province) in a country but not necessarily overlapping among the same population (e.g. occurring in 2 different districts in the state). For example, any epidemics or natural disasters that occurred within the timeframe and location of a large-scale CE, regardless of the number of deaths  were included. Natural disasters were linked to a large-scale epidemic if they occurred within six months before the onset of the epidemic and within the same geographic location. Conversely, epidemics were linked to a large-scale natural disaster if they occurred within the following six months after the natural disaster and within the same geographic location. Events that affected many countries, such as the 2004 Asian tsunami, meningococcal epidemics in the African meningitis belt, and the 2003 heat wave in Europe were counted as one event. Terrorism events, such as the 2001 World Trade Center attack, bioterrorism, and human-made disasters, such as transportation and industrial accidents  were not included in the three event categories. Chronic diseases, such as HIV/AIDS and tuberculosis, were excluded from the category of epidemics.Only events occurring within the past decade 1995–2004) that met the definitions of a natural disaster, CE, or epidemic for this article (table 995–2004 a was 13% (figure Throughout the decade, our research found 3,197 recorded natural disasters, 363 recorded complex emergencies, and 1,374 recorded epidemics. The median duration of the largest 30 natural disasters table  during t% figure .The median duration of the largest 30 CEs during the past decade table  was 4,56b was 10% (Figure The median duration of the largest 30 epidemics during the past decade table  was 107 % Figure .The need to prepare for the possible occurrence of epidemics following natural disasters ,15-18 anAlthough epidemics do not commonly follow large-scale natural disasters, when large-scale epidemics do occur, they often occur during CEs of any magnitude, and to a lesser extent following natural disasters. One-third of the 30 largest epidemics during the last decade occurred on their own; 47% occurred during at least one CE, 30% following at least one natural disaster, and 10% with both events. Thus, governments, United Nations agencies and non-governmental organizations must continue to prepare for the possibility of epidemics following natural disasters and particularly during CEs.The occurrence of natural disasters and CEs in the same geographic location has not been well studied . Some arEarly warning systems for natural disasters and epidemics, although technologically challenging and costly, have been shown to be effective . DespiteThere are a number of limitations in this article. The data show only an ecological association between events and not a cause and effect relationship. The temporal and spatial occurrence of events may not necessarily be related to one another as the data did not allow us to definitively ascertain if they occurred among the same population. If there was a relationship among these events, its effect was not examined. Some events may not have been captured by the databases used. However, whenever possible, we attempted to triangulate the data from different sources. Some disasters were not easily classified into one category and thus misclassification may have occurred; for example, the famine in North Korea could be categorized as a natural disaster or CE; we chose the former. The largest natural disasters, CEs and epidemics during the past decade were arbitrarily limited to the biggest 30 according to mortality; this limited our sample size and does not include other important ways to categorize disasters, such as morbidity and persons affected. Mortality was chosen because it is an essential outcome and the most commonly reported data in the databases used. However, for some types of epidemics and natural disasters, mortality may not be a major outcome and thus morbidity may have been a better outcome to measure the magnitude of these events. Chronic diseases, such as HIV/AIDS and tuberculosis, although causes of major mortality throughout the world, have not been classified as epidemics according to the definition and databases used in this paper.One strong conclusion of the article is that the longer an event, the higher the risk to have a concomitant event. Since CEs occur over a much longer time period than natural disasters and epidemics, the conclusion that epidemics occur much more commonly during large-scale CEs than following natural disasters is intuitive; however, it is important to have data to support this assertion, which our paper clearly provides. Furthermore, this conclusion has important policy and programmatic implications. Appropriate stockpiling of vaccines, medications and other essential supplies need to be kept up to date and accessible over a long period of time. Since accessibility may be difficult in these emergency situations, proper preparedness planning must occur, including having multiple stockpiles within the same country in order increase the possibility of distributing the supplies. This type of stockpiling must be weighed against the increased cost of having multiple stockpiles in a country. A functioning epidemic alert and response system needs to be established and maintained. Furthermore, the high turnover of staff working in CEs means that continuous training needs to be provided over many years.Ethiopia has been subject to recurrent drought and food shortages which have sometimes been exacerbated by civil strife ,29. ThesIn early 2000, cases of severe malnutrition and measles began to be reported by non-governmental organizations but it was not until April 2000 when media attention began to focus on Gode in Somali region that a large-scale international humanitarian response was triggered. The humanitarian response was not only delayed but was also overly focused on food-based interventions such as the general food ration and therapeutic and supplementary feeding for severely and moderately malnourished individuals-the so called 'food first bias' . While sLarge scale natural disasters and CEs have occurred primarily in Africa and Asia from 1995–2004. Epidemics with mortality have occurred much more frequently during large-scale CEs than following large-scale natural disasters during the past decade. The data presented in this paper do not support the common assertion that epidemics, especially large-scale epidemics, commonly occur following large-scale natural disasters. There is a significant and previously unrecognized overlap between natural disasters and CEs. Training and tools are needed to help bridge the gap between the different type of organizations and professionals who respond to natural disasters and CEs to ensure an integrated and coordinated response. Further study of the relationships among natural disasters, CEs and epidemics is needed to define the extent to which the occurrence of one type of disaster enhances the risk of another.PBS conceived of the study, designed the research plan, supervised the literature review and data analysis, and wrote the paper. PL participated in the study design, co-wrote the paper, undertook the literature review and data analysis. MTV participated in the study design, co-wrote the paper, undertook the literature review. PS assisted in the critical interpretation of the intellectual content and drafting of the paper.The authors and their institutions have no financial or other conflicts of interests. There were no grants or outside funding for this work.