Out-of-pocket expenditures of over $34 billion per year in the US are an apparent testament to a widely held belief that complementary and alternative medicine (CAM) therapies have benefits that outweigh their costs. However, regardless of public opinion, there is often little more than anecdotal evidence on the health and economic implications of CAM therapies. The objectives of this study are to present an overview of economic evaluation and to expand upon a previous review to examine the current scope and quality of CAM economic evaluations.The data sources used were Medline, AMED, Alt-HealthWatch, and the Complementary and Alternative Medicine Citation Index; January 1999 to October 2004. Papers that reported original data on specific CAM therapies from any form of standard economic analysis were included. Full economic evaluations were subjected to two types of quality review. The first was a 35-item checklist for reporting quality, and the second was a set of four criteria for study quality .A total of 56 economic evaluations  of CAM were found covering a range of therapies applied to a variety of conditions. The reporting quality of the full evaluations was poor for certain items, but was comparable to the quality found by systematic reviews of economic evaluations in conventional medicine. Regarding study quality, 14 (36%) studies were found to meet all four criteria. These exemplary studies indicate CAM therapies that may be considered cost-effective compared to usual care for various conditions: acupuncture for migraine, manual therapy for neck pain, spa therapy for Parkinson's, self-administered stress management for cancer patients undergoing chemotherapy, pre- and post-operative oral nutritional supplementation for lower gastrointestinal tract surgery, biofeedback for patients with "functional" disorders , and guided imagery, relaxation therapy, and potassium-rich diet for cardiac patients.Whereas the number and quality of economic evaluations of CAM have increased in recent years and more CAM therapies have been shown to be of good value, the majority of CAM therapies still remain to be evaluated. Complementary and alternative medicine (CAM) has a reputation for good value among health conscious consumers . In the vis medicatrix naturae . As such, a thorough and external review of economic and health outcomes of CAM is necessary for evidence-based consideration of CAM therapies as a covered expense. That being said, it is also known that affirmative evidence on economic and health outcomes is a necessary, but not sufficient step toward CAM coverage, and not the decision itself. Other factors such as historical demand, political expediency, consumer demand, and practitioner enthusiasm may also be considered in the decision to incorporate CAM into a health insurance policy [The paucity of outcomes research in CAM has likely depressed access to CAM therapies by impeding their integration into financial mechanisms commonly found in conventional health care. Most US consumers who have health insurance coverage, either through public or private institutions, bear the entire cost of CAM therapies out-of-pocket . Theorete policy ,4,5.The need for economic evaluations is also growing in conventional healthcare. An increasing number of health plans and hospitals have moved from a simple budgetary focus in formulary decisions to requiring detailed evidence on the economic value of considered therapies relative to alternatives ,7. BeyonIn their systematic review of CAM economic evaluations, White and Ernst  identifiThe objectives of this paper are: 1) to introduce concepts commonly applied in economic evaluations of health technologies  so that practitioners and CAM users can translate and benefit from published evidence; and 2) present a systematic review of the current scope and quality of economic evaluations of CAM. We begin with an overview of economic evaluation, including didactic examples from the CAM economic literature to help clarify the concepts presented. Readers familiar with this type of analysis can skip this section and proceed directly to the methods section.In our systematic review we expand upon and update the initial review by White and Ernst. We evaluate study quality in more detail, using both additional study design criteria and quality of reporting criteria, and present a summary of the results from exemplary studies. While their review was the first of its kind, economic evaluations in the CAM literature have improved greatly in the last five years. We end the paper with a description of the attributes of CAM that make economic evaluation challenging and how these issues may be addressed. We hope that practitioners' interest in economic evaluation will continue to grow, leading to greater incorporation of this research into CAM trials.An economic evaluation is a comparison of outcomes among alternative ways of achieving common objectives. These analyses are conducted according to explicit, systematic, and consistent criteria, and take into account both the positive and negative consequences of each alternative. Consequences may include economic, clinical, and humanistic outcomes, known as the ECHO model . EconomiThere are several forms of economic evaluations that can be performed  and each differs based on the selection and measurement of health outcomes. The perspective (or point of view) taken for the analysis also influences the selection and measurement of consequences, because not all outcomes are important to all decision makers. Generally, there are three perspectives for economic analysis: individual , institutional , or societal. The societal perspective accumulates all outcomes, while individual and institutional analyses are more selective. Regardless of perspective, the objective of an economic evaluation is to provide information on consequences relating to alternatives faced by a decision maker.The most basic form of economic evaluation is a table that lists the individual economic and health outcomes of alternative interventions. This table is known as a cost-consequence study. Cost-identification studies and cost-minimization analyses only address economic outcomes and are discussed below in that section. The remaining forms of economic evaluations summarize economic and health outcomes into a single result Table .The advantages of performing cost-benefit and cost-utility analyses are that multiple outcomes are summarized into a single unit, either monetary units such as dollars (CBA) or QALYs (CUA) and that therapies with different sets of health outcomes can be compared based on the differences in the summary measures. Cost-benefit analysis has the additional benefit of directly indicating whether the therapy pays for itself.The disadvantages of CBA and CUA come from the techniques required to produce a summary measure. Cost-benefit analysis requires putting a monetary value on all health outcomes (and ultimately on life), and cost-utility analysis assigns value to health outcomes based on their contribution to quality of life under the presumption of population-based preferences. An extensive literature addresses the methodological and theoretical issues involved in the construction of these summary measures. The process usually occurs in two steps. In the first step, health outcomes of the intervention are measured, and in the second the outcomes are valued in summary units and aggregated. Cost-benefit analyses often assess the monetary value of health outcomes based on willingness-to-pay using a technique called conjoint analysis -14. WillCost-effectiveness analysis (CEA) is the current standard in the literature, and has the most straight forward interpretation. Under CEA, therapies useful for a specific disease or condition can be directly compared using a metric of effectiveness relevant to that condition, such as blood pressure control. Although these types of analyses do not allow a summary of multiple outcomes they tend to respond well to the most urgent questions, such as how much would it cost to reduce the number of gestational diabetes cases by 10%? Clearly, a reduction in gestational diabetes cases has measurable implications in quality of life and economic units, but the creation of a summary measure is not necessary to address the decision maker's question.No matter the approach taken, it is recommended that the estimated outcomes  of health care alternatives used in economic evaluation are best estimated in pragmatic clinical trials that directly and realistically compare the therapies of interest . Rarely Economic outcomes are the net bundle of resources forgone due to an intervention valued at the opportunity cost of those resources . Since the cost of a therapy differs depending on whether you are a patient, a health plan, or a health care provider, the economic outcomes  of each therapy depend on the perspective of the study. Studies that only measure the economic outcomes of interventions are known as cost-identification studies. A study that describes the economic and health outcomes of a single therapy can also be called a cost-identification study. These studies inform full economic evaluations. That is, they provide the data needed to better design future studies that consider both the economic and health outcomes of two or more alternative therapies. A cost-minimization analysis (CMA) explicitly assumes equivalence in health outcome among alternative therapies, and examines only economic outcomes. In practice, it appears the same as a cost-identification study, but under the assumption of equivalence, a CMA is a full economic evaluation.Table It is recommended that economic outcome data are best collected prospectively as part of a pragmatic clinical trial ,4,19,20.Our systematic review of the CAM economic evaluation literature (presented below) revealed no cost-consequence studies and no cost-benefit analyses. However, we did find examples of a cost-identification study, cost-minimization analysis, cost-effectiveness analysis, and cost-utility analysis. These examples are presented below.Frenkel and Hermoni, 2002 , performHerron and Hillis, 2000 , retrospFranzosi et al, 2001 , prospecKorthals-de Bos et al, 2003 , performThe National Center for Complementary and Alternative Medicine (NCCAM) defines CAM as "a group of diverse medical and health care systems, practices, and products that are not presently considered to be part of conventional medicine ." We furWe searched the following electronic databases from January 1999 to October 2004: Medline, AMED, Alt-HealthWatch, and the Complementary and Alternative Medicine Citation Index via NCCAM and the National Library of Medicine (NLM). Searching was restricted to English language journals and human studies with the keywords: complementary medicine or alternative medicine, and costs or cost analysis or cost-benefit or cost-effective or economic analysis or economic evaluation.We removed duplicates from the search results and selected papers that reported original data on specific CAM therapies from any form of standard economic analysis, analysis of costs, or economic modeling. Studies were then excluded if they were cited in the White and Ernst review , or if tThe following data were extracted from each of the included studies: full citation information (author(s), date, title, journal, etc), form of economic evaluation (stated or inferred), the therapies being compared and whether the CAM therapies were being used in addition to usual care (complementary) or instead of usual care , the perspective of analysis (stated or inferred), the study design, the sample size, and summary results.The studies were categorized as either full economic evaluations  or partBMJ Economic Evaluation Working Party [We captured data on quality of the full economic evaluations using two approaches. The first approach was to gather from each study the data needed to assess quality according to a 35-item checklist developed by the ng Party . This chAs the purpose of economic evaluations is to inform clinical practice and health policy decisions, the best evaluations are timely and use the best data available at the time . On the Based on the study quality criteria, we report summarized results of the exemplary studies – ie, those meeting all four study quality criteria. If the health outcomes for one therapy are better than that of its alternative and the economic outcomes are better or equal , that therapy is said to dominate (be clearly better than) its alternative. This is also the case if both therapies have equal health outcomes and one has lower costs. In all other cases, the decision maker must elect whether the increase (loss) in health benefits is worth the increase (savings) in cost.The database search rendered 1765 potential studies to screen. Application of inclusion and exclusion criteria reduced the list to 56 economic evaluations ,27,31-82BMJ 35-item quality checklist [Table hecklist  to the 3hecklist ,83,84.These checklist items indicate whether essential components of the study design were reported. About half the studies stated the form of the economic evaluation, however, several were stated incorrectly and only one justified the form chosen. The bulk of the studies presented cost-effectiveness analyses (36 or 92%), five presented cost-utility analyses, and one was a cost-minimization study . Only onThese checklist items relate to the presence of information essential to the generalizability of study results. All studies that included health outcomes  reporteAbout two-thirds of studies reported resource use quantities separate from unit costs, or described the methods used to estimate both quantities and unit costs. Whereas, almost all reported the currency used, only a minority (16 or 41%) reported the currency year. A smaller number reported the details of adjustments for inflation or currency conversion, but this was not often required in studies collecting and reporting data in the same year and currency. Models (one decision tree model  and two All studies stated the time horizon for costs and benefits and most 35 or 90%) reported a time horizon of one year or less. Items 23 through 25 apply only to the four remaining studies with time horizons longer than one year. The discount rate is reported in two of these studies (one with a time horizon of 42 months ), but on or 90% rIn about one-third of studies there was some comparison of study results to that of other studies. In most cases this was done as a simple statement noting that the results were either similar, or that they were dissimilar and that this might be because of differences in study design. Incremental cost-effectiveness or cost-utility ratios are usually only required when one therapy offers clearly better health outcomes than the other, but at a higher cost. In the 13 studies where this was the case over one-half reported incremental analyses. In most cases the major outcomes of the studies were shown disaggregated, and the study question was answered. We did require that a proper research question be stated (see the answers to item 1) for it to be answered. In all cases, we felt that the conclusions followed the data, but in about one-third of cases the conclusions were not presented with the appropriate caveats. For example, if a study did not explicitly discuss its limitations, it was not included as meeting the last item.Twenty-seven studies (69%) gathered cost data prospectively and 21 54%) used randomly assigned comparison groups. In 32 studies 82%) the physicians and patients were not blinded to the treatment received and participation was not mandatory (a worksite intervention ), and th% the phy4% used rThe number of economic evaluations of CAM has increased in recent years, even if we only count full evaluations of alternative therapies. Study quality has also increased, and although reporting quality can use improvement, it is on the whole similar to that seen in economic evaluations of conventional medicine. Nevertheless, there are still too few good quality evaluations to draw many conclusions about the cost-effectiveness of specific CAM therapies for particular conditions.A possible explanation for the paucity of studies is that there may be less of an incentive to perform economic evaluations of CAM. Consumers are already spending a large amount of their disposable income on CAM without formal proof of effectiveness or cost effectiveness. Economic evaluations are typically required for the incorporation of therapies under traditional financing mechanisms and for adjustment of coverage under these mechanisms. Therefore, the market for economic evaluation in CAM may be small due to reduced involvement of third-party payers in CAM financing.Some CAM practitioners do not see the need for economic evaluations. An interesting study by Kelner et al  asked chBMJ checklist was mostly objective , a fair amount of interpretation was still required for many items. For example, in our review we interpreted Item 1 as whether the study stated either a specific research question or study objectives in terms of economic and health outcomes. Three-quarters of the full economic evaluations of CAM met this criterion. However, in Jefferson et al, 1998 [The reporting quality was poor for certain items, but was comparable to the quality found by systematic reviews of economic evaluations in conventional medicine ,83,84. ASeveral studies have shown that at least some aspects of quality in economic evaluations improve over time ,87. Our We found that 14 (36%) of full economic evaluations met all four study quality criteria and were identified as exemplars. However, the evidence from these criteria must be interpreted cautiously; meeting all study quality criteria does not guarantee an adequate study design. Some aspects of what makes a good pragmatic trial could not be judged by what was reported. For example, pragmatic trials enroll patients typical of normal caseload in typical settings with average physicians following them under routine conditions . JudgmenA study may also be of "poor" quality because it applied the CAM therapy inappropriately. This can happen when a study is designed by researchers not familiar with a therapy. In response to this problem researchers and practitioners of several CAM therapies have begun development of standards for research and reporting. Reporting standards do not guarantee that the therapy was used appropriately, but they at least allow determination of what was done. One such set of reporting standards are the STRICTA recommendations for acupuncture . Of the The exemplary studies summarized in Table On the surface one might expect that therapies that substitute for usual care  would be much more likely to be cost effective. In this sample of exemplar studies, of the nine study comparisons where CAM therapies were shown to be superior to usual care , four were studies of complementary therapies. Therefore, there is evidence that even though complementary therapies are given in addition to usual care, they can improve clinical outcomes without increasing costs.In many ways the economic evaluation of CAM therapies is similar to that of conventional medicine. However, there are a number of issues specific to CAM that must be considered. These issues can roughly be divided into three groups: those involved with the impact of economic evaluation on CAM in general, those involving the estimation of health outcomes , and those specific to CAM's economic and humanistic outcomes. The first group of issues has already been addressed above under the potential reasons for paucity.The methodological challenges involved in determining the clinical effectiveness of CAM have been discussed at length in a number of papers. These include the appropriateness of population-based studies when individualized treatments are used and individualized outcomes are expected -91, reduHowever, there are several additional issues specific to CAM humanistic and economic outcome measurement which must be considered. First, although CAM therapies can be used to treat acute conditions, they are more commonly used to treat chronic disease, to prevent future disease (risk reduction), and to optimize health and well-being. Using CAM for those indications requires that long term studies be performed . HoweverEconomic evaluations in CAM must recognize that the process of healthcare itself can be effective for patients. Attributes of the process of using CAM that may have value include patient empowerment, the operationalization of patient preference for a particular type of intervention, the length and process of the consultation, and still having treatment options open when other medical approaches have failed ,98. TherOptimizing health, maximizing wellness, and enhancing well-being are patient-centered outcomes – ones that by definition require subjective measurement . EconomiThe collection of economic outcome data is complicated by that fact that in the United States and other countries many CAM therapies are available over the counter and/or are often paid for out-of-pocket. The lack of administrative claims data on CAM therapies in countries where these costs are not covered or reimbursed means that cost studies require primary data collection  . In theiDespite the challenges described for economic evaluations of CAM therapies, these studies ought to be done. Every planned trial of CAM therapies should at least consider the feasibility of including an evaluation of economic impacts. Observational studies should also include these data, and as information accumulates regarding economic impacts, these costs and cost savings can be estimated more accurately. Although in the ideal every cost category shown in Table The limitations of this study are similar to those of the other reviews. First, the reader was not blinded to journals and article authors, which may have influenced results. Second, our measures of study quality depend on the information reported in an article, and no attempt was made to judge the merits of clinical or modeling assumptions made in the analyses. Third, only one reader read all the papers and extracted all the data. This may have lead to inaccurate reporting of results, and/or a biased interpretation of study quality. To maximize accuracy, data extraction was performed at least twice for each paper with several months break between extractions. Also, the approach and assumptions used to determine study quality were discussed at length with the other authors. These discussions led to a homogeneous approach being taken to both the application of the reporting quality criteria and the definition as to what constitutes an economic evaluation.As health care costs continue to rise, decision makers must allocate their increasingly scarce resources toward therapies which offer the most benefit per unit of cost. Economic evaluations inform evidence-based clinical practice and health policy. To be considered by these decision makers, CAM therapies and their outcomes must be known and compared to conventional approaches. However, CAM practitioners must themselves decide whether the cost of performing these studies is worth the potential impacts to their profession of being considered in managed care. Nevertheless, these evaluations will be done and they will be better done with practitioner involvement. Whereas the number and quality of these studies has increased in recent years and more CAM therapies have been shown to be good value, there are still not enough studies to measure the cost effectiveness of the majority of CAM. If CAM providers wish to increase the provision of therapies to improve population health, they must report the potential outcomes of CAM therapies widely and well.The author(s) declare that they have no competing interests.PH had the main responsibility for the manuscript and for bringing together the concepts of CAM and economics. PH also read and evaluated the quality of all papers included in the review. BC ensured that the health economic concepts were presented appropriately. PH and OC conceived the idea for the paper. OC contributed clinical and methodological insights. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:Descriptions of included studies ordered by complementary and alternative medicine (CAM) modality, form of economic evaluation, and publication date. The appendix contains a table summarizing each of the 56 economic evaluations found in the systematic review. For each evaluation the following are reported: therapies compared, study population, study design and sample size, whether it was a full or partial economic evaluation, form of the evaluation, perspective, and summary results.Click here for file


Some organizations have established a set of minimal standards every medical web site should meet. On the other hand, certain evaluation and rating systems rank plenty of web resources following subjective criteria. Finally, some authors have proposed the citation analysis of medical resources on the Web as a democratic way to evaluate and rank large amounts of medical web sites.A sample of pediatric web sites compiled from eight evaluation and rating systems which give the results of their evaluation as numeric or analog scales were studied in april 1998 about three characteristics: number of daily visits, time since their last update, and the number of inbound links. The reliability of those three characteristics as quality markers was studied by comparing them to the web sites evaluations, and the same web sites were studied again a year later in order to know their predicitive power as quality indicators. In addition, we studied some cybermetric parameters defined in basis to the number of inbound links and the size of each web site, which were calculated by means of the search engine Infoseek.363 pediatric web sites were included in the 1998 survey. After removing some not available web sites (11%), we studied 134 domains or subdirectories and 188 web sites in 1999. The variables showed certain stability a year later, that is, the more updated and visited a web site was, and the higher number of inbound links received in 1998, the more updated, visited and linked was in 1999 . In the 1999 study, several quality markers were found. So, the number of inbound links, their absolute increment for the last year, the grade of web sites update, and the size (number of pages) of the domains and subdirectories, significantly correlated with their evaluations by the rating systems.This study demonstrates not only that certain web sites characteristics are good quality markers of them, but also that they have certain predictive power as quality indicators. Similarly, a dinamic quality parameter has risen, such as the absolute increment of inbound links for a period of time. Certain cybermetric parameters based on the absolute number or the increment of inbound links to a medical web site could be good cybermetric indexes. However, this method have got some technical limitations. For instance, it is sensible to changes in the syntaxis of the web sites URLs, and it depends on the power of the search engine database we employe. In any case, a ranking based on this method would be so reliable as one based in the web sites evaluation by third parties. Moreover, some cybermetric indexes based on this method could provide a means of dinamic evaluation of a given web site


Although vaccination has been proved to be a safe, efficacious, and cost-effective intervention, immunisation rates remain suboptimal in many European countries, resulting in poor control of many vaccine-preventable diseases.The Summit of Independent European Vaccination Experts focused on the perception of vaccines and vaccination by the general public and healthcare professionals and discussed ways to improve vaccine uptake in Europe.Despite the substantial impact and importance of the media, healthcare professionals were identified as the main advocates for vaccination and the most important source of information about vaccines for the general public. Healthcare professionals should receive more support for their own education on vaccinology, have rapid access to up-to-date information on vaccines, and have easy access to consultation with experts regarding vaccination-related problems. Vaccine information systems should be set up to facilitate promotion of vaccination.Every opportunity to administer vaccines should be used, and active reminder systems should be set up. A European vaccine awareness week should be established. Compared with other healthcare interventions, vaccination has proved to be one of the most cost-effective health measures of the 20th century, and it is credited with a substantial role in the overall increase in life expectancy. However, despite the proven safety and efficacy of vaccines, immunisation rates remain suboptimal in many European countries, and some common vaccine-preventable diseases are not controlled to the extent to which they could. For example, in 1997, the World Health Organization set the objective of eliminating measles from Europe by 2007. To reach this goal, at least 95% of the population should receive two doses of the measles vaccine. Recent data demonstrate that this is not the case. Measles vaccine coverage remains too low to eliminate the disease, and the virus is still circulating in many western European countries. In 2006, Germany was hit by a large measles epidemic [Several factors, including inconsistent vaccination systems, lack of political will, and poor understanding or false perceptions of vaccination by the public and healthcare professionals (HCPs) have been identified as barriers to vaccination in Europe . The rolIn developed countries, the general public usually has a positive view of vaccines and considers vaccination important. This has been shown in several surveys carried out in Canada ,10, the A closer look at the results demonstrates, however, that the situation is not as clear-cut as the figures might suggest. A detailed analysis of the EVM survey showed that only 37% of the respondents had a "very positive" perception of the value of vaccinations, whereas 45% of them were more reserved and indicated only a "somewhat positive" perception. A recent study in the US  showed tWhy is the public perception of vaccines so variable? Concerns and misconceptions about vaccines are common and have a negative influence on attitudes towards vaccination. In several studies, fear of side-effects has been expressed as the most frequent reason for not vaccinating children and adults ,21. OtheA lack of knowledge about vaccines among the general public, especially regarding safety issues, has been confirmed by recent studies. In a Canadian study, one-third of respondents thought that they did not know enough to comment . In a stIt is paradoxical that one of the main barriers to vaccination is the huge success of many vaccination programmes. After the near disappearance of the target diseases from everyday life, the diseases and their complications no longer serve as a healthy reminder of the continuing need for prevention through vaccination. Many individuals, HCPs included, have never witnessed the debilitating diseases that the vaccines prevent. This has resulted in increased negligence about vaccination, and the balance has shifted from emphasizing the true benefits of vaccination to increased suspicion of adverse effects of vaccination. Parents who are reluctant to have their children or themselves vaccinated often consider vaccine- preventable diseases as either rare or mild, or irrelevant to their own children ,25,29,30The media plays an important role as a source of information for the general public. However, the quality of the information provided by the media is variable and can sometimes be sensational. Many individuals have strong opinions on vaccinations despite being ill-informed about vaccine-related issues . There iSunday Times that revealed a conflict of interest of the author of a report linking the measles, mumps and rubella (MMR) vaccination to autism did much to reduce the credibility of the author's conclusions in the eyes of the public. However, in many respects, the media were responsible for giving the false alarm about this report in the first place [The media can have a positive or a negative effect on the public perception of vaccination. Unfortunately, recent vaccination scares have sometimes been mixed by the media with unrelated health scares such as bovine spongiform encephalopathy or blood contaminated with human immunodeficiency virus, which have undermined public faith in government healthcare policies . Howeverst place .It must be emphasized that a long-term confidential relationship between medical experts and journalists is of crucial importance. Their relationship is an essential foundation for educating the public in health-related matters.Common misconceptions could be overcome by delivering accurate, reliable, and positive information on the benefits of vaccines and the minimal risks associated with their use, as well as increasing awareness of the diseases that the vaccines prevent . Such prWe identified some key factors in communicating with the general public. First, the public deserves credible and trustworthy information; that is, clear, reliable, and up-to-date data on vaccine-preventable diseases and vaccine safety and efficacy from sources they trust. Second, it is important to communicate positively, not in a "defensive" way or by raising fears about vaccine-preventable diseases. Vaccination should be seen as an initiative for promoting good health, and getting vaccinated could be seen as establishing "peace of mind". For parents, vaccination of children could be viewed as complying with a high standard of childcare and as part of "good parenting practices". Of course, the information should be adapted to the different subsets of the population . DecisioAlthough demographic considerations are important in identifying target groups, individual perspectives and attitudes must also be considered for such an emotive subject as vaccination. For example, four distinct groups of parents have been described on basis of the subjects' attitudes towards vaccination: "vaccine believer", "cautious", "relaxed", and "unconvinced" . MessageAlthough education of the general public has been shown to be successful, the most effective methods to be used have not been well elucidated . A combiHCPs have a key position in vaccine uptake. They serve as an important source of information for the general public and are the main drivers of vaccination programmes. In the EVM survey ,20, 68% With respect to vaccination, the opinions of HCPs are often more important than the parents' or patients' own points of view. Many parents of fully immunised children express similar attitudes and beliefs to parents of underimmunised children. The determining factor in vaccine acceptance is the HCP's attitude . PhysiciA role in vaccine acceptance is not limited to family physicians and paediatricians. Nurses, pharmacists , midwiveHCPs have generally positive attitudes towards vaccination. In the EVM study, 84% of the 800 healthcare professionals interviewed affirmed very positive perceptions of vaccines and the value of vaccination, and another 14% had somewhat positive perceptions ,20. HoweHCPs often lack precise information on specific issues. In an American study , over 90Most HCPs participating in the EVM study ,20 were . Created in Switzerland in 2000 and established in France in 2003, INFOVAC is an interactive consultation/information system on vaccines and vaccination for general practitioners and paediatricians. It is run by a network of academic paediatric experts in infectious diseases, who can be contacted at a central e-mail address. HCPs can address every question concerning vaccination and be assured of an answer from an expert within 24–48 hours.HCPs need rapid access to relevant, up-to-date, and objective information, instead of having to rely on press releases, leaflets from pharmaceutical companies, or the media. One successful initiative in this context is INFOVAC INFOVAC also distributes monthly bulletins on the latest developments in vaccinology . One of Another successful initiative, started in 2001 by the University of Antwerp, Belgium, is an annual vaccination "Q&A" day. The day is organised without any support from industry in order to avoid any real or perceived bias or conflict of interest. The participating HCPs (on average 350–500 physicians and nurses) set up the programme by sending in their questions in advance, and a panel of local experts is invited to respond. The goal of these annual meetings is to provide up-to-date information for the participants .HCPs not only need information regarding particular vaccines but they also need to be informed about the epidemiology and effect of infectious diseases and about the vaccination rates in their country or region to be able to design appropriate strategies.In many countries, HCPs act as the principal source of information on vaccines and vaccination for their patients, and thus they are essential for the population at large. Several studies have shown that this places a heavy time burden on the already tight schedules of HCPs. Lack of time is an important cause of missed vaccination opportunities . In one Any extra time spent by HCPs addressing vaccine issues is not time wasted. Practices that allowed more time for acute care visits and used more immunisation promotion activities were found to have higher influenza and pneumococcal vaccination rates among adults >65 years of age .Many people do not get fully vaccinated, not because they are against vaccination, but because they tend to forget about it and/or because vaccination schedules are too complex to remember. Active reminder systems such as postcards, telephone calls, or other forms of communication should be set up. Reminders to both patients and providers have been shown to increase vaccination uptake ,24, but Centralised computer systems that would rapidly inform the HCPs about the vaccination status of their patients could substantially promote vaccination. Besides providing important information about the patients to the HCPs, centralised computer systems could be used to provide timely information on local vaccine coverage levels. Availability of reliable data on vaccine coverage is essential for the evaluation of the effectiveness of any intervention strategy . ComputeComputerized systems are already in place in some regions of Europe. For example, in the autonomous region of Murcia Province, Spain, 99% of newborn babies are entered into the Registry of the Computerised Vaccination Programme of the Directorate General of Public Health. All parents receive a letter of introduction, a vaccination booklet, and a card with a barcode for each vaccine that the child should receive during the first 2 years. Vaccines administered are recorded on the card, which is connected to the registry managed by the data management centre. This registry provides: (i) a list of properly vaccinated persons; (ii) a list of insufficiently vaccinated persons who are periodically reminded by mail or telephone of the convenience of keeping up with the schedule; and (iii) a certificate of vaccination status .In Flanders, Belgium, a pilot electronic vaccination database, Vaccinnet, was set up in 2003 for infants visiting "well baby" clinics . By 2006Every opportunity to administer vaccines should be used. For example, patients discharged from hospital for pulmonary disease should be offered immediate injection of pneumococcal vaccine and, in the autumn, of influenza vaccine. MMR vaccine should be offered to seronegative women when they are discharged from the maternity ward. Easy access to vaccination in non-traditional settings should also be considered. Depending on national and local circumstances, vaccinations could be performed in settings such as childcare centres and nursing homes and during home visits. In addition, where available, access to vaccination centres should be facilitated, with evening and weekend opening hours. Increasing opportunities for vaccination will increase vaccine uptake.Standing orders that authorise nurses and pharmacists to administer vaccines without a physician contact according to an institution-approved or physician-approved protocol have proved successful in increasing vaccination rates in adults . They weVaccination rates in Europe are not what they should or could be. Public perception of vaccination is important, but rather than concentrating on failures of vaccination it might be better to identify and implement strategies that focus on HCPs and that have been shown to work.Clearly, HCPs play a central role in vaccine uptake. They remain the most important sources of information to the public and the most important advocates of vaccination although their contributions are often unappreciated and underused. To feel confident in carrying out their responsibilities, they need the support of health authorities and they also need to be sufficiently informed about vaccine-related issues and problems. The INFOVAC network is a good step in this direction, and it could be expanded to other countries. In addition, medical education should be revised to include additional training in vaccinology in the medical, pharmaceutical, and nursing curricula. The general public could also benefit from better education. Many of the fears and concerns of lay persons could be overcome by easy access to accurate information presented in an understandable format.Even if all the information would be available, however, the lack of time for communication remains a great barrier to vaccination and must be addressed. Moreover, once the idea of vaccination is accepted, the opportunities for administering the injections themselves need to be increased.Many of the issues highlighted here could be addressed during a European Vaccination Awareness Week. Such an event could involve the media to inform the general public, to raise awareness, and to promote dialogue between HCPs and the public. Such events have been successfully implemented in the USA  and CanSIEVE is supported by an unrestricted educational grant from the Stiftung Präventive Pädiatrie at the Johannes-Gutenberg-University, Mainz, Germany. There was no other funding source.All authors contributed to the manuscript by a country-specific presentation and by discussing the topic at a 2-day meeting held in Frankfurt, November 12–14 (2004). Following the meeting, the manuscript was prepared by a science writer (Dr Britta Gröndahl) employed by the University of Mainz at the department of the first author. The first author was in charge of producing the final version of the manuscript with input via "electronic discussions".The pre-publication history for this paper can be accessed here:


The coverage of insecticide-treated nets (ITNs) remains low despite existing distribution strategies, hence, it was important to assess consumers' preferences for distribution of ITNs, as well as their perceptions and expenditures for malaria prevention and to examine the implications for scaling-up ITNs in rural Nigeria.Nine focus group discussions (FGDs) and questionnaires to 798 respondents from three malaria hyper-endemic villages from Enugu state, south-east Nigeria were the study tools.There was a broad spectrum of malaria preventive tools being used by people. The average monthly expenditure on malaria prevention per household was 55.55 Naira ($0.4). More than 80% of the respondent had never purchased any form of untreated mosquito net. People mostly preferred centralized community-based sales of the ITNS, with instalment payments.People were knowledgeable about malaria and the beneficial effects of using nets to protect themselves from the disease. The mostly preferred community-based distribution of ITNs implies that the strategy is a potential untapped additional channel for scaling-up ITNs in Nigeria and possibly other parts of sub-Saharan Africa. A current challenge that is facing many sub-Saharan African countries like Nigeria is how to achieve widespread distribution and use of insecticide-treated nets (ITNs) for the control of malaria. The Africa Malaria Report shows that many countries are quite far from reaching the target of 60% ITNs coverage in sub-Saharan African countries by the year 2005, which was set in Abuja by the African Heads of State for the provision of ITNs to children under five and to pregnant women . MalariaThe determination of distribution mechanisms that will assure high coverage with the ITNs, especially in rural areas, remains a topical issue in Nigeria and in many sub-Saharan African countries (SSA). The public health care system was initially used to distribute ITNs in Nigeria, but the coverage was quite low. Currently, commercial sector distribution and social marketing of ITNs is being promoted in some states in Nigeria, but the coverage remains low. Community-based distribution has not been tried on a large scale, but it could possibly be added to existing strategies to successfully distribute and scale-up ITNs in rural areas. Similar strategies are being successfully used for the distribution of ivermectin for the control of onchocerciasis , filtersChavasse et al  identifiThis paper examines the perceptions, expenditures and preferences of consumers for the prevention of malaria, as well as ITNs ownership and preferred strategies for distributing ITNs. Such information should precede the design of sustainable and effective locally relevant strategies for scaled-up distribution of ITNs. The information will also inform policy makers on the nature of information, education and communication campaigns that could be used to increase ITNs coverage. A successful social marketing strategy for ITNs in Tanzania was developed based on a similar assessment . The infAnopheles gambiae, while Plasmodium falciparum causes more than 90% of all malaria infections. Achi has an estimated population of 45,000 people. Untreated mosquito nets and ITNs are not sold in these villages, but they are sold in urban areas like Enugu and Onitsha. It usually takes about one hour to reach Enugu and one hour and 20 minutes to reach Onitsha by bus.The study was conducted in three villages in Achi community, Oji-river local government area (LGA) of Enugu State, south-east Nigeria. The villages were Ahani, Amaetiti and Enugwu-Akwu. Achi has a high malaria transmission rate, year round, with an average malaria incidence rate of 15%. The majQualitative data was collected through three focus group discussions (FGDs), with separate groups of men, women and youths. Hence, a total of three FGDs were held in each village . The number of participants per FGD ranged from six to nine people and the participants were purposively selected with the help of the village heads, so that all sections of the villages were represented. The agegroups for men and women were from 35 to 60 years and for youths 20 to 34 years. Youth groups had equal numbers of male and female participants. A discussion guide was used to direct the discussions during the FGDs, which were moderated by a social scientist. Each lasted a maximum of two hours. The discussions focused on causes of malaria, how to prevent the disease, level of net ownership, preferences for distributing and paying for the nets within their villages.Quantitative data was collected using a pre-tested interviewer-administered questionnaire that was administered to respondents from a total of 900 households (300 from each village). Adequate sample size was determined, using a power of 80%, 95% confidence level and a malaria incidence rate of 15%. The EPI-info software programme was used to calculate the sample size. The heads of households or their representatives (if the household head was not available) from the selected households were interviewed. The participants to the FGDs were excluded from the quantitative survey so as not to bias the results, since they already had better information than others. Data was collected on the socio-economic and demographic characteristics of the households, the level of their ownership of mosquito nets and their expenditure on the prevention of mosquito nuisance and malaria in the month prior to the survey. Expenditure included mosquito nets, coils, insecticide-sprays, drugs, body creams etc. The Ethics Committee of the University of Nigeria Teaching Hospital, Enugu, Nigeria approved the study.The variables that were explored by both the FGDs and the questionnaire focused on determining the most common diseases in the community, local names for malaria, the different types of malaria, its symptoms and causes. Other points explored were the perception of mosquito nuisance, the subject' s health care seeking behaviour, mosquito control effort and solution to the malaria problem. Participants were also asked their preferences for paying and distributing ITNs. The possible major avenues for delivery of ITNs, such as through the commercial sector, the public health system, community-based distribution and social marketing were explored. The records of the FGDs were transcribed on the same day the FGDs were held and content analysis was used to categorize the responses into domains representing the common themes. The areas of consensus and divergence in the responses according to the groups and villages were determined for better identification of factors that influence health seeking behaviour. Tabulations and tests of statistically significant differences using non-parametric chi-squared tests were used to analyse the quantitative data.The local terminology for presumptive malaria was "Iba", which denotes fever. Many causes of Iba were identified by people, though as a consensus, most of the participants in all the groups identified mosquitoes as its major cause. The methods given by the participants on mosquito nuisance control included insecticide sprays, keeping doors and windows closed in the night, use of electric fans, applying mosquito repellent cream on the exposed body at night, clearing weeds around the house, wearing long dresses, applying kerosene to the body and in the house, use of mosquito nets and physically killing mosquitoes. Other mosquito control techniques were draining stagnant water and covering water containers, especially during the rainy season, and burning or placing local leaves (osigbu) in and around the house. The leaves are good mosquito repellents because of their smell. Some participants said that preventive drugs such as paludrine were used, especially by pregnant women."I will never use mosquito bed-nets even given free of charge. They make me feel as if I am being suffocated". Only one man from Enugu-Akwu had heard about ITNs through the radio. Many questioned their safety, since they felt that sleeping under nets treated with insecticides may be harmful. Most people were excited when told about ITNs and their mode of action and indicated their willingness to buy the nets when available, if affordable and not harmful.The discussions showed that there was low mosquito net ownership and use in these three villages. A few Amaetiti youths said that they had used bed-nets in secondary school boarding houses. All the Enugu-Akwu women said that they did not use mosquito nets due to the high cost, while one of them said that nursing mothers used nets to cover their babies. Though, window-nets were preferred to bed-nets, lack of money was the major constraints on net ownership, as people indicated that they would have bought nets, but for the high cost. Window-nets were preferred over bed-nets because they provided better aeration to the users and did not generate heat. A woman from Amaetiti said, "If the ITNs are taken from house to house, people will value them less, because they will feel that the government was unable to sell them and is begging them to buy the nets".Almost all groups preferred community-based distribution of the ITNs to sales by the commercial sector , public health system and by health teams that occasionally visited the villages. The major reason for their preference was that community-based distribution would make access to the ITNs easier and the nets would not be too expensive, since the distributors would be resident in the villages and the nets would be sold at uniform prices that would be fixed by the government or the major suppliers. The majority of people felt that the profit motive could make the nets from the commercial sector very expensive. Also, they felt that teams coming to their villages to promote and sell the nets might come irregularly and, when they did, people might not have money to buy the nets. Most of the participants agreed that ITNs should be sold centrally within their communities. Some people also suggested the use of the door-to-door sales method for the nets, especially for people unable to go to the central location to buy the nets. However, the Ahani youths were against door-to-door sales. As one of them explained "The price of the nets should be very low. They should be fixed prices and the ward leaders will tell all households about the price". Another stated that "Trained field workers should educate the people on the importance of the nets, and then seek their opinion on the prices for the net. Then, the average price should be used as what people should pay".There was a group concensus that single full payment was the best payment system, if people could afford to pay the full price at once. However, in the event that the price of the nets is high, then instalment payments should be allowed. A one-off pre-payment system was not preferred, because income in rural areas fluctuates and even people in the formal wage-earning sectors, who get paid monthly, have competing needs, or their salaries may go unpaid for many months. Additionally, a one-off payment could prove to be a big burden, especially if a household purchases many nets. Some people from all the groups suggested that the nets should have a subsidized, but fixed price. In the words of a man from Amaetiti: The usable number of questionnaires for the analysis were 798. The respondents were mostly heads of households, middle-aged, females, married and farmers Table . More thA small proportion (25.2%) of the households spent money on malaria prevention and the average monthly expenditure to prevent malaria was 55.55 Naira ($0.5) Table . ConversPeople were knowledgeable about the mode of transmission of malaria and the benefits of using malaria preventive methods such as nets and insecticides, but very few households spent money on malaria preventive tools. The acquisition and usage of untreated mosquito nets was low and nil for ITNs and very few people had heard about ITNs. The results of this study are comparable to those of a marketing survey in Nigeria, where 10% of 5,000 households owned at least one net . The resThe lack of substantial expenditure on malaria prevention means that getting people to pay for ITNs at the unit cost of 450 Naira ($3.8) per net, which is the average prevailing market price of ITNs in Nigeria, would be an uphill task. People would need to be convinced to increase their budget on prevention so as to cover the expense of using nets, especially in areas without a net usage culture. This depends, too, on whether they perceive ITNs as a complement to or a substitute for existing malaria preventive measures. Since, health care expenditures are the minimum amount or lower bound estimate of the amount that people are willing to pay for health care , the expAn implication of these findings for scaling up of ITNs in rural areas is that malaria control programme managers should design and fine-tune how community-based distribution of ITNs could be added to existing distribution strategies so that ITNs could penetrate into the rural areas in large numbers. Community-based distribution in this context involves the recruitment and training of community residents to become ITNs community-based distributors (CBDs). The CBDs could be recruited in conjunction with the community leaders and trained by promoters of ITNs distribution such as the state and local government malaria control programme officers as well as other non-governmental promoters of ITNs. It is hoped that the CBDs would regularly obtain the ITNs from both public and non-governmental sources for sale to their community members. For sustainability of community-based distribution, the CBDs should either be paid a stipend directly by the body that supplies them the nets or allowed to slightly add a mark-up on the sale prices of the nets, which they would collect as their commission. The primary healthcare system would be expected to supervise, monitor and evaluate the CBDs as well as provide them with continuous re-training. They will also train new CBDs, when there is CBD attrition.The malaria control programme managers should also consider how they would tackle the villagers' reluctance to prefer the commercial sector and vertical teams, so that these other strategies would also be useful in rural areas in the medium to long term basis. This is because the community-based distribution strategy would be used alongside the distribution of the ITNs through public and private healthcare facilities as well as through the commercial sector.The payment mechanism within community-based distribution of ITNs should be designed to limit the occurrence of payment defaulters. The mostly preferred instalment payment before ITN acquisition ensures that people will get the ITNs whenever available, even if they have irregular availability of cash. Nonetheless, a mixture of instalment payment and one-off payment could be used in order to avoid the pitfalls of instalment payment, where people default in completing their payments after collecting the ITNs. The village heads could be made to act as guarantors for people who would be allowed to pay by instalment. However, subsidy mechanisms such as vouchers , subsidiA cost-effective use of resources is the promotion of multiple compatible distribution strategies in communities by government and other organizations, so that there would be an appreciable scaling-up of the supply and use of ITNs. The use of multiple distribution strategies within rural areas is in line with the pluralistic approach which has been advocated by the WHO Strategic Framework for Scaling-Up ITNs . Using sWhat is needed to scale up ITNs in rural areas is more effort to develop and implement consumer preferred distribution strategies, which will complement more widely used distribution strategies such as the commercial sector and social marketing. Motivational health education would encourage people to increase their current low level of expenditure on malaria prevention, so that they would be able to buy ITNs and re-treatment services. Examples of where community-based distribution did not work  and wherOO conceived and designed the study. All the authors participated in data collection and analysis. OO wrote the first draft and all the authors revised the drafts until the final draft was produced for publication.


NorthStar, a major product of the ReBEQI project.The Research-Based Education and Quality Improvement (ReBEQI) European partnership aims to establish a framework and provide practical tools for the selection, implementation, and evaluation of quality improvement (QI) interventions. We describe the development and preliminary evaluation of the software tool NorthStar on the design and evaluation of QI interventions. A lead individual from the ReBEQI group drafted each section, and at least two other group members reviewed it. The content is based on published literature, as well as material developed by the ReBEQI group. We developed the software in both a Microsoft Windows HTML help system version and a web-based version. In a preliminary evaluation, we surveyed 33 potential users about the acceptability and perceived utility of NorthStar.We focused the content of NorthStar consists of 18 sections covering the design and evaluation of QI interventions. The major focus of the intervention design sections is on how to identify determinants of practice (factors affecting practice patterns), while the major focus of the intervention evaluation sections is on how to design a cluster randomised trial. The two versions of the software can be transferred by email or CD, and are available for download from the internet. The software offers easy navigation and various functions to access the content. Potential users (55% response rate) reported above-moderate levels of confidence in carrying out QI research related tasks if using NorthStar, particularly when developing a protocol for a cluster randomised trialNorthStar is an integrated, accessible, practical, and acceptable tool to assist developers and evaluators of QI interventions. Clinical evidence is essential to enhance the delivery of care and improve patient outcomes. Yet the transfer of research findings into clinical practice tends to be a slow and haphazard process. One of the central aims of quality improvement (QI) activities is to enhance such transfer. In spite of the growing body of rigorously conducted studies on the effectiveness of various QI strategies , the strIn 1999, the European Commission (EC) published a report that considered various ways to improve the effectiveness and efficiency of European health care systems . The rep1. To establish a framework for the selection, implementation, and evaluation of interventions to promote evidence-based clinical practice.2. To provide clinicians, policy-makers, and researchers with a suite of evidence-based, Internet tools, guided by the framework, to inform the development and evaluation of interventions to promote evidence-based clinical practice.NorthStar is a major product of the ReBEQI project. It is a software program that packages information on the design and evaluation of evidence-based QI interventions into an integrated, easily accessible, and practical tool. In this paper, we provide a general overview of the guiding principles, content development and software development of NorthStar and report its preliminary evaluation. We refer readers to the ReBEQI website for further information on ReBEQI and to see the NorthStar content in full [ in full .NorthStar was guided by four principles. First, the information and tools it offers need to be evidence-based, while being explicit about the limitations of existing evidence. For example, whilst there is a growing body of evidence on the effectiveness of various QI interventions, it is recognised that no single intervention will be consistently effective across a wide range of problems and contexts [NorthStar has to be appropriate to the diverse needs of its target audience, who may have varying levels of skills and experience in quality improvement and research. Third, NorthStar has to be simple to read and navigate to ensure its relevance, acceptability and usefulness.The development of contexts . Second,The target audience of the tool include QI researchers, healthcare professionals and managers responsible for developing, delivering and evaluating continuing education (CE) and QI programmes at a national or regional level. Consequently, the two areas we covered were the design of QI interventions and their evaluation. The intention was for the tool to be useful to users with varying levels of experience in QI research, and for the text to be instructional as well as informative. For example, when describing the different types of cluster, randomised controlled trials (cluster RCTs), in addition to outlining the features of each type, the text would guide the user with regard to when the use of each design might be appropriate.A lead individual from the ReBEQI group was identified for each section of the tool according to individual expertise. This lead person took responsibility for preparing the first draft of that section, based on a review of the published literature as well as other outputs of the ReBEQI project (e.g. a literature review of QI models and frameworks). At least two other ReBEQI members reviewed the section. An editor ensured that the different sections were consistent with regard to content, structure and writing style. All group members reviewed the final version for wider feedback and final approval.NorthStar section on cluster RCTs is based, in part, on the Trial Protocol Tool, a product of the Pragmatic Randomized Controlled Trials in HealthCare (PRACTIHC) group [The content builds on the framework developed by ReBEQI for the selection, implementation and evaluation of QI interventions. It guides the user on how to identify "determinants of practice" in designing the intervention and on how to design a cluster RCT as the preferred evaluation study design. Determinants of practice are the factors that affect practice patterns and explain the variation in the effectiveness of different QI interventions in changing practice patterns; they are usually categorized as "barriers and facilitators" or as "moderators and mediators" ,8. The NC) group .NorthStar as a Microsoft Windows HTML help system, the standard help system used with Windows 95 and above. This choice gives the tool a familiar Windows interface and provides a range of off-the-shelf functionality. The software for running Windows HTML help, including Internet Explorer upon which it relies, is part of the Windows operating system. The basic HTML help development tool, Microsoft HTML Help Workshop, is available free from Microsoft's website. We also created a web-based version using chm2web by A!K Research Labs [We developed rch Labs  that is NorthStar, while accompanying documents such as pdf or Word files were organized as external files in the same folder as the compiled .chm file.For the content, we created HTML versions of Word documents using Mozilla 1.4 for Mac OS X. We then linked the HTML documents to the help system in HTML Help Workshop. The HTML documents can contain links to the full spectrum of electronic document formats and can, therefore, include not only textual information but graphics, documents created by other programs, sound, video, as well as links to websites and other programs. Once the HTML documents have been added or updated, the collection is compiled into a single file (a .chm file). The compiled .chm file included the main text of NorthStar users. The first group comprised eight postgraduate students attending a research methodology course in Göteborg, Sweden. The course focused on the planning of intervention studies within the broader context of health systems research. The second group consisted of 25 individuals, nominated by the seven ReBEQI partners, responsible for developing, delivering, and/or evaluating CE and QI activities. Most of them were based in the ReBEQI partner countries. We surveyed the first group during a class session in 2006, and contacted the second group by e-mail during 2005–2006. We sent non-responders one e-mail reminder two weeks after the initial contact.We undertook a questionnaire survey of two groups of potential NorthStar (with no constraints to the browsing sequence) and to fill out a questionnaire. First, we asked respondents about their main professional roles, their roles in CE and QI, and their experience in QI research. We then asked them to estimate the time spent browsing NorthStar and which sections they reviewed the longest. We also asked respondents to use a seven-point scale  to rate their level of confidence with the use of NorthStar, and in carrying out 10 named tasks relevant to QI research. A final question requested open-ended feedback on the format and user-friendliness of NorthStar. A copy of the questionnaire is available as an Additional File. We used descriptive statistics to report aggregated responses from the two groups because of the relatively small number of participants from each.We asked respondents to spend around twenty minutes browsing NorthStar consists of 18 sections covering the design and evaluation of QI interventions. After two introductory sections (About NorthStar and Introduction), NorthStar discusses how to involve stakeholders (Stakeholder Involvement). It then examines how to identify and select priorities in order to use resources efficiently (Priority Setting). Next, it provides guidance on how to identify, critically appraise, adapt, and develop relevant clinical practice guidelines . It also discusses the why, what and how of measuring baseline performance defined as the measurement of actual clinical practice and its comparison to desired clinical practice (Measuring Baseline Performance).NorthStar discusses what the determinants of practice are and how to identify them using both explorative methods and theory-based methods (Identifying Determinants of Practice). It then provides guidance on what intervention strategy to use, and how to select determinants of practice, and use them to tailor the intervention (Designing the Intervention).NorthStar assists the user in the choice of appropriate study design to test the QI intervention (Choosing the Study Design). It then discusses the usefulness, limitations, and types of pilot studies (Pilot Studies). The major section of NorthStar is dedicated to the development of protocols for cluster RCTs (Cluster RCT Protocol Tool). Next, it examines when to use and how to design an interrupted times series evaluation . NorthStar reviews two other types of evaluation . It also provides guidance on how to form or identify networks of QI researchers, health professionals, health service managers, and policy makers (CE and QI programs). Finally NorthStar provides useful resources . Table NorthStar is available as an HTML Help system, or as a browser-based version. Figure NorthStar. Both versions can be transferred by e-mail and on CD, and downloaded from the Internet [NorthStar requires an Internet connection for use, although links to websites do require a live Internet connection. The July 2006 version of the tool comprises a compiled .chm file (808 kB), an external set of libraries (9.2 Mb), and a collection of other supporting external material (6.8 Mb). Most of the main sections of NorthStar also are being translated into French and Italian.Internet . Once doNorthStar in many different ways:The user can access the content of Contents: An outline at the left helps the user to locate what he/she needs. "Books" (each containing one of the 18 sections described above) can be expanded or collapsed by clicking on the +/- sign to the left of the book symbols.1. Search: The user can search for words anywhere in the text.2. Hypertext: The hypertext links help to navigate NorthStar, for example by bringing up relevant definitions or explanations in the glossary, examples or illustrations.3. Print: Selecting 'Print' in the browser or from the Windows Help system will print the contents of the main NorthStar window.4. Libraries: These can be accessed from the 'Libraries' menu at the top of the screen or from the bottom of the contents and include: checklists, an annotated bibliography of key methodological papers, structured abstracts of selected references, summaries of key theories, and examples of protocols and final reports of evaluations of QI interventions.5. Eight participants from the student group and ten from the ReBEQI nominees responded to the survey . One respondent from the ReBEQI group provided free text responses only, so we included only 17 responses in the quantitative analysis.NorthStar was 20.6 minutes. Participants spent most of the time looking at 'Identifying Determinants of Practice'  and 'Designing the Intervention' . The levels of confidence in carrying out, with the use of NorthStar, the ten tasks related to QI research, were all above the mid-point of the scale  respondents described themselves as health care researchers; others were health care practitioners and managers. Nine (53%) reported intermediate levels of experience in QI research compared with seven (41%) who described themselves as either beginners or having no role in QI research and one (6%) experienced QI researcher. The mean time spent browsing % respondNorthStar  might also find selected components useful.NorthStar has a number of limitations. Because of the limitations of the existing evidence, many recommendations within NorthStar are based on pragmatic interpretations of that evidence. We used structured rather than systematic reviews to draft sections – although these drew upon systematic reviews. In addition, we do intend to incorporate into NorthStar further evidence on the effectiveness of QI strategies as it becomes available.However, NorthStar will form a useful reference and educational tool for QI research in general, and for developing a protocol for a cluster randomised trial, in particular. Potential users seemed mostly interested in the sections that identify determinants of practice and the design of interventions, which are both recognised as challenges requiring greater attention in the implementation field [NorthStar was limited by a small sample size, the limited representativeness of participants, the lack of a comparison, and the assessment of intermediate outcomes. For example, we have not evaluated the impact of the use of NorthStar on the processes and outcomes of development and evaluation of QI interventions. It also is possible that actual or perceived technical barriers to using NorthStar will discourage some potential users.The results of our modest evaluation suggest that on field . HoweverNorthStar is an integrated and practical tool to assist QI researchers, healthcare professionals, and managers responsible for developing, delivering and evaluating CE and QI programmes at a national or regional level. Our preliminary evaluation is broadly positive. In its current form, NorthStar is mainly designed to meet the needs of QI researchers.NorthStar is available from the 'Tools and databases' menu.Windows HTML Help version works in Windows 95 and above.Web-based system works with Linux, Mac OS 9/OS × and Windows.Windows HTML Help requires that Internet Explorer version 3 or above is installed. The browser-based version requires nothing more than a modern browser.NorthStar within a for-profit organisation in any way, please contact us to discuss a donation to our work.ReBEQI is happy to have its products used for personal and non-commercial use and at no charge in free educational programmes. However, when tuition is charged, ReBEQI royalty fee is 10% of gross receipts. If you are planning to use The author(s) declare that they have no competing interests.NorthStar. EAA and RF were the content editors. ST was responsible for the technical development of NorthStar. ADO was the central coordinator of the project. All authors read and approved the final manuscript.All authors participated in the development and evaluation of


Little is known about how sexual and reproductive (SRH) health can be made accessible and appropriate to adolescents. This study evaluates the impact and sustainability of a competitive voucher program on the quality of SRH care for poor and underserved female adolescents and the usefulness of the simulated patient (SP) method for such evaluation.28,711 vouchers were distributed to adolescents in disadvantaged areas of Managua that gave free-of-charge access to SRH care in 4 public, 10 non-governmental and 5 private clinics. Providers received training and guidelines, treatment protocols, and financial incentives for each adolescent attended. All clinics were visited by female adolescent SPs requesting contraception. SPs were sent one week before, during (with voucher) and one month after the intervention. After each consultation they were interviewed with a standardized questionnaire. Twenty-one criteria were scored and grouped into four categories. Clinics' scores were compared using non-parametric statistical methods (paired design: before-during and before-after). Also the influence of doctors' characteristics was tested using non-parametric statistical methods.Some aspects of service quality improved during the voucher program. Before the program started 8 of the 16 SPs returned 'empty handed', although all were eligible contraceptive users. During the program 16/17 left with a contraceptive method (p = 0.01). Furthermore, more SPs were involved in the contraceptive method choice . Shared decision-making on contraceptive method as well as condom promotion had significantly increased after the program ended.Female doctors had best scores before- during and after the intervention. The improvements were more pronounced among male doctors and doctors older than 40, though these improvements did not sustain after the program ended.This study illustrates provider-related obstacles adolescents often face when requesting contraception. The care provided during the voucher program improved for some important outcomes. The improvements were more pronounced among providers with the weakest initial performance. Shared decision-making and condom promotion were improvements that sustained after the program ended. The SP method is suitable and relatively easy to apply in monitoring clinics' performance, yielding important and relevant information. Objective assessment of change through the SP method is much more complex and expensive. Nicaragua has one of the highest adolescent fertility rates of Latin America, with 119 births annually per 1000 young women aged 15–19. By the age of 19, 45% of the girls are either pregnant or have already given birth. High fertility rates are associated with low socio-economic status and low educational attainment [Major reasons for the low use of contraceptive methods among adolescents are lack of access to information and services on sexual and reproductive health (SRH) and a low quality of care. While obstacles to appropriate care have been documented extensively -5, thereEvidence of interventions that have succeeded in improving the quality of SRH care for adolescents in existing health centres in Latin America is scarce. Indications existed that the competitive voucher program for sex-workers in Managua, Nicaragua, succeeded in improving the quality of care for these women ,9, howevThe evaluation showed that voucher receipt increased use of SRH care among female adolescents  and of contraceptives and condoms in specific groups . FurtherThe objective of the simulated patient (SP) study reported here was to assess whether the technical quality, the content of the communication and the general treatment were improved by the voucher program, and whether any change was sustained after the intervention ended. The second aim of this exploratory study was to evaluate the usefulness of the SP method to evaluate such changes. SPs were sent to the participating clinics, before, during and after the intervention to request a contraceptive method. This paper reports on their experiences with the health care and our experiences with the SP method.The intervention took place in Managua, the capital of Nicaragua, one of the poorest countries of Latin America. Primary health services in Managua consist of public health centres run by the Ministry of Health, municipal public health centres, private clinics, and clinics run by non-governmental organisations (NGOs). Most participating clinics are staffed by two doctors, and in most larger clinics two doctors were allocated to receive adolescents.and follow-up visit was US$ 4.56. Adolescents were free to redeem their voucher in a clinic of their choice, and the clinics received reimbursement for each adolescent consultation with a voucher. The voucher program started with four clinics and new clinics were added periodically.Over fifteen months, 28,711 vouchers were distributed to adolescents aged between 12 and 20, at markets, outside schools and door-to-door in disadvantaged neighbourhoods. The only selection criterion was age. The vouchers gave free of charge access to SRH care in any of the four public, five private or ten NGO clinics contracted by ICAS, for one consult and a follow-up visit. The selection of clinics was based on suitability and proximity to the areas where vouchers were distributed. Identified clinics were invited to participate, and the price per consultation was negotiated based on their customary fees. The average price negotiated per consultation Vouchers were valid for three months and 20% of the vouchers distributed to girls were redeemed (boys 6%). This is a relatively high redemption rate considering the limited period over which the vouchers were valid and that a large percentage of the receivers was not yet sexually active. Among sexually active girls, 51% used their voucher, while among girls who were not yet sexually active, use was only 14% .The voucher program addressed various aspects of quality of care. Doctors had to complete a standardized clinical form based on 'best practice' protocols, which guided them during each consultation. The form was designed to ensure that every adolescent was asked about their sexual activity; their need for information; their need for contraceptives; and was given a package with two condoms plus health education material on adolescence and STIs. Contraceptives, drugs for syndromic treatment of STIs and laboratory tests for pregnant girls were made available through the program. It was assumed that the competitive nature of the voucher program would prompt providers to improve the quality of services to attract more voucher users . DoctorsIn the SP method patients are 'standardized' making it possible to compare the performances of different doctors or of one doctor at different moments. The use of the SP method in developed countries has proven to be reliable, valid, feasible and acceptable -15. The All 19 participating clinics signed a contract of agreement that included information that the quality of care would be evaluated on a regular basis. They were not informed specifically on the methods to be used. The week before the intervention started in a certain clinic, a SP without a voucher was sent. During the course of the program, SPs were sent with a voucher to monitor the quality of care provided under the program. One to two months after the program ended, SPs without a voucher visited each clinic again.The SPs were female adolescents active in NGOs involved in the voucher distribution, and university students participating in social work activities. They were contracted on a day-to-day basis and were paid a days' wage and enough money to pay for the consultation fees, if needed, and transportation costs. In total, 17 SPs took part in the survey. Their mean age was 18.7 years (range 16–22 years) and seven had secondary school education and 10 studied at the university. The mean number of doctors visited was 3.5 (range 1 – 12).All SPs received a short training on the task to be performed. The objective of the study was explained and the girls were instructed to present a story that over the past three months they had formed a relationship with a boyfriend and were presently using withdrawal or periodic abstinence, but did not want to risk pregnancy. Furthermore, they had to ask for information on HIV/AIDS. All were told that they were free to accept or reject any information or services offered to them. If necessary, they were advised to decline vaginal examination by saying that they were currently menstruating.The SPs were not accompanied to the clinic. If needed, the SP would make an appointment. After the consultation, the SPs were immediately interviewed by one of the two doctors of the research team at the ICAS office, using a standardized questionnaire. The answers were recorded by the interviewer on the questionnaire. The study was approved by the ethical review committee of ICAS.The questionnaire contained 35 open-ended questions and included commonly used elements for evaluating quality of family planning services: choice of contraceptive methods, information given to users, technical competence, interpersonal relations, mechanisms to encourage continuity and appropriate organisation of services . QuestioThe 21 criteria for evaluation are based on the program objectives, and were extensively discussed by the research team and defined before the analysis was performed. They are grouped into four categories. Two categories relate to contraception: A. method choice and B. continuity in use; category C. relates to prevention of STIs; and D. to the organisation of the services. The last category is composed of a selection of criteria that have been found to influence patients' perception of the quality of care, but do not relate directly to the doctors performance. A maximum score of 10 points could be attained if there was a positive response for all criteria in one category. The addition of the scores of the first three categories serves as an indicator of the quality of the consultation. For more details on the components of each category see Tables Information on date of interview, clinic, and doctor visited was disconnected from the file, and records were put in a random order. Codification was done by one person, and checked by another. All data were entered twice in Epi-info 6.04 by two different data processors. Stata 7.0  was used for further analysis.The response to each criterion is tabulated in table To get an indication as to whether doctors' characteristics influence their performance while counselling for contraception and STI/HIV, the sum scores of the three categories reflecting the quality of the consultation are calculated, at the three relevant time-points. Because the doctors are not necessarily the same at the different time-points, only comparisons per time-point are made. Per time-point, the non-parametric Mann Whitney test is used to assess the influence of doctors' gender and the Kruskal-Wallis test to assess the influence of the age group and the type of clinic on their responses. Since doctors had to complete a medical form for all patients with a voucher, this included the SPs. These forms were used to evaluate the SPs' performance and doctors' registration.before the intervention. In one clinic, the SP was received by the counsellor instead of the doctor, one was sent too late and treated according to the voucher protocol, and one is missing for unknown reasons. During the intervention, 17 clinics were visited by 30 SPs with a voucher. One of the clinics was excluded due to administrative problems and one resigned from further participation. After the pilot, one clinic was temporarily closed and another permanently closed. Eighteen SPs visited the remaining 15 clinics.All clinics received SPs. In total 65 visits were made to 16 clinics before, 17 during and 15 after the intervention. Seventeen SPs visited 16 of the 19 participating clinics 18%) of the doctors worked in a public facility; 19 (58%) in an NGO clinic and 8 (24%) in a private clinic. Sixty-one percent are female and 39% younger than 35 years, broadly conforming to the gender and age profile of primary health care doctors in Nicaragua.Thirty-three of the 40 doctors who participated in the voucher program received at least one SP. Their characteristics are similar to the entire group of participating doctors. Six . However, the number of barriers to use diminished. Barriers related to unnecessary diagnostic tests disappeared during the voucher program , but returned after the program.There was considerable variation in the methods recommended by the doctors. Some doctors promoted the condom for its safety or dual protection, where other doctors advised that condoms were unsafe, boyfriends would resist using them, or warned about possible allergic reactions. The same variation was seen in methods such as the intra-uterine device and monthly injectables. Before and after the program, some doctors recommended natural methods like periodic abstinence as best method for adolescents. The quality of the explanations given improved during the voucher program and remained better after the program ended .Before the intervention started, most SPs indicated that it was the doctor who decided on the type of contraceptives to be used. Patients with a voucher reported more frequently that they made the choice jointly with the doctor . The percentage of SPs reporting shared decision-making remained relatively high after the intervention (p = 0.03).Although all SPs were eligible contraceptive users, before the program eight of the 16 SPs left the consultation room without a contraceptive method or prescription. Six of these eight SPs were told to come back on the first day of their next menstrual cycle. One doctor preferred to wait for the results of unnecessary laboratory tests  and one advised periodic abstinence (the 'rhythm method').During the voucher program, 94% of the girls left the clinic with a contraceptive method . This improved practice continued after the program ended (p = 0.10), but the frequency of follow-up appointments decreased .Although the percentage of doctors discussing STI/HIV-risks during a consultation increased, a considerable number of patients did not receive the desired information. More than 10% of the SPs in each round felt the doctor was in a hurry to end the consult. Some felt pressured to the extent that they did not ask their question on STIs.Although in less than half of the consultations before, during and after the intervention SPs were satisfied with the explanations provided, patients seen after the intervention more frequently received correct information than before  and condom promotion increased from 53% to 93% .There was considerable variation in the content of the information on prevention on STIs/HIV. Some doctors gave good and reliable information where others confined themselves to observations like: "One has to know the person with whom one is with"; "The only way to prevent AIDS is not to sleep with someone who has AIDS"; "One should not use clothes from other persons"; or "If someone's partner is sweating and having weight loss he might have AIDS".No clear differences in the organisational aspects of the clinic could be established, except that privacy during consultation appeared better respected after than before and during the intervention (p = 0.08).Patients with a voucher were not always treated according to the contractual agreements. SPs reported that receptionists sometimes completed considerable parts of the medical forms and that other patients were favoured in the waiting room.Experiences of the SPs with vouchers suggest that some private clinics tried to make as much money as possible out of the program by erroneous declarations, and by selling or withholding the materials provided by the program.The gender of the doctor was strongly associated with the results. Female doctors scored better than male doctors before, during and after the intervention. Male doctors improved by a greater degree during the voucher program. However, these changes were not sustained except for the category on STI/HIV prevention (not shown). Elderly doctors had the lowest initial scores, which improved during the program, but still remained the lowest. Doctors younger than 35 improved most during the voucher program. In contrast to the analyses mentioned above, individualdoctors arehere the unit of analysis. The results are reflected in first part of table Comparing the public, NGO and private clinics, the NGO clinics performed best before the intervention. During the intervention, the improvements observed were more pronounced among the public and private clinics, that is, the clinics with weakest initial performance. The performances became of a comparable quality during the intervention, but the improvements were not sustained after the intervention ceased . Judged from the medical records of patients who went with a voucher to the clinic, the SPs performed their role correctly.SP studies designed to measure differences usually limit themselves to record doctors' actions because inter-observer variability for criteria related to doctor-patient interaction is much higher ,14,21,23We cannot exclude that the simulated patients were treated differently because of their somewhat higher level of education and higher age than the average voucher redeemer. However, accessing sexual and reproductive health care is difficult for many adolescents in Nicaragua, even for those who attended secondary school. Because the same group of girls participated in the pre- during and post assessment, the changes observed do not relate to their background.Significant changes were foremost observed in criteria directly influenced by the program, such as how a decision on type of contraceptive method was taken and the improved delivery of methods. This may indicate that these criteria were the only ones influenced by the program. Because we compared clinics and not individual providers and the sample size was modest, it is also possible that less pronounced effects, or effects with lower inter-observer and/or inter-provider agreement, may not have been detected.The voucher program had a substantial impact on the number of women that left the clinic with a contraceptive method. This is in accordance with the programs instruction to give any women who requested a contraceptive method, a one month supply of that method with clear instructions on when to start (first day of the next menstruation) and to use condoms before. Patients were asked to return during the month following their menstruation and to relate their experiences. In this return visit, still covered by the voucher, patients could then receive a three months supply.Clinics and doctors with weakest initial scores showed biggest improvements, suggesting they made serious efforts to comply with the program requirements. Unfortunately, most changes did not sustain beyond the program. Furthermore, even during the program there is room for further improvement. A longer intervention period and specific strategies to enhance quality improvements might realize more substantial and more sustainable changes. The voucher program carries this possibility. On one hand, the participating doctors were enthusiastic about the program, appreciated the experience, and were interested to improve their communication and other skills . On the Another strategy that could be combined with the voucher program is explicit use of the criteria for 'adolescent friendliness' . RecentlPreparing and sending SPs, and recording their experience is the simplest part of the SP method. Their experiences provide valuable information about the actual performance of health services. The SP method, when openly employed, can furthermore be a useful support for regular feedback and identification of further training needs. It becomes more complicated when the experiences of SPs are to be converted into indicators to assess and evaluate changes in quality of care in a scientifically correct manner. This methodological challenge is insufficiently emphasized in the literature. Although the SP method is particularly recommended for evaluating programs with a focus on the interaction between patients and doctors, difficulties and solutions related to standardized assessment of the inherently subjective patient's opinion are not reported. This is an interesting field for future research in view of the importance of patients' perception in health care use.Assessing quality improvements -including through the SP method- is complicated and requires highly trained human resources, time and financial resources. Although many health care interventions in developing countries aim to achieve quality improvements, programs are seldom supplied with sufficient funds to make such valuable and important evaluations possible.The adolescent voucher program seemed to have succeeded in increasing the accessibility of contraceptives through health services, and had the greatest impact on the quality of care in clinics with lowest initial quality levels. This is an important achievement, because if unwanted and untimely pregnancies are to be prevented, adolescents who engage in sexual relations need access to information and contraceptive methods.Although in Nicaragua wide agreement exists that health care services should respond to these needs -as is explicitly outlined in norms and guidelines of the Ministry of Health  – implemThe SP method proved to be a valuable and feasible method for program monitoring in a developing country. The results of this pilot study suggest that scaling up of the voucher program could be an effective strategy to make SRH care accessible, acceptable and appropriate for underserved teenagers.AIDS Acquired Immune Deficiency Syndrome.DFID The British Department for International DevelopmentFP Family PlanningHIV Human Immune Deficiency VirusICAS instituto CentroAmerica de Salud, Central American health instituteNGO non-governmental organisationsRTI Reproductive tract infectionSP Simulated patientSRH Sexual and reproductive healthSTI Sexually transmitted infectionsThe author(s) declare that they have no competing interests.LE M was involved in all stages of the process from conception and design of the survey, the analysis and interpretation of data. AC G was involved in the conception and design, the analysis and interpretation of the data, participated in writing and revising the paper critically for substantial intellectual content until the final approval. ADM K was involved in the statistical analysis and interpretation of the data, participated in writing and revising the paper critically for substantial intellectual. JA K gave support in the development of the research, the analysis and interpretation of the data, and contributed substantially in preparing the manuscript, revising it critically for substantial intellectual content. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here: