Fetal Alcohol Spectrum Disorder (FASD) is underdiagnosed in Canada. The diagnosis of FASD is not simple and currently, the recommendation is that a comprehensive, multidisciplinary assessment of the individual be done. The purpose of this study was to estimate the annual cost of FASD diagnosis on Canadian society.Canadian Guidelines for Diagnosis. The per person cost of diagnosis was calculated based on the number of hours (estimated based on expert opinion) required by each specialist involved in the diagnostic process. The average rate per hour for each respective specialist was estimated based on hourly costs across Canada. Based on the existing clinical capacity of all FASD multidisciplinary clinics in Canada, obtained from the 2005 and 2011 surveys conducted by the Canada Northwest FASD Research Network, the number of FASD cases diagnosed per year in Canada was estimated. The per person cost of FASD diagnosis was then applied to the number of cases diagnosed per year in Canada in order to calculated the overall annual cost.The diagnostic process breakdown was based on recommendations from the Fetal Alcohol Spectrum Disorder Using the most conservative approach, it was estimated that an FASD evaluation requires 32 to 47 hours for one individual to be screened, referred, admitted, and diagnosed with an FASD diagnosis, which results in a total cost of $3,110 to $4,570 per person. The total cost of FASD diagnostic services in Canada ranges from $3.6 to $5.2 million (lower estimate), up to $5.0 to $7.3 million (upper estimate) per year.As a result of using the most conservative approach, the cost of FASD diagnostic services presented in the current study is most likely underestimated. The reasons for this likelihood and the limitations of the study are discussed. Prenatal alcohol exposure (PAE) can lead to a myriad of adverse developmental outcomes, and is recognized to be the most common preventable cause of mental deficiency in the western world. Despite attempts to increase public awareness of the risks associated with drinking during pregnancy, a significant proportion of pregnancies among the general population (about 14%) Fetal Alcohol Spectrum Disorder (FASD) is a non-diagnostic term that encompasses four alcohol-related clinical diagnoses, including: Fetal Alcohol Syndrome (FAS), Partial Fetal Alcohol Syndrome (pFAS), Alcohol-Related Neurodevelopmental Disorder (ARND), and Alcohol-Related Birth Defects (ARBD). The effects of PAE may include growth impairments (height and weight), birth defects, neuropsychiatric disorders, and mental disorders , which are likely to have lifelong implications.Given the wide range of deficits and disabilities associated with FASD, it is absolutely necessary to screen for individuals who may be affected by PAE. This is especially important for ARND since these disorders are not apparent from physical features Importantly, screening of children for PAE and early diagnosis can also facilitate the prevention of subsequent FASD-affected births by providing appropriate interventions, treatment, counseling, and support for birth mothers with unrecognized alcohol dependence and mental health problems In 2005, the Public Health Agency of Canada (PHAC) endorsed the Canadian Guidelines for the diagnosis of FASD Screening is a process by which members of a defined population are asked a question or offered a test to determine which of those individuals are more likely to be helped by further tests/assessments. In the case of FASD, ideally, a positively screened individual will undergo a diagnostic evaluation for FASD. It is important to understand that not all positively screened individuals will receive an FASD diagnosis. The purpose of FASD screening is to facilitate referral to a diagnostic clinic, and to call attention to the need for referral and support for the birth mother Despite having the most commonly cited rough prevalence estimates of 1 per 1,000 for FAS It is also important to understand that the screening tools utilized in a population of children and youth may not be applicable to a population of adults and thus, tools should be age appropriate. Some tools have been recommended specifically for use in adults Screening is only effective if the right individuals are being screened. In regard to FASD, there are populations of special concern that should receive special attention in regard to the administration of screening for FASD, including (in no particular order):Women utilizing maternal substance abuse treatment programsChildren seen in neurodevelopmental and/or birth defect clinicsChildren in special education programsChildren in childcare systems Prison populationsYouths in juvenile justice programsClients of the mental health care systemThe role of the family physician or primary care health provider in screening for FASD is to recognize patients with dysmorphic features and/or developmental/learning and behavioural problems that may be due to PAE, confirm the alcohol history, and then refer the patient to a multidisciplinary diagnostic team with expertise in FASD diagnosis The diagnosis of FASD is not simple owing to the fact that determining if a patient has diffuse brain damage as an explanation for behavioural problems and that this damage is more likely due to PAE and not to other causes requires a comprehensive, multidisciplinary assessment involving physicians , psychologists or neuropsychologists, joined by other developmental specialists , as well as a coordinator for case management The criteria for FASD diagnoses have been described thoroughly in the current literature FASD diagnosis involves the following components: the physical examination, the dysmorphology assessment, the neurobehavioural assessment, and PAE confirmation. The general physical examination involves appropriate measurements of growth , assessment of characteristic findings and documentation of anomalies . While, the dysmorphology assessment is intended to identify specific facial features related to PAE . The neurobehavioural assessment involves an evaluation of multiple domains including: hard and soft neurological signs, brain structure, cognition (IQ), communication (receptive and expressive), academic achievement, memory, executive functioning and abstract reasoning, and attention deficit/hyperactivity. The specific diagnostic tests/evaluations will differ depending on the age of the individual  Confirmation of PAE  is also required for FASD diagnosis  and can be obtained through a direct interview with the mother , or other sources such as reliable clinical observation, reports by a reliable source, or medical records Based on the information presented above, FASD diagnosis can easily be seen as a lengthy process that requires a number of trained specialists. Therefore, it is understandable that FASD diagnosis, like other complex brain based developmental disorders, is likely a very costly practice.The purpose of this study was to estimate the per person cost of FASD diagnosis, as well as the annual cost of FASD diagnostic services in Canada.Not applicable.The diagnostic process breakdown was based on the Fetal Alcohol Spectrum Disorder: Canadian Guidelines for Diagnosis The number of hours required by each specialist involved in the diagnostic process was estimated, based on experts’ opinion . The average rate per hour for each respective specialist was estimated based on hourly costs across Canada.In order to estimate the annual cost of all FASD cases diagnosed for Canada as a whole, it is necessary to know how many cases are referred, evaluated and diagnosed per year in Canada. Unfortunately, these figures are not available. Therefore, the number of FASD cases diagnosed per year in Canada was estimated based on the data obtained from the 2011 survey conducted by the Canada Northwest FASD Research Network to determine the existing clinical capacity (the maximum number of assessment slots available for diagnostic purposes per year) of all FASD multidisciplinary clinics in Canada in 2011 Fifty percent was then used as the lower boundary, in order to calculate the minimum number of people diagnosed with FASD in Canada per year .The cost of FASD diagnosis per person was applied to the number of cases diagnosed per year in Canada. All cost figures are presented in Canadian dollars.Screening and Referral was estimated to cost from $150 to $300 per person screened (1–2 hours per person), Intake into the diagnostic clinic, the point at which information is gathered, was estimated to cost from $160 to $320 (2–4 hours per person), Diagnosis was estimated to cost between $2,650 to $3,750 per person (23 to 33 hours per person), and General Support was estimated to cost between $150 to $200 per individual (6 to 8 hours per person). Thus, the total cost for one individual to be screened, referred, admitted, and diagnosed ranges from $3,110 to $4,570 .The estimated number of hours, average cost per hour, and total cost for each component of the multidisciplinary diagnostic process are presented in A recent survey by Clarren and colleagues , one can calculate that there are at most approximately 1,602 cases of FASD diagnosed per year in Canada.Therefore, it was estimated that based on a capacity of 2,288  in all FASD clinics in Canada per year Using an assumption that only 50% of all individuals referred and evaluated in the multidisciplinary clinics are diagnosed with FASD, one can calculate that 1,144 cases  of FASD are diagnosed per year in Canada. This estimate - 1,144  to 1,602  cases of FASD are diagnosed per year in Canada - was used to estimate the annual cost of FASD diagnosis in Canada. Therefore, it can be estimated that in Canada the total cost of diagnosing FASD ranges from $3.6 to $5.2 million (lower estimate), up to $5.0 to $7.3 million (upper estimate) per year .It could be argued that a large portion of the cost of FASD diagnostic services may be attributed to the need for the multidisciplinary approach. This approach could be considered to be too expensive and cumbersome for a high frequency disorder like FASD, where about one percent of all live births are affected and in some communities may even be much higher (up to 20%) Given the fact that FASD is not widely recognized by health care practitioners There are several other reasons to believe that the estimated cost of FASD diagnostic services in this study is most likely underestimated. To begin, the current cost estimate of FASD diagnosis per person includes the cost of the core team member only. However, additional team members may include addiction counselors, childcare workers, mental health workers, probation officers, psychiatrists, teachers, vocational counselors, nurses, neuropsychologists, and family therapists Finally, the estimated cost of diagnosis does not include the cost of completing an intervention plan and/or follow-up. It is important to point out that it would be unethical to identify an individual with an FASD and not provide them with the necessary referrals, education, resources, and services specific to their needs. Intervention plans should not only consist of identifying the deficits and impairments of the individuals with FASD, but also identifying the strengths of that individual. Therefore, an individualized intervention plan and follow-up must follow any diagnosis.In addition, Canada is a multicultural country and thus, the entire diagnostic process must be culturally appropriate, sensitive, and use appropriate language. For this reason, the use of cultural interpreters may be required, and additional costs would likely be incurred in such cases.Further, both the cost of negatively screened individuals and the cost of screening pregnant women are not included in the current total annual estimate. Such costs would not be incurred if maternal alcohol consumption while pregnant had a prevalence of zero. The inclusion of such costs would increase the current estimate.The estimated cost of FASD diagnosis per person is comparable with the estimates obtained from the above-mentioned survey where the programs were asked to provide the average cost of an assessment There are several limitations of the current study. First, the estimated cost of FASD diagnosis is based on hourly wages. However, it should be noted that some members of the multidisciplinary team work on a salary basis, while others work on a sessional fee basis. This could affect the cost estimate in either direction depending on the level of compensation for salary and sessional fees relative to the hourly wages utilized here.Second, not all individuals screened get referred for a full diagnostic assessment, not every person referred will show up for their evaluation, and not all individuals referred to a diagnostic assessment will receive an FASD diagnosis and therefore, not all individuals will proceed through the listed services in a linear fashion. As a result, the total cost of FASD diagnosis in Canada might be higher or lower.Third, it was not possible to estimate the costs for the rest of the provinces and territories  separately, since data on the clinical capacity of these provinces/territories were not available.It is necessary to point out that the diagnostic schema could differ slightly from clinic to clinic, and therefore, variations are likely to exist in regard to the overall diagnostic process, for example, in the team members involved. This is likely true in remote areas, and small communities. Such variations will alter the overall cost per person. It is also important to highlight that the cost of diagnosing an individual with FASD in the general population, as presented here, will differ from the cost of diagnosing an individual with FASD in certain special populations, such as the prisoner population. For example, when diagnosing an individual in the criminal justice system additional costs are likely to be incurred, including the cost of transportation (transporting the prisoner or transporting the FASD-trained team members to the facility) and the cost of security/prison guards.It should be highlighted that early detection of disease is not beneficial to the patient or his/her family if existing treatment is not offered or is not available


Dear Editor,We read with interest the paper of Estévez, et al, recently published in Hepatitis Monthly . That arIn spite of doubts in the statistical analyses used in their study, we should emphasize that the study of Nikui Nejad, et al, was clearly a clinical trial involving comparison between two vaccines. In that study, the sample size did not depend on the prevalence of vaccine response rate in the population-instead, it did depend on the difference between the response rates from the two vaccines. The authors also used an incorrect assumption for the calculation of minimum sample size according the "prevalence approach" and assumed a large d with a small P! We know that in clinical trial, if the sample size is too small, a well-conducted study may fail to answer its research hypothesis or to detect important effects and associations. Therefore, correct assumptions for the calculation of minimum sample size are of paramount importance in conducting research. Approaches for estimating sample size and performing power analysis depend primarily on the study design and the main outcome measure . Clinica


The Drug Effectiveness Review Project was initiated in 2003 in response to dramatic increases in the cost of pharmaceuticals, which lessened the purchasing power of state Medicaid budgets. A collaborative group of state Medicaid agencies and other organizations formed to commission high-quality comparative effectiveness reviews to inform evidence-based decisions about drugs that would be available to Medicaid recipients. The Project is coordinated by the Center for Evidence-based Policy (CEbP) at Oregon Health & Science University (OHSU), and the systematic reviews are undertaken by the Evidence-based Practice Centers (EPCs) at OHSU and at the University of North Carolina. The reviews adhere to high standards for comparative effectiveness reviews. Because the investigators have direct, regular communication with policy-makers, the reports have direct impact on policy and decision-making, unlike many systematic reviews. The Project was an innovator of methods to involve stakeholders and continues to develop its methods in conducting reviews that are highly relevant to policy-makers. The methods used for selecting topics, developing key questions, searching, determining eligibility of studies, assessing study quality, conducting qualitative and quantitative syntheses, rating the strength of evidence, and summarizing findings are described. In addition, our on-going interactions with the policy-makers that use the reports are described. Created from efforts to improve formulary policy for the Oregon Health Plan (OHP) – Oregon’s innovative Medicaid program – the Drug Effectiveness Review Project (DERP) now has almost a decade of experience conducting comparative effectiveness research to determine what works best in the effort to inform health policy decisions with evidence. Recent health care legislation and, in particular, the creation of the Patient-Centered Outcomes Research Institute (PCORI) make this experience increasingly relevant to the national effort to make better and more cost-effective health policy.By 2001, a rapid increase in the cost of medications[At the outset, the Oregon Evidence-based Practice Center (EPC) was asked to conduct systematic reviews of several classes of drugs; each class contained multiple expensive drugs and thus a potential for increased benefit, reduced harm, reduced cost, or a combination of these. EPC investigators met with the relevant State appointed topic-specific committees (comprising of clinicians and community representatives) initially to develop the key questions and later to provide an overview of the findings and to answer questions. These meetings were held publicly and public testimony was allowed; typically a few physicians, patients (or caregivers), and representatives of pharmaceutical companies attended. The committees made recommendations to the Health Resources Commission, who in turn, made final recommendations to the state Medicaid agency.Oregon reaped early success with this process: In 2001 a systematic review showed that among the five proton pump inhibitors available, there were no clear differences in benefits or harms between medications[After leaving office in 2003, Kitzhaber formed the Center for Evidence-based Policy (CEbP) at Oregon Health and Science University (OHSU). The first CEbP project was an expansion of the alliance of Medicaid agencies. In all, 15 organizations – 14 state Medicaid agencies and the Canadian Office for Health Technology Assessment  – formed the collaboration that became the DERP  in any pharmaceutical company for at least the duration of the time the person is doing work for DERP. We have found that this policy, which is much more restrictive than those at other organizations where modest interests are allowed with disclosure, both simplifies interactions with teams and reassures the stakeholders. At the same time, engagement with industry is a critical part of the DERP process as described below.Organizations participating in DERP follow an explicit process to ensure equal opportunity for input in the selection of topics for systematic review. The CEbP solicits topics from each organization, typically on an annual basis. After discussing each nomination, the organizations select up to five topics for further workup. Limiting to five topics reflects the workload required to prepare topic briefs, the limited resources available for conducting research, and the improved level of discussion with fewer choices. Consisting of the original submission details, a summary of pros and cons, and an overview of available systematic reviews and randomized trials, this workup is used to inform organizations as to whether sufficient evidence is available to support a full review. Over the years, the content of these briefing papers has evolved. For example, previous versions asked participating organizations to estimate the level of use and potential economic impact of the proposed drugs in their respective programs. Because this was not always possible to do, and created additional workload, this aspect was changed to a more qualitative question about pros and cons. To identify high-quality systematic reviews relevant to the proposed topic, the EPC investigators search MEDLINE and the Web sites for Agency for Healthcare Research and Quality (AHRQ), the Canadian Agency for Drugs and Technologies in Health, the Cochrane Collaboration, and the National Coordinating Center for Health Technology Assessment, the National Institute for Clinical Excellence, and the Center for Reviews and Dissemination. Identification of an existing recent high-quality review may indicate that it is not necessary to undertake a new review and the number of trials is an indicator of the volume of the evidence on a topic. At a face-to-face meeting, participating organizations discuss the briefing papers and select topics for which DERP reports will be commissioned.Key questions define the scope of a DERP report. Preliminary key questions are formulated by EPC investigators on the basis of the discussion that led to the topic selection. Participating organizations review the draft key questions and clinical experts, described below, are consulted. After modifications, the draft key questions are posted to the DERP Web site for public comment. Public comments and responses proposed by investigators are discussed with the participating organizations. Our records indicate that we receive a mean of 2.5 sets of comments per posting.After further modifications, approved by the participating organizations, the final key questions and related eligibility criteria, which will define the scope of the upcoming report, are posted to the DERP Web site. The eligibility criteria specify the Populations, Interventions, Comparisons, Outcomes, Timing, Settings and Study designs or characteristics (PICOTS) of interest. Each of the PICOTS is selected to reflect participant needs. For example, the list of interventions is selected to reflect the drugs under consideration by participating Medicaid agencies and similar drugs available in Canada. In DERP, the primary outcomes of interest are health outcomes reflecting effectiveness rather than short-term efficacy outcomes . However, there are examples where intermediate outcomes are valuable to policy-making and these are included on a case-by-case basis . Eligible harms outcomes typically include overall rates of adverse events (AEs), withdrawal due to AEs, and AEs specific to the drug class, including serious AEs. An example of a set of final key questions and eligibility criteria are shown in FigureThe Clinical Advisory Group provides clinically relevant counsel throughout the development of a report. We began using such an advisory group in 2007, and they are convened for all original reports on new topics. For updates of existing DERP reports, advisory groups are formed on a case-by-case basis, depending largely on whether a substantial change in scope has occurred between the previous report and the pending update. Prior to 2007, we consulted with clinicians on an ad hoc basis.The CEbP identifies potential clinical advisors among clinicians suggested by the participating organizations, which recommend clinical experts who best represent their constituencies and who also have significant recent experience in direct patient care. The participating organizations review the qualifications and declared conflicts of interest of the group prior to selecting the group’s final composition. While there are no rules on exclusion based on conflicts of interest, if conflicts are declared members are selected with these in mind, in an effort to compile a group with balanced interests. Members of the Clinical Advisory Group are not compensated for their time, typically two to eight hours each, excluding time spent reviewing the draft report if they serve as a peer reviewer. Their names are listed both in the report and on the DERP Web site.Searches of electronic databases are conducted in consultation with an information specialist. All DERP reports entail, at minimum, a search of Ovid Medline – or PubMed – and the Cochrane Library. Other databases  are searched depending on availability to the EPC conducting the review and on the topic of the report (e.g. PsychINFO is searched for topics that include psychiatric indications). Search strategies generally combine all included interventions (using proprietary and generic names) and populations. Literature searches are repeated 2 to 3 months before submission of the draft report. Additionally, references of key papers are searched by hand.The CEbP requests information from all pharmaceutical companies that manufacture a drug included in a DERP report. We request a complete list of citations for all relevant studies of which the manufacturer is aware. We also request information on unpublished studies or unpublished data relating to a published study, with the understanding that once the report is published the public may obtain the information by requesting a copy of the information submitted. An accounting of companies providing information is included in the report. We have received over 100 submissions since DERP began.The FDA Center for Drug Evaluation and Research “Drugs@FDA” Web site is searched for statistical and medical reviews written by FDA personnel about drugs approved for specific indications. These reviews contain information about trials submitted by pharmaceutical companies as part of New Drug Applications, including study design, results, and analysis of results. We compare the studies submitted to the FDA with those found in the published literature to identify additional unpublished studies or data, including identifying variation in outcomes reporting. FDA documents may not be available for every drug included in DERP reports: posting of a new drug’s documents to the Web site has sometimes been delayed despite approval of the drug and the Web site does not generally contain documents related to drugs approved before 1997.a priori. Studies with results presented only in a language other than English are excluded because of resource limitations. Studies reported only in conference abstracts are excluded because experience has shown us that typically there is inadequate information provided to evaluate study quality, including selective reporting of results, and to match up multiple abstracts related to a single unique study.For a study to be selected for inclusion, it must meet all eligibility criteria, as described above, including study designs, as explicitly specified Randomized controlled trials directly comparing two or more drugs in the review are included to evaluate both benefit and harms outcomes.all DERP reviews include observational studies to assess harms, to date seven also include them to assess benefits.Our methods for decisions on including observational studies concur with the recently published guidance for the AHRQ Effective Healthcare Program,4. Head-directly address the key questions and meet minimum standards for quality are considered for inclusion. Reviews must meet the following minimum criteria to be considered systematic: The review must include a comprehensive search for evidence from multiple sources of information , describe the terms used in the search (in some way), and use dual review of studies for inclusion. The review must address questions that are similar enough to the key question to provide useful information; reviews that examine a class effect or only a portion of the drugs in a class are unlikely to be useful in a DERP report. For example, while we identified many prior systematic reviews of atypical antipsychotic drugs, none made comparisons of all drugs in the group to each other[Existing systematic reviews that ch other. Most coWhere there are gaps in direct comparative evidence, we consider including placebo-controlled trials for indirect comparisons of benefits and harms. The decision to pursue a quantitative indirect comparison takes into account the degree of clinical, methodological, and statistical heterogeneity across the studies under consideration. We follow established guidance on methods for quantitative indirect synthesis,9-14.When indirect comparisons are not undertaken, findings from placebo-controlled trials can be discussed to identify individual drugs with evidence of benefit (and to identify the magnitude of effect) and those with no such evidence.“Pooled analyses” is our term for published meta-analyses that are not based on a comprehensive literature search and do not include assessment of the quality of individual studies. Pooled analyses differ from systematic reviews that utilize meta-analysis as part of the synthesis of evidence (e.g. DERP reports) and potentially introduce bias. Therefore, as with placebo-controlled trials, pooled analyses are considered for inclusion in DERP reports only where other evidence is absent or insufficient. An example is where the only information about a subgroup of patients available is from a pooled analysis of data from manufacturer-sponsored trials.Studies without a well-formed comparison group have a high risk of bias, particularly in assessing benefit. Single-Open-label extension studies have limited applicability and higher risk of bias than do the original trials. These study populations are derived from clinical trials, where patients are already winnowed to meet a narrow set of eligibility criteria, and patients in the extension often are those who had an adequate drug response and tolerance during the trial period. An extension study without a comparison group is especially subject to bias, becoming essentially an observational study in a highly selected population. As such, we include them only under the conditions stated above, but view them as reflecting a very narrowly defined population.Unpublished studies  may be included if they meet eligibility criteria and provide sufficient detail to assess study quality. At minimum, information must be provided on the comparability of groups at baseline, attrition, number of patients analyzed, the statistical tests used for data analysis, and whether an intention-to-treat analysis was conducted. When a manufacturer provides additional unpublished outcomes or subgroup data from a published study, these data will be included if 1) the study is a direct, head-to-head comparison of included drugs, and 2) the study reports the statistical tests used for data analysis, attrition, numbers of patients analyzed in each group, and whether an intention-to-treat analysis was conducted. Prior to 2010, inclusion criteria were determined on a case-by-case basis by each review team, largely based on the ability to conduct a quality assessment. Experience such as the limited usefulness of including various post-hoc subgroup data submitted by industry for our fixed dose combination products review led us to tighten and formalize these criteria in 2010.In order to reduce potential reviewer bias and ensure accuracy and reproducibility, all study reports identified in literature searches are assessed for eligibility by two investigators. Before full-text studies are assessed, titles and abstracts identified by searches are evaluated. For this evaluation one investigator may complete eligibility assessments with a second reviewer evaluating only those studies that the first investigator chose to exclude. With this “carry-forward” strategy, only studies that are unequivocally ineligible are rejected at this stage. Full-text articles of potentially relevant citations are retrieved and dually assessed for inclusion by two reviewers. Disagreements are resolved through consensus. In accordance with recommendations on reporting, a diagram indicating the flow of inclusion and exclusion of studies is presented. ExcludeData routinely abstracted from studies are: study design, population characteristics, eligibility criteria, interventions, numbers randomized/treated and analyzed, and results. Abstraction is performed by one reviewer and independently checked by another; differences are resolved by consensus. We record intention-to-treat results when reported. If not reported, but loss to follow-up was very small (< 5%), we note that they were modified intention-to-treat results.Historically, to assess quality  of trials, cohort studies, and case-control studies, we have applied predefined criteria that are based on those used by the US Preventive Services Task Force and the (UK) National Health Service Centre for Reviews and Dissemination,18. StudThe items assessed for trials, observational studies, and systematic reviews are shown in TableThe EPC program has recently updated their recommendations on assessing the internal validity of studies. While tStudies are stratified by key question and study quality. The best evidence informs the synthesis that will address each key question. Studies that evaluate one drug against another provide direct evidence of comparative benefits and harms and are preferred over indirect comparisons. Similarly, studies of effectiveness and long-term or serious harms outcomes are preferred over studies of efficacy and short-term tolerability. Data from indirect comparisons are used to support direct comparisons and as the primary comparison when no direct comparisons exist. Based on a number of statistical assumptions, indirect comparisons are interpreted with caution; among their defects is a greatly reduced statistical power to detect differences among groups,13,20,21Evidence tables report study characteristics, quality ratings, and findings of included studies. Key findings of the review are encapsulated in “summary bullets” (followed by a detailed discussion of the evidence) and in a summary table at the end of the report.2 are calculated to assess statistical heterogeneity among studies and our interpretation of the importance of observed heterogeneity depends on the magnitude and direction of effects and on the strength of evidence for heterogeneity[a priori. Meta-regression models may be used to formally test for differences among subgroups with respect to outcomes[For meta-analysis we follow the AHRQ EPC Methods Guide recommendations. In brieogeneity-26. If soutcomes,28. We pStrength of evidence is assessed for the main outcomes of each key question, generally by following the approach suggested by the EPC Methods Guide. Using tThe applicability of the evidence related to each key question is described in terms of populations and interventions to which the evidence applies. Exceptions – groups or interventions to which the evidence does not apply – are specified. Further consideration of how the evidence applies to each organization’s local population is left to the local decision-makers, with EPC investigators available to respond to questions.DERP reports are subject to peer review and public comment prior to finalization. The participating organizations also review the reports and provide comments to the investigators. These processes help to optimize completeness and minimize bias in the final report. Peer reviewers are identified by professional societies, acknowledged expertise in a particular field, prominent authorship in the published literature, or participation as a Clinical Advisory Group member. For two weeks, draft reports are posted to the public DERP Web site for public comment. Manufacturers and groups of individuals with expressed interest in the topic are notified before the posting and a mean of 2.7 sets (range 1–11) were received per draft. The investigators are responsible for documenting a resolution for each comment. The participating organizations are the final arbiters of whether comments have been adequately addressed. Typical comments received are suggestions to add additional studies  and corrections to recorded details of individual studies .The participating organizations consider report updates annually. This decision is based on an EPC-produced scan that identifies new trials, drugs, approved indications, or serious harms relevant to the key questions of a report. The report update process is as for an original report. FDA approval of relevant new drugs is the most common impetus for updating. In the Once a report is produced, participants use them in making evidence-based decisions on drugs in their respective programs and investigators are available upon request to present an overview of findings and to answer questions. The investigators have found these interactions, monthly phone conferences, and biannual meetings to be of great value in identifying and understanding the needs of policy-makers that use DERP reports. Lessons learned through these interactions and from surveys about the organizations’ opinions on various aspects of the project, including the research reports, directly impact future reports. Changes made to the methodology of reports based on these interactions include 1) broadening the scope to include multiple drug classes; 2) the creation of Single Drug Addenda as noted above; 3) creation of “black box warnings” appendices that do not qualify as evidence in the review but are needed by the participants in making decisions; and 4) formatting of summary material (e.g. executive summaries and slides) to fit the needs of the state Medicaid administrators, among others.DERP’s current evidence review methods were developed primarily to meet the needs of State Medicaid programs for informing evidence-based health policy decisions using public processes. Development of these methods began prior to the existence of the current Methods Guide for EPCs. While many methods are now congruent, there are nuances where DERP methodology is explicit rather than allowing selection from multiple options, largely due to having a specific audience. After 9 years of experience with DERP, we view the advantages of these methods to be:1. Incorporating input from the participants makes the reports more useful to them for making policy decisions,2. Encouraging transparency through outside evaluation of work quality and independent assessment for any risk for bias,3. Inviting public comment at the initial stages and again near the final stages of the review allows for consideration of viewpoints beyond those of academic peer reviewers and the participants of DERP,4. Using a regular process to scan for new, important developments in a given topic area allows for the best use of participant funds, and5. Providing specialized summary documents, for example slide sets and executive summaries with new information highlighted to increase the usefulness of the reports for the participants.While other groups may need to modify the approach depending on the ultimate users of their work, we recommend that organizations producing comparative effectiveness reviews to inform health policy decisions draw on the insights of policy-makers to develop and refine evidence review methods. Our methods may not be directly transferrable to topics other than comparative effectiveness reviews of pharmaceuticals, such as nonpharmacological interventions or assessment of prognostic and diagnostic tools, which may require special methodologic features.DERP has developed a timely and efficient method of creating high-quality systematic reviews designed to meet the needs of policy-makers. DERP continues to be an innovator in methods that involve stakeholders. The recent Institute of Medicine report on conduThe authors declare that they have no competing interests.All authors participated in the development of the methods described, with participation over 5 to 9 years. MM led methods discussions and drafted the initial manuscript. DJ, GG, KP, SC, and MH were significant contributors to systematic review methods development over 9 years, and provided comments on the draft manuscript. AL and MG represented the Center for Evidence-based Policy in discussions of methods and contributed significantly to the development of methods for industry relations. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/12/140/prepub


The ability to work effectively on a team is highly valued by employers, and collaboration among students can lead to intrinsic motivation, increased persistence, and greater transferability of skills. Moreover, innovation often arises from multidisciplinary teamwork. The influence of personality and ability on undergraduate teamwork and performance is not comprehensively understood. An investigation was undertaken to explore correlations between team outcomes, personality measures and ability in an undergraduate population. Team outcomes included various self-, peer- and instructor ratings of skills, performance, and experience. Personality measures and ability involved the Five-Factor Model personality traits and GPA. Personality, GPA, and teamwork survey data, as well as instructor evaluations were collected from upper division team project courses in engineering, business, political science, and industrial design at a large public university. Characteristics of a multidisciplinary student team project were briefly examined. Personality, in terms of extraversion scores, was positively correlated with instructors’ assessment of team performance in terms of oral and written presentation scores, which is consistent with prior research. Other correlations to instructor-, students’ self- and peer-ratings were revealed and merit further study. The findings in this study can be used to understand important influences on successful teamwork, teamwork instruction and intervention and to understand the design of effective curricula in this area moving forward.The online version of this article (doi:10.1186/2193-1801-2-16) contains supplementary material, which is available to authorized users. Fostering effective teamwork in the curriculum is a necessity. The ability to work effectively on a team is highly valued by employers, in addition to communication and problem-solving skills  . The FFTeam composition is commonly studied as an important group-level factor. Team composition can vary in gender, race, education, and functional background, in addition to measures of ability and personality.Prior work has found correlations between individual personality and performance on a team. Neumann and Wright found that agreeableness and emotional stability predicted peer evaluations beyond skills and ability in a study of human resources teams . In allThe personality composition of a team has been widely studied in a variety of disciplines and settings as predictors of team outcomes. A comprehensive compilation of group FFM clusters on engineering design team performance in the literature is presented in ). SchilpSome studies  describAbility within a group cannot be ignored when analyzing team performance. Steiner’s task typologies are commonly used to link measures of ability to team performance. The four typologies and hypothesized links to group ability are as follows: (1) additive tasks, where team performance is the sum of individual performance of team members, hypothesized to correlate to measures of mean ability of the group; (2) compensatory tasks, where team performance is proportional to the average of the individual contributions, also hypothesized to correlate to average group ability; (3) conjunctive tasks, where all team members must perform at some minimal level for successful team performance, hypothesized to correlate to the minimum ability member of the team; and (4) disjunctive tasks, where team performance is judged by best performance from any of the team members, hypothesized to correlate to the highest ability member of the team .There are many studies in the literature testing these hypotheses. Day et al.  argued The overarching objective of the current study is to examine the influences of personality and ability on teamwork and team performance in the context of our multidisciplinary student population and culminating capstone projects in engineering and other disciplines. Personality and ability seem to be very important contributors to group dynamics, attitudes, and consequently performance, and worthy of targeted study in the current population. Increased understanding of these influences can be applied to designing and implementing more effective assignments and instruction in teamwork and team skills. Furthermore, this work, along with prior related literature, is another step towards understanding the pedagogy behind effective, wide-scale, multidisciplinary team-based instruction at all types of institutions, which is one of the eventual goals of this line of inquiry.Question 1: How is group teamwork influenced by personality and/or ability?Question 2: How is individual performance on a team influenced by personality and/or ability?A distinction is often made between teams and groups in the literature. In this paper, we use both terms interchangeably to refer to tasks performed collectively by more than one person. The specific research questions addressed in this paper are the following: Data was collected from five courses at a large public university. The courses were: mechanical engineering senior project, electrical engineering senior project, industrial design senior project, green entrepreneurship, and public policy. There were between 16 and 35 students in each course for a total of 121 undergraduate students, spanning four colleges at the university. Each course assigned a substantial group project comprising a large percentage of the overall course grade. The green entrepreneurship and public policy courses were one semester courses. The three senior project courses were two semester sequences; only data from the first semester is reported from these classes to maintain consistency with the other two courses. In all five courses examined, the scope of the group project assigned included research, analysis, and a proposal of a solution or action plan addressing a contemporary issue. These five particular disciplinary areas were chosen in this particular study because of their relevance to sustainable energy, a common theme in large culminating-type projects at our institution; we did happen to have one multidisciplinary project during the time period of this study involving subteams from all five participating courses. Nevertheless, this methodology can be applied to any collection of disciplines that might weigh in on other project themes. This study seeks to discover and understand trends that exist across disciplinary boundaries. It does not, therefore, investigate differences between the teams that were involved in the multidisciplinary project and those that were not.The data collected for this study includes responses to online student surveys, and artifacts of student ability and achievement. At the start of the semester, a ten-item personality test developed by Gosling  was admDependent variables describing team outputs included self-, peer-, and instructor-assessment of team skills, team performance and individual contributions. Team skills were quantified by students’ responses to survey questions. Team performance was quantified by students’ responses to survey questions and instructors’ scores on class deliverables. Individual contributions to the team were quantified using responses to the teamwork survey, both by the students themselves, as well as their teammates.Independent variables affecting team outputs are often grouped into three categories: (1) individual-level factors, such as team member attributes, (2) group-level factors, such as team composition, and (3) environmental-level factors, such as task characteristics . In thiCharacteristics of the students in the participating courses were reported in a previous study  and areThe average GPAs of the courses participating in the study are listed in Table The average scores for each of the FFM personality attributes are shown in Table Student teams were identified in the five participating courses, and their average team personality attributes and GPA were computed. The teams were largely self-selected, except in a few instances where the interest in a particular project exceeded demand, in which cases there was some selection required by the instructor. In Figures The average GPA in each group is shown in Figure The research questions posed in this study are systematically examined in the context of our student population in this section.For this analysis, the students’ rating of their groups’ sharing of responsibilities, resolution of conflict, and overall productivity were examined, along with the instructors’ assessment of group course deliverables. For the student rating, the students’ responses to the questions in Part I of the teamwork survey shown in Additional file The students’ responses to the questions in Part I of the team work survey are graphed in Figures The scores  received by each group by their instructor for their final written report and oral presentation, if applicable, are graphed in Figure As shown by Figures For the first question, “To what degree did all members of the group share in the team’s responsibilities?”, a student’s response was found to be negatively correlated to his or her emotional stability score on the personality survey . In other words, students who were more calm and less easily upset were less likely to feel that the group shared in the team’s responsibilities. This result was also reported in Rhee et al. , and isFor the second survey question, “Which of the following best describes the level of conflict at group meetings?”, a student’s response correlated positively with his or her GPA . In other words, a student with a higher GPA tended to report a higher level of conflict in his or her group in our population. A higher GPA student might be more perceptive and observant of group conflict than a lower GPA student, but again, this result bears further study.There were no statistically significant correlations with FFM personality traits or GPA with the third survey question, “How productive was the group overall?” This result suggests that one student’s ability has little bearing on group performance perception.The students’ responses to the three survey questions in Part I of the teamwork survey were then probed for correlations with team characteristics to determine if the composition of the group was associated with these outcomes. For each group, the mean, maximum, and minimum values for each of the FFM personality traits were computed, as well as the difference between the maximum and minimum values called the delta. The delta value was used to quantify the variability in a particular trait for a given group. All of these group characteristics were then used as independent variables.The only statistically significant correlation from this exercise was a negative correlation between the second question (Which of the following best describes the level of conflict at group meetings?) and the mean agreeableness of the group . In other words, groups with a higher average agreeableness score tended to report less conflict. It is intuitive and not surprising that groups comprised of agreeable members generate and experience less conflict. There were no other significant associations with the other team characteristics, or for either of the other two questions in Part I of the teamwork survey.What is noteworthy of this report is not only which correlations were significant, but also which ones were not. For example, there was no association found between students’ rating of group productivity and group GPA characteristics. Other correlations found in prior literature for self- and peer-rated team outcomes have not been replicated in this particular study.Lastly, direct measures of team achievement were examined through the project deliverables in the courses studied. In all participating courses except for the industrial design senior project, a written report was required which was assigned a team grade. The same team grade was assigned to all team members. In the engineering senior design courses, an oral presentation was additionally required which assigned a team grade equivalently to all team members. Pearson’s correlation coefficients were computed for the scores in both cases, as shown in Figure The written reports  were similar in scope in that they all required some research, analysis, and a proposed solution in each discipline. Due to the inherent variability in the topics covered, identical grading rubrics were not used for grading purposes, and reliability and validity measures were not computed. It is accurate to say, however, that the team grades in this study were measures of how well each group met the expectations of the instructor, which allows for some flexibility from discipline to discipline. The lack of validity and reliability between disciplines and instructors is noted as a limitation of this study, and should be considered in the interpretation of the results. For the data examined, extraversion was linked to both written report and oral presentation scores, and GPA was linked to oral presentation scores. The written report score was positively correlated to the mean extraversion score of the group  and the maximum extraversion score in the group . It was not correlated with the minimum extraversion or delta extraversion scores, nor was it correlated with any other FFM trait or any measures involving GPA. The oral presentation scores that were available were positively correlated to the maximum extraversion score in the group , the mean GPA of the group , and the maximum GPA score in the group .Extraversion and cognitive ability of teams were found to be relevant in supervisor’s rating of team performance in a manufacturing environment in the prior literature . ExtravThe association between oral presentation score with both mean and maximum GPA of the group suggests that a team’s oral presentation task has conjunctive and disjunctive characteristics. The group output is a sum of the individual contributions, and perhaps strong ability members shoulder the bulk of the responsibility. The written report task, on the other hand, did not exhibit correlation to team ability measures.In summary, individual students’ perceptions that their group members shared equally in the responsibilities was negatively correlated to the individual’s emotional stability. The conflict reported by students was positively correlated to individual students’ GPAs, as well as negatively correlated to the mean agreeableness score of the group members. Written report scores were positively correlated to the mean and maximum extraversion scores in the group. Oral presentation scores were positively correlated to the maximum extraversion, mean GPA, and maximum GPA scores of the group.For this analysis, individual performance on a team was quantified through student self-assessment and peer-assessment of their teammates. The instructor cannot be present for much of the interactions within a group, and the students are in a better position to rate the individual performance of his or her team members.The Pearson’s correlation coefficient was computed between the students’ responses to the questions in Part II of the teamwork survey found in Additional file Resulting statistically significant correlations between self-assessment and individual-level traits  are indicated in Table Resulting statistically significant correlations for peer-assessment are shown in Table It is interesting to note that the attributes that are correlated with a student rating himself or herself favorably on individual performance were not necessarily the same as the attributes that appear to lead one’s peers to rate him or her favorably. Self-assessment of various aspects of individual performance on a team was correlated to GPA and all FFM personality traits except for openness, and arguably the most strongly with GPA, extraversion, and conscientiousness. Peer-assessment, on the other hand, was correlated most strongly with GPA.An obvious question to ask is whether self-assessment correlates to peer-assessment. The correlation coefficient was computed for each question in Part II of the survey between a student’s self- and peer-rating for each semester. Significant results are summarized in Table As shown by Table In addition, t-tests were performed using paired samples with the self- and peer-ratings in both semesters, and the following significant differences were found. Students rated themselves more favorably in taking leadership roles  and encouraging the group to be timely  compared to their peers’ rating of them. Although both questions showed positive correlation between self- and peer-assessment in Table In summary, self- and peer-assessments of individual performance on a team were largely, but not entirely, correlated, although self-assessment of individual performance exhibited correlation with a greater number of individual traits. There was a tendency for students to rate themselves more favorably than their peers in some of the individual performance questions posed.Research on effective teamwork in multidisciplinary teams is increasingly important as universities strive to incorporate it into curricula. In the current study, one student group from each of the five participating courses in this study also collaborated on a multidisciplinary project of mutual interest – in this case, the design and construction of a solar-powered house. Each student group was graded for their disciplinary piece of the overall project by their course instructor, but was required to exchange information and collaborate with the other teams outside of their discipline to do so. There are, of course, other significant outcomes required for successful multidisciplinary projects compared to disciplinary ones, such as the need to identify contributions, information needs, and constraints of multiple fields, as well as valuing, integrating, and learning from multiple fields . This iAlthough the sample size is small (only 5 participating teams), this pilot effort was briefly examined in the context of the current teamwork analysis. The students participating in the multidisciplinary project were asked to rate each of the participating disciplinary teams using the questions in Part II of the survey. Students’ ratings of their own team were averaged for each question and denoted the ‘self team’ rating. A team’s ratings from students in the remaining four disciplinary teams were averaged for each question and denoted the ‘peer team’ rating.Unlike the disciplinary teams, the self team- and peer team-ratings were not well-correlated in the multidisciplinary project case study. The only significant correlation between self- and peer-ratings was “difficulty negotiating” . Furthermore, peer team-rating was generally more negative than self team-rating. Significant differences indicated by a t-test included the items “delivered work when promised”  and “communicated ideas clearly and effectively .This preliminary analysis highlights the increased importance of effective communication in multidisciplinary projects. It is likely that additional skills beyond those needed in disciplinary projects are required. The importance of negotiation skills is shown by the high awareness of teams having trouble negotiating, as indicated by the strong correlation between the self team- and peer team-ratings. In addition, the significant discrepancies between the self team- and peer team-ratings suggest a tendency to value one’s own disciplinary contributions over others and/or difficulty communicating disciplinary contributions to others.There are several important limitations to note about this study. First, the sample was drawn from five specific courses at our large public university, and was male-dominated. Although this is typical of the disciplines involved in this study, to what extent this sample is representative of the upperclassmen population in higher education as a whole is not known. Reliability and validity measures were not computed in the written report and oral presentation grades between instructors due to the inherent differences in disciplinary topics covered, and should be considered in the interpretation of the results. Also, some of the courses in the current study were the first semester course in a two-course sequence. The student teams in this study were formed at the start of the semester and were studied for one semester. Student teams in formation for a longer period of time may exhibit different dependencies and dynamics as the members get to know each other than newly formed teams. Finally, the scope of the project assigned to the groups was not varied in this study, and the results may change if the nature of the task assigned is also changed.The purpose of this study was to examine the influences of personality and ability on teamwork, team performance, and individual performance on a team, in a multidisciplinary student sample. It measured personality using the five-factor model personality traits and measured students’ ability in terms of GPA. The dependent variables included self-, peer-, and instructor-assessment of team skills, team performance and individual contributions.Notwithstanding its limitations discussed in the previous section, our study presents several findings, which are summarized below:Ability and personality on group performance: Students’ perception that their group members shared equally in the responsibilities was negatively correlated to their emotional stability. This result is contrary to that of prior studies (Ozer and Benet-[·d Benet-) and coud Benet-). The cod Benet-). These On the other hand, ability measured by GPA scores was negatively related to students’ assessment of teamwork in terms of the degree of conflict within the team. This implies that ability could be related both positively and negatively to team performance depending on the aspect of performance being measured and whose assessment we look at, whether that of team members (students) or external assessors (instructors).Implications for instructors and teams can be deduced from these results. Understanding the positive roles of extraversion in team composition in addition to ability can help form or guide effective teams. Along the same lines, high emotional stability or ability has been shown to have negative impacts on team perception, and this awareness can be used to address related issues if they arise.Ability and personality on individual performance on team: Individual performance was measured by ten questions that focused on aspects such as engagement, leadership, and communication. Self- and peer-assessments of individual performance on a team were largely, but not entirely, correlated. Self-assessment of individual performance exhibited correlation with GPA and all of the FFM personality traits except openness. Peer-assessment largely correlated with GPA. There was a tendency for students to rate themselves more favorably than their peers on (a) taking a leadership role and (b) encouraging the group to be timely.·Implications for instructors and teams, again, include promotion of increased awareness of trends found and application of the results. Furthermore, the role of GPA and the FFM personality traits on self- and peer-assessment, in addition to the tendency of students to rate themselves more favorably than their peers can be used in the interpretation of group conflict descriptions, and ultimately the course of action if teaching moments or intervention is required.Furthermore, a pilot multidisciplinary project team consisting of five disciplinary teams of approximately five members each was briefly examined in the context of this study. There was agreement as to which disciplinary teams were having difficulty negotiating. In addition, the disciplinary teams tended to rate themselves much more favorably than the peers outside of their discipline in (a) delivering work when promised and (b) clearly communicating. These results suggest the need for better negotiation and communication skills to manage diversity across disciplinary teams to promote a better understanding of different disciplinary backgrounds, cultures, and contributions.The findings in this study illuminate associations between personality, ability, and teamwork for a type of task that is commonly assigned as a group project at many universities. The range of correlations found in this study and their comparison to the prior literature indicates the dependence of the results on the population and/or the environmental-level factors that vary from study to study. This dependence is not well-understood, and merits further research. This study outlines the results found in our multidisciplinary undergraduate population and has possible extension to other similar populations. It is the hope that by better understanding important influences on successful teamwork, teamwork instruction and intervention will continue to become more effective.Instructional materials promoting awareness of personality influences on teamwork could be developed for project-based team instruction. The awareness itself may help students predict, explain, and help resolve difficulties that often arise in teams. For example, extraverted individuals on a team appear to be correlated with various measures of team performance. Teams without extraverted members who are aware of the correlation could try to compensate in other ways such as development of other social skills.Effective intervention strategies for instructors supervising group projects could be developed. Instructors (or managers) seeking to form high-performing teams, or those seeking to teach students how to maximize productivity in any given team would benefit from such strategies.Further research is required to map correlations that exist for external-level factors .Standardized rubrics for quantifying achievement on written reports and oral presentations could be developed and used to ensure validity. A calibration routine among instructors using the rubric could also be developed to ensure reliability.Lastly, the influence of personality and ability in project teams that span more than one semester was not studied in this work, and would be of interest in longer term projects.This work answered some questions, and raised many others. Suggestions for instructors seeking to apply the results of this and related papers, or for education researchers continuing this line of inquiry include the following: Jinny Rhee is an Associate Professor in the Department of Mechanical and Aerospace Engineering at San JoseÂ´ State University, where she has been since 2002. Her interests include renewable energy technologies, thermal management of electronics, and multidisciplinary and interdisciplinary education for engineers. Dr. Rhee completed a B.S. (1989), M.S. (1990), and Ph.D. (1995) at Stanford University, all in mechanical engineering. Dr. Rhee is an active member of ASME, IEEE—CPMT , and SWE (Society of Women Engineers).David Parent is an Professor in the Department of Electrical Engineering at San JoseÂ´ State University where he has been since 1999. His passion is bringing practical engineering experiences to the classroom and using his research in the multidisciplinary areas of solar cell fabrication, artificial neurons, and neural interfacing as case studies for undergraduate and masters degree students. He earned his BSEE (1992), MSEE (1996) and Ph.D (1999) from the university of Connecticut. He is an active member of IEEE.Anuradha Basu is a professor in the Department of Organization and Management in the College of Business, and Director of the Silicon Valley Center for Entrepreneurship, at San Jose State University. Her research interests include the impact of entrepreneurship education and immigrant entrepreneurship. She obtained a BA (Honors) degree in Economics from the University of Delhi, India, an MBA from the Indian Institute of Management, Calcutta, and MPhil and PhD degrees in Economics from the University of Cambridge, UK.Online survey questions on teamwork. (DOC 50 KB)Additional file 1:


To the Editor: As a resident in pediatrics with an interest in infectious diseases, I was fascinated by the range and scope of conditions I saw in May and June of 2005 in local children in Kampot Province, a mainly rural area in southeastern Cambodia. This province, near the Vietnamese border, is the epicenter of H5N1 avian influenza in Cambodia, with all 4 known human cases of the disease. All of these documented human cases have been fatal.While the World Health Organization and the Cambodian Ministry of Health have engaged in public education, the village-level response to a pathogen of potential global importance is evolving. As part of my work in Cambodia, I made numerous information-gathering visits to villages in Kampot Province. Though village elders and health workers had often heard of the "bird flu," most of the villagers I spoke to had not. Many persons did not know whom to contact should their chickens or other birds die. Those who knew about H5N1 influenza told me that, without adequate compensation for culling flocks, little incentive would exist to report bird deaths. In a typical village, chickens, ducks, and pigs intermingled with each other and with humans underneath or around homes on stilts. General knowledge of infection control practices among villagers was minimal.The dissemination of information into a rural, agricultural society such as that in the southeast of Cambodia is a difficult task. Many rural inhabitants do not have televisions or radios and may infrequently travel to larger towns. Health workers from international groups, nongovernmental organizations, and the government are often required to travel on foot or motorbike through fields and forests to reach and educate the population. Government health workers lack the personnel and resources to adequately identify and investigate potential cases, and Cambodia has substantially fewer microbiology laboratories than do neighboring Thailand and Vietnam.Should a pandemic of avian influenza occur, it will almost certainly originate in Southeast Asia. Cambodian and international health organizations have recognized the country’s potential key role in propagation of an impending pandemic agent. However, because of its history and current economic state, Cambodia is less able to respond to the avian influenza threat than its neighbors. In recognition of this fact, the World Health Organization and the Cambodian Ministry of Health have stated that the prevention, control, and identification of avian influenza are national priorities. Additionally, international funds have been flowing into Cambodia to assist with avian H5N1 influenza surveillance and case investigation. Much work remains to be done; we hope that by combining international resources and policy with domestic expertise and effort, Cambodia will mount a successful response against this emerging threat.


Spatial planning affects the built environment, which in turn has the potential to have a significant impact on health, for good or ill. One way of ensuring that spatial plans take due account of health is through the inclusion of health considerations in the statutory and non statutory appraisal processes linked to plan-making processes.A systematic review to identify evaluation studies of appraisals or assessments of plans where health issues were considered from 1987 to 2010.A total of 6161 citations were identified: 6069 from electronic databases, 57 fromwebsite searches, with a further 35 citations from grey literature, of which 20 met the inclusion criteria. These 20 citations reported on a total of 135 different case studies: 11 UK HIA; 11 non UK high income countries HIA, 5 UK SEA or other integrated appraisal; 108 non UK high income SEA or other integrated appraisal. All studies were in English. No relevant studies were identified reporting on low or middle income countries.The studies were limited by potential bias , lack of detail and a lack of triangulation of results. Health impact assessments generally covered the four specified health domains  more comprehensively than SEA or other integrated appraisals, although mental health and wellbeing was an underdeveloped area. There was no evidence available on the incorporation of health in Sustainability Appraisal, limited evidence that the recommendations from any type of appraisal were implemented, and almost no evidence that the recommendations had led to the anticipated outcomes or improvements in health postulated.Research is needed to assess (i) the degree to which statutory plan appraisal processes (SA in the UK) incorporate health; (ii) whether recommendations arising from health appraisal translate into the development process and (iii) whether outcomes are as anticipated. Spatial planning affects the built environment, which in turn has the potential to have a significant impact on health, for good or ill For example, the level of active travel  and outdoor recreational activity is strongly affected by accessibility to local facilities. Access to green, natural environments, and to local social networks, are factors in mental well-being. The wider sub-regional pattern of housing, economic development, land use and transport is a determinant of social exclusion and therefore health inequalities . One wayDifferent appraisal and assessment techniques deal with health to different degrees; Health Impact Assessment (HIA) of course has health as its raison d'être, but is not a statutory requirement. In contrast, Strategic Environmental Assessment (SEA) and Sustainability Appraisal (SA) should, if properly undertaken, include consideration of all the main environmental determinants of health. (SEA) is a requirement in all countries in the European Union under the European Directive 2001/42/EC, and this assessment must consider both 'Human Health' and 'Population'. This has recently been extended with the Protocol on SEA to the United Nations Economic Commission for Europe (UNECE) Espoo Convention, of which the European Union is party, which came into force on 11th July 2010, and provides a legal basis for enhanced attention to human health in the SEA process, and for the health sector to to be routinely consulted on development plans [The study   aimed toThe search strategy to identify evidence from electronic databases was developed in an iterative manner to explore the concept areas of assessment/appraisal processes, plan initiatives and health outcomes No language restrictions were applied when conducting electronic database searches. All references identified were screening using title, abstracts or full texts, facilitated through the use of a checklist screening tool. Titles and abstracts of de-duplicated citations were screened independently by two reviewers to determine eligibility where adequate information was available. A data extraction form was developed for included studies to review the extent to which each study provided evidence on any/at least one of the following:• Health issues were considered in the appraisal• Health-related recommendations were incorporated into the plan• Health-related recommendations were implemented• Post plan adoption health outcomes were evaluated.Four health issues were explicitly considered: physical activity, mental health and well being , environmental health factors  and unintentional injury. If other specified potential impacts  were described these were noted on the data extraction form. Data extraction was undertaken by a single reviewer who was not blind to the name of the authors, institution or source of the citation. Difficulties in data extraction were resolved through discussion within the review team.To assess study quality each included paper was critically appraised using the methods developed by NICE . An InteBecause of the differing regulatory frameworks within developed and less developed countries, and the particular interest of appraisal in the UK, the studies were grouped by UK, other high income countries and medium/low income countries.A total of 6161 citations were identified: 6069 from the electronic databases, 57 from website searches, and afurther 35 citations were identified from grey literature, primarily a call for evidence by NICE on the topic of spatial planning and health. De-duplication, followed by screening of title and abstracts, excluded 5,926 citations. The full text of 234 remaining citations were obtained and screened. Of these, 20 met the inclusion criteria and quality checks , lack of detail and a lack of triangulation of results.A summary of all included papers is shown in Table Seven citations were identified, reporting eleven case studies, from Scotland, England and Northern Ireland over the previous 10-12 years ,18,24. TWhilst there is comprehensive consideration of health issues in the plan appraisal, there is little reported evidence that this consideration led to changes in the plans themselves, nor that changes were implemented nor had an impact on health outcomes. Three report evidence of health recommendations being incorporated into the adopted plan ,12,18, bNine citations were identified that report 11 relevant case studies in four countries ,19,22-24In eight of the 11 case studies, it was reported that health recommendations were incorporated into the plans, there is no clear evidence that health considerations influenced the implementation of the strategy, either because the citation did not report on it or the policy process was still not advanced enough at the time of writing to report on post adoption impacts.Generally speaking, the case studies covered all the four specific health issues but only three case studies dealt with all the four specified issues. Ten covered other health outcomes, including access to services, urban design and housing, availability and control over housing, social connectedness, housing, transport, engagement with Maori, neighbourliness, and social cohesion were considered.Three citations were identified reporting five case studies ,20,21. IWhilst all case studies considered health issues in the appraisal process, it is unclear or not reported if health recommendations were incorporated into the plan, or whether the relevant policies were acted upon or implemented. No post plan impacts were reported although Plant  notes thThree citations were identified that report 105 relevant case studies of SEA in four countries, including three detailed studies in Germany and the Netherlands,  of a widThe SEA Directive (Directive 2001/42) on the assessment of the effect of certain plans and programmes on the environment is implemented by all EU member states and serves as legislative basis for case studies in Germany, the Netherlands and Denmark. The Hong Kong Special Administrative Area's Government issued a circular in 1988 integrating environment assessment process consistent with SEA within the planning process of Hong Kong.considered in SEA and consequently did not report on how health considerations impacted on the specific plans [Whilst all citations provided evidence that health issues are considered in SEA, only one case study reported that health recommendations were incorporated into the plan - a transport study in Hong Kong . Anotheric plans . None ofThe range of health issues considered in the case studies varied, although only one refers to mental wellbeing, and the two studies from Hong Kong do not appear to report on any issues other than those relating to environmental health . Other issues included were light pollution, biodiversity, and risk of crime.Figure Figure Figure Equity issues, and consideration of the differential distribution of impacts appears to be relatively underdeveloped in all appraisals.Equity was mentioned explicitly as an important area that had been considered in the appraisal in six of the citations, three of which related to HIA ,17,18,24Whilst SEA and SA are widely used and are statutory requirements across a wide range of juridstictions, there is a conspicuous lack of evidence of evaluations in this critical area relating to UK practice, with only three studies identified, and two relating to other forms of integrated appraisal (one an SA and one an EqA). Given the need public authorities to fulfil statutory equalities duties in the UK it is suprising that only one EqIA was identified.There is little evidence that health issues were incorporated nor that health-related recommendations were incorporated into the adopted plan documents, and there is no information given about implementation. Whilst these case studies are highly applicable to the UK and the current spatial planning system, as only three case studies were identified, it is important to recognise that these examples may not be representative of SA/SEA practice in the UK. Outside the UK, there is strong evidence from all five case studies that health is considered in SEA, but no evidence that the SEA health recommendations had been implemented at post-adoption stage. One might argue, as Fishcher  has doneSimilar issues in terms of evaluation are found in relation to HIA. Of the eleven UK case studies identified, only one case study reported HIA effectiveness in terms of completion of all stages from health recommendations, to implementation and post adoption evaluation . Many reThe case studies strongly suggest that factors such as the timing of appraisal (late HIAs have been reported to have limited impact), and community engagement are critical in the success of appraisal. Full integration of comprehensive health assessment into existing formal and statutory processes increase the likelihood of health being properly considered and incorporated into the plan. However, there is a lack of data on outcomes to support this supposition.There are limitations in the literature reviewed. Many of the publications are reports from authors who have themselves been directly responsible for undertaking the appraisal, with little independent evaluation or triangulation of reported findings, thus leading to potential bias. We were unable to access the full text of four potentially suitable articles with English abstracts but full text in other languages. Given the complexity and timescale for development, there are practical difficulties in both tracking, and attributing recommendations and changes in plans and subsequent developments to appraisal processes. Whilst the lack of evidence per se does not mean that there is a lack of effectiveness, the dearth of evidence linking appraisals to implementation and subsequent changes in outcomes is challenging. Concerns about the lack of evaluation of the impact of HIA have also been noted in the past by others , and guiThe study suggests that there is considerable variation in the degree to which health issues are comprehensively considered, with evidence that mental health and wellbeing issues may be particularly under-reported in SEA and other integrated appraisals. Equity issues, and consideration of the differential distribution of impacts appears to be relatively underdeveloped in all appraisals. This has implications for the training of those involved in undertaking appraisals. It is possible, that particularly during HIA, a fuller more comprehensive range of health issues was considered at the scoping and screening stage, but if no significant impacts were identified that these were not considered further.Posas summarises the development of HIA in the context of SEA , highligHowever, health appraisal is only one of part of the development plan process; health considerations need to be built in at the very early conception and development of plans , and critically, followed through to the development management process. There are clear implications for the training of planners, developers, and those involved in undertaking appraisals.A particular point of note is the dearth of evidence from low and middle income countries. Outside the EU some countries have adopted SEA practice, or some strategic form of EIA, but there is very variable uptake and use of HIA as highlighted by Erlanger  who in aIn conclusion, action is required; firstly, to ensure that a firstly that a comprehensive approach to examining potential health impacts is undertaken, ensuring that relatively neglected areas such as mental health and well being and equity are addressed; secondly that due attention is paid to ensuring that the recommendations arising from consideration of health issues in stand alone or integrated appraisals are embedded into plans; thirdly that attention needs to be given to the current regulatory framework to ensure that evaluation and post-development monitoring is undertaken; and finally that there is more work undertaken to ensure that recommendations translate into the development process and that outcomes are as anticipated.No financial competing interest declaredHugh Barton is Director of the WHO Centre for Healthy Urban Environments, UWE.SG and HB conceived the concepts; JM developed the search strategy and data screening and extraction tools; JJ, LM and HL undertook searching of electronic databases, screening and data extraction; SG, HL, JM and HB wrote project report and SG prepared paper for publication. All authors have commented on and approved of final draft.NICE Centre for Public Health Excellence.The views expressed in this paper are those of the authors and do not represent the views of NICE as funders of this work.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/11/889/prepubThe effectiveness of health appraisal processes currently in addressing health and wellbeing during spatial plan appraisal: a systematic review.Click here for file


School closure was employed as a non-pharmaceutical intervention against pandemic 2009 H1N1, particularly during the first wave. More than 700 schools in the United States were closed. However, closure decisions reflected significant variation in rationales, decision triggers, and authority for closure. This variability presents the opportunity for improved efficiency and decision-making.We identified media reports relating to school closure as a response to 2009 H1N1 by monitoring high-profile sources and searching Lexis-Nexis and Google news alerts, and reviewed reports for key themes. News stories were supplemented by observing conference calls and meetings with health department and school officials, and by discussions with decision-makers and community members.There was significant variation in the stated goal of closure decision, including limiting community spread of the virus, protecting particularly vulnerable students, and responding to staff shortages or student absenteeism. Because the goal of closure is relevant to its timing, nature, and duration, unclear rationales for closure can challenge its effectiveness. There was also significant variation in the decision-making authority to close schools in different jurisdictions, which, in some instances, was reflected in open disagreement between school and public health officials. Finally, decision-makers did not appear to expect the level of scientific uncertainty encountered early in the pandemic, and they often expressed significant frustration over changing CDC guidance.The use of school closure as a public health response to epidemic disease can be improved by ensuring that officials clarify the goals of closure and tailor closure decisions to those goals. Additionally, authority to close schools should be clarified in advance, and decision-makers should expect to encounter uncertainty disease emergencies unfold and plan accordingly. As concern about pandemic influenza in the United States grew in recent years, school closings increasingly were seen as a means of "social distancing" capable of slowing the spread of disease through the population . Thus, wDuring the 2009 H1N1 outbreak in the United States there appeared to be extensive variation across the country in decisions made about school closures. As a result, decisions often appeared inconsistent, contributing to a sense that the government did not know how to respond, and perhaps was ineffective in meeting public health goals. In the systems improvement perspective, excess variation in health system structures and processes is a cause for concern, indicating possible inefficiencies and opportunities to improve the system ,6. Adoptper se but rather to use them to identify problems that are likely to arise in future pandemics and other public health emergencies and learn from them to improve public health systems' capabilities. Similarly, we do not presume to say what school closing policies should be, but rather focus on the lessons learned from the 2009 experience to strengthen public health systems' response to future events.In particular, we describe three key issues that challenged decision-makers during the 2009 H1N1 outbreak: 1) clarifying goals, 2) clarifying authority and the decision-making process, and 3) dealing with uncertainty in preparing and issuing official guidance. Although we refer to a number of specific decisions that were made, our purpose is not to critique those decisions Many aspects of the 2009 H1N1 outbreak were well described in the media, on the Internet, and in scientific publications, so we have based this analysis primarily on monitoring key sources and Lexis-Nexis news and Internet searches. In particular, we identified media reports via Google alerts and Lexis-Nexis' database of all major newspapers in the United States using the search terms "swine flu," "H1N1," and "school closure" (as well as appropriate truncated and extended variations of these search terms). We included local and national media sources in the analysis, and we accepted overlapping stories in multiple sources in order to gain a full picture of events taking place. As appropriate, additional media reports were actively sought to better understand the context for particular localities' decisions. The purpose of the searches was not to systematically quantify each issue that was identified but, rather, to identify the scope of issues that arose and decisions that were made, so we stopped searching when new issues were no longer appearing. Our focus was on events during the Spring, but we consider some distinctions with the Fall in the discussion below.ad hoc during flu clinics or meetings, as decision-makers did not have time for extensive formal interviews.At least two members of the research team read each media report and noted key issues that arose. In addition, we observed conference calls and meetings between health departments and school officials in the Boston and Washington, DC metropolitan areas, and we held informal interviews with decision-makers and community members as feasible. Many interviews were conducted We organized the data collected from media reports and interviews into major themes, which we then vetted with public health researchers, practitioners, and education officials to ensure we captured the key issues and did not exclude any obvious concerns. While these methods cannot tell us the frequency with which specific issues arose, we are confident that we have identified and developed at least a basic understanding of the key public health issues.ad hoc basis. After CDC issued preliminary guidance on April 26, schools with 2009 H1N1 cases generally closed for at least seven days. On May 1, CDC changed its guidance to suggest 14 days of closure, and most schools with cases closed with the intent to remain shuttered for 14 days. However, CDC guidance changed again on May 5 to say that closure was generally not necessary. After May 5, most schools did not close after a case was identified, with the notable exception of the New York City school system, which continued closing schools to protect particularly vulnerable children into June. A timeline of major school closure events is provided in Table School closures during the pandemic's first wave occurred in several rough phases. Initially, schools were closed on a somewhat Three different rationales for closing schools during a public health emergency emerged in the 2009 H1N1 outbreak: (1) limiting spread of the virus in the community, (2) protecting vulnerable children, and (3) reacting to staff shortages or children kept at home because of infection or parents' fears of infection.In the United States as in Europe , the mosper se, many of which had significant methodological challenges  confusion about authority and jurisdiction helped lead to distrust in health officials and political leaders." The 2009 as well as the 1918-19 experiences suggests that, at the very least, officials must consider in advance who has the authority to close schools and identify what goals they wish to accomplish through a potential closure.Inconsistencies of this sort can be particularly obvious in a region such as the Washington DC metropolitan area, which includes two states as well as the District of Columbia, plus numerous local jurisdictions, but one media market. In such settings, and even in isolated jurisdictions, it seems important to be sure that reasons for differences in decisions be clearly communicated to the public.Regardless of who has the formal authority to close schools in a jurisdiction, the 2009 experience shows that it is helpful to solicit input from a range of stakeholders such as local and state health and school officials as well as students, their parents, and school staff. Broader inclusion of stakeholders both improves the likelihood that decisions are made with full information and promote consideration of benefits and costs that will accrue to different affected groups. Coordination of this sort also helps to ensure the credibility of the message . The NewDecisions about whether and how to close schools logically depend on epidemiological information about who is likely to be infected, the severity of illness in those infected, and periods of infectiousness. Infectious disease outbreaks, however, are often characterized by scientific uncertainty , and theIf the 2009 H1N1 pandemic is any guide to the future, public health and other officials should expect and plan for uncertainty about the facts and frequent changes in official guidance that is based on a constantly evolving epidemiologic knowledge base. This begins with acknowledging the uncertainty and requires flexibility in policies and procedures such as consideration of variations of school closing and alternatives as discussed above. During the 2009 H1N1 pandemic, for instance, the goal of the acting CDC Director Richard Bessor was to "tell everything we knew, everything we didn't know, and what we were doing to get the answers" .In particular, because H1N1 has and will likely continue to affect communities differentially; both socially and epidemiologically, monitoring the situation at the local level will play an important role in closure decisions. Communities will experience varying levels of transmission, and some populations have more people who are highly susceptible to complications than others. Wealthier communities may be able to keep schools closed as parents have childcare alternatives, while communities with more disadvantaged populations may depend on schools for a variety of resources such as childcare and school lunches. The 2009 experience suggests that local school and health officials must monitor the situation in their communities and work together closely to integrate local information with state and federal guidance.Planning for uncertainty includes development of systems to track the epidemic and its consequences as well as evaluate the impact of control efforts, that is, to provide "situational awareness." It also requires a degree of humility in presenting decisions to both senior policy makers and the public, clearly stating the basis on which decisions were made and noting the likelihood that they can, or are likely to, change.A careful analysis of the 2009 H1N1 outbreak finds extensive variation across the United States in expressed rationales, decision triggers, and decision-making authority for school closures. This led to decisions which were often inconsistent from one locale to another and over time, contributing to a sense that the government was unsure how to respond and perhaps ineffective. From a systems improvement perspective, however, such excess variation in is a cause for concern, indicating possible inefficiencies and opportunities to improve the system.Because school closure decisions require local public health concerns to be balanced with broader societal concerns, analysis of the 2009 H1N1 pandemic suggests three issues public health and school officials should consider in planning for and making school closure decisions in the future. First, the goal of school closing should be made clear and specific measures should be tailored to the goal and modified over time as evolving knowledge requires. Second, legal and practical authority to close schools should be clarified in advance, as part of planning and preparedness efforts. Finally, decision-makers should expect uncertainty and maintain situational awareness, be flexible in policies and procedures, and act with humility.The authors declare that they have no competing interests.JK and TK searched the literature to identify information for this analysis. All authors participated in designing the study, analyzing and interpreting the data, drawing policy conclusions, and drafting the manuscript. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/11/73/prepub