There has been increasing interest in enhancing accountability in health care. As such, several methods have been developed to compare the quality of home care services. These comparisons can be problematic if client populations vary across providers and no adjustment is made to account for these differences. The current paper explores the effects of risk adjustment for a set of home care quality indicators (HCQIs) based on the Minimum Data Set for Home Care (MDS-HC).A total of 22 home care providers in Ontario and the Winnipeg Regional Health Authority (WRHA) in Manitoba, Canada, gathered data on their clients using the MDS-HC. These assessment data were used to generate HCQIs for each agency and for the two regions. Three types of risk adjustment methods were contrasted: a) client covariates only; b) client covariates plus an "Agency Intake Profile" (AIP) to adjust for ascertainment and selection bias by the agency; and c) client covariates plus the intake Case Mix Index (CMI).lower than the WRHA. In the WRHA, individual agencies were likely to experience a decline in their standing, whereby they were more likely to be ranked among the worst performers following risk adjustment. The opposite was true for sites in Ontario.The mean age and gender distribution in the two populations was very similar. Across the 19 risk-adjusted HCQIs, Ontario CCACs had a significantly higher AIP adjustment value for eight HCQIs, indicating a greater propensity to trigger on these quality issues on admission. On average, Ontario had unadjusted rates that were 0.3% higher than the WRHA. Following risk adjustment with the AIP covariate, Ontario rates were, on average, 1.5% Risk adjustment is essential when comparing quality of care across providers when home care agencies provide services to populations with different characteristics. While such adjustment had a relatively small effect for the two regions, it did substantially affect the ranking of many individual home care providers. In both Canada and the United States efforts are underway to develop systems to assess the quality of health care as a first step to improving services. In the US nursing home sector, the implementation of the Minimum Data Set has been linked to improvements in quality of care -4. RecenWhen comparing health care providers across a set of QIs there is a concern that they may differentially admit clients with a greater likelihood of triggering on quality issues. Since the indicators are defined as events that are preferable to avoid , in the absence of risk adjustment, these providers would appear to be delivering poorer quality of care. Risk adjustment attempts to adjust for different populations of clients who may be at greater risk for experiencing quality issues that is a function of their clinical status rather than the quality of care.In the US nursing home sector, the CMS recently funded a large-scale project to review all potential long-term care QIs and the risk adjustment process. The evaluation of risk adjustment included a review of both client-level and agency-level covariates. The research team revised the original set of client covariates to create a set of new models and assessed the effects of using an agency-level covariate, the Facility Admission Profile (FAP). The FAP represents the prevalence of the quality issue for residents newly admitted to the facility and was intended to adjust for potential selection bias  and ascertainment bias . However, after extensive analyses, the research team did not recommend the use of the FAP, given that the FAP did not appear to be a particularly useful measure of ascertainment bias .Nevertheless, in October 2002, the CMS included the FAP in three quality measures reported on the Nursing Home Compare web page. As partThe current research was part of a project to develop a set of 22 Home Care Quality Indicators (HCQIs) based on items in the Minimum Data Set for Home Care (MDS-HC). Implementation of the MDS-HC is complete or underway in 15 states and in 7 Canadian provinces and territories, where it is being used mainly as a clinical assessment instrument. In Michigan, the MDS-HC is being used as part of the MI-Choice waiver program to reduce nursing home admissions. In Ontario, Community Care Access Centre (CCAC) case managers use the instrument to determine needs, allocate services and make placement decisions.The HCQI derivation was accomplished using data from Ontario and Michigan since these were the regions with the largest scale implementation in Canada and the US, respectively. In their paper describing the HCQI derivation, Hirdes et al. recommenRAI, a non-profit multinational organization dedicated to the development and refinement of assessment instruments for older adults and persons with disabilities, and their related applications. The HCQIs represent outcome measures that document an agency's rate for triggering on a quality issue. For example, one indicator measures the prevalence of weight loss among individuals who are not considered to be palliative clients. The HCQIs include a mix of prevalence measures  and incidence measures . All HCQIs are defined as events to be avoided such that a higher rate on the indicator is indicative of poorer performance.The set of HCQIs used in this study were developed by members of interRAI HCQIs. It explores the effects of risk adjustment at both the level of the region and at the level of the individual home care provider.This paper uses a dataset from two Canadian provinces to further explore these three types of risk adjustment for the interThe RAI Health Informatics Project was a two and a half year research study begun in 1999 in which fourteen of Ontario's 43 CCACs used the MDS-HC as part of usual practice for all adult (18 years and older) home care clients. CCACs provide several different services: information and referral, case management and placement in long-term care facilities. CCACs purchase in-home services from external contracted agencies through a "request for proposal" process. Services provided include in-home physiotherapy, occupational therapy, personal support and homemaking, medical supplies and equipment, and care by other professional such as social workers, dietitians and speech-language pathologists. Case managers oversee the assessment process, make referrals to service providers and then monitor the care provided in the home. Funding for CCACs is based on an annual budget provided by the Ontario Ministry of Health and Long-term Care. CCACs are governed by independent, non-profit boards of directors, accountable to the Ministry of Health and Long-term Care.In Ontario, it is now mandatory for all long stay home care clients to be assessed with the MDS-HC. However, at the time of the RAI Health Informatics project, the MDS-HC was not mandated for use. As such, some clients  did not receive an MDS-HC assessment and were not included in the current project.At this same time, Manitoba conducted a pilot implementation of the MDS-HC in the fourteen offices of the Winnipeg Regional Health Authority (WRHA). The WRHA is one of twelve health authorities in the province, and is responsible for providing health services to the approximately 646,000 residents of Winnipeg and the surrounding suburban area. The WRHA receives annual funding from the government of Manitoba. The home care program of the WRHA was established in 1974 with a mandate to provide effective and responsive health care services in the community to support independent living and facilitate admission into LTC facilities when independent community living is no longer an option. Home care services include personal care, nursing, counselling, occupational therapy and physiotherapy assessment, referral to other agencies and coordination of services.In the WRHA region, an MDS-HC assessment was completed by care coordinators only if the client was anticipated to be on service for at least 90 days. As a result, the study sample was focused on a long stay population of home care clients.All participating sites in the WRHA and in Ontario identified a set of case managers who received a two-day training session led by a member of the research team. They then used the MDS-HC instrument as part of their usual in-home assessment.The data used for this study represent a cross-sectional cohort of home care clients assessed between November, 1999 and December, 2002. Two CCACs in Ontario and four WRHA sites, were removed from the database because they submitted fewer than 20 assessments, resulting in a total of ten WHRA and twelve Ontario sites. Of these, three Ontario CCACs and eight offices of the WRHA also submitted client reassessments at approximately 90 days, which allowed for the calculation of failure to improve/incidence HCQIs.The protocol for data collection was reviewed and received ethics clearance through the Office of Research Ethics at the University of Waterloo, Canada.Risk adjustment attempts to adjust for differences in client populations that may bias the HCQI rates. Organizations that provide care to more impaired clients will tend to have higher unadjusted rates, regardless of the quality of care they provide. As such, risk adjustment methods are used to maximize the ability to make fair comparisons across providers. With any type of risk adjustment, caution must be exercised to prevent over adjustment . The choice of risk adjustment covariates is therefore highly important and should not include variables that would be considered to reflect suboptimal clinical care.There are two generic types of risk adjustment that have been recommended for the HCQIs. The first adjusts for differences in the population at the client-level. PotentiThe second type of risk adjustment was intended to control for two types of bias at the agency level: a) the agency's ability to identify differences in clients' clinical characteristics and b) differences in who an agency selects for admission. These risk adjustments are performed at the agency level after the individual-level risk adjustments are applied.The Agency Intake Profile (AIP) was used following the methodology outlined by Morris et al., for the MDS 2.0 quality indicators. The AIPMore recently, an alternative form of agency-level risk adjustment has been proposed, to control for potential selection and ascertainment bias. This adjustment employs the Case Mix Index (CMI) associated with the RUG-III/HC methodology. In thisThe process followed for adjusting HCQI rates was the same as that used by Morris et al. in adjusting the US nursing home QI rates. Three sThe final adjusted value can be thought of as an estimate of an agency's HCQI rate if the agency had clients with an average level of risk. This riFor each participating agency, the unadjusted HCQI rates were calculated for all 22 indicators. These rates represent the average rate across all eligible clients for a given agency. For prevalence indicators, the rates were calculated only for clients who had been on service for at least 30 days to avoid penalizing an organization for quality issues that were newly recognized.Several methods were utilized to assess the impact of risk adjustment, both at the regional and at the agency-level. The unadjusted and three adjusted rates were compared between Ontario and the WRHA, to assess the effects at the regional level. At the level of the individual agency, ranks were calculated for each HCQI and a count was created to examine how often a given agency was among those with the four highest rates. Since higher rates on each HCQI were indicative of higher prevalence or incidence of undesirable outcomes, agencies ranked within the four highest rates were considered to be among the "worst performers." The range between agencies with the highest and fourth highest rates was also examined for both unadjusted and adjusted rates to assess the degree of variation among the group of worst performers.The two regions were very similar on average age and sex. The WRHA clients were significantly less likely to have some level of cognitive impairment, as measured by the Cognitive Performance Scale (CPS), compareWhen comparing the two regions, there were statistically significant differences for five of the 22 unadjusted HCQIs .Ontario also had a significantly (p < 0.0001) higher Case Mix Index (CMI) on intake , at 0.84 , compared with the WRHA at 0.70 .higher in Ontario than in the WRHA. Following the +AIP adjustment, Ontario had a rate that was 1.4% lower than in the WRHA.In general, the risk adjustment process minimized the differences between the two regions compared with the unadjusted rates. For example, the unadjusted difference between regions for the prevalence of disruptive/intense daily pain was 9.1%, however, the difference was reduced to 8.0% with the CC adjustment, 5.7% with the +CMI adjustment and 2.8% with the +AIP adjustment Table . The +AIA summary measure was created to count the frequency with which an agency was ranked among the worst performers. Overall, only three of the ten offices within the WRHA did not change their ranking when rates were adjusted Table . For exaIn Ontario, only CCAC 900 experienced no change in their ranking following risk adjustment Table . In anotExamining only the ranking of agencies may be misleading if very small changes in the actual rate resulted in an increase or decrease in the rank for a given agency. For example, the change in the rate for a given agency could be very small, say 5%, but could result in the organization moving from the fifth highest rate  to the third highest rate. Examining only the ranking tells one little about the actual magnitude of change among the worst performers. Therefore, it is also important to explore the range in the rates across the four highest agencies.The unadjusted and CC adjusted rates had the largest degree of variation between the highest and fourth highest agency, with a mean of 11% across the 19 HCQIs. The +CMI adjustment had the next largest amount of variation at 10% and the +AIP adjustment at 8% Table . These rOntario CCACs exhibited several key differences when compared to the WRHA. Ontario sites had significantly higher unadjusted rates across four HCQIs. However, the degree of variation between the sites was small, with a mean difference across indicators of less than 1%. Ontario as a region had significantly higher AIP values for eight HCQIs, indicating a greater propensity in Ontario to admit clients exhibiting quality issues, and they also had a higher mean Case Mix Index.Overall, the change in the actual rates at the regional level was small following risk adjustment. When risk adjustment was applied, for the client covariates alone or the client covariates with the CMI covariate, there was little effect on the mean difference between the two regions. In each case, the mean difference was positive  and less than 1%. The +AIP adjustment, however, resulted in a negative mean difference  across the set of risk-adjusted HCQIs.At the agency level, there was a greater influence of risk adjustment, as assessed by agency rankings across the set of indicators. In general, sites in Ontario benefited from risk adjustment and were less likely to be ranked among the worst performers. The opposite was true within the WRHA. Again, the +AIP adjustment had the largest influence when compared to the other two types of risk adjustment.It is possible that the level of variation between agencies on the AIP covariate was greater than the level of variation between clients on the individual-level covariates. Although a detailed analysis is beyond the scope of this paper, use of a single HCQI example may prove beneficial. When examining the prevalence of falls (an HCQI that showed the largest changes in the agency rankings following risk adjustment), the coefficient of variation (CV) for the AIP value was 23.0. The CV is a relative measure of dispersion about the mean and is calculated by dividing the standard deviation by the mean and multiplying by 100. This CV value was much lower than any of the corresponding CV values for the various MDS outcome scales that reflect differences at the level of the individual client. For example, the CV for the Cognitive Performance Scale was 158.5 and for the ADL Self-performance Hierarchy Scale, the CV was 195.6 (data not shown).Clearly, the +AIP adjustment had the effect of minimizing differences between the individual providers and it also resulted in rates that had less variability, as demonstrated by the drop in the CV value when comparing the unadjusted and +AIP adjusted rates for the prevalence of falls indicator . However, this effect cannot be explained by differences in the variances of the adjusters since the AIP had a lower level of variance than the individual covariates.It is also possible that the AIP covariate serves as a proxy for regional differences in practice patterns. If the Ontario agencies are grouped into four main geographic regions, the AIP value for the falls HCQI ranged from 24.1% to 34.0%. The degree of variation appears modest, but cannot be discounted as one factor in the ability of the AIP adjustment to minimize differences between providers and between regions.Although there continues to be support for the conceptual notion of risk adjustment, the potential to over adjust remains a concern. In a recent publication, Mor et al recognizIn the current project, the AIP covariate represents a continuous, numeric value that can range from zero to one. Thus, it does not simply serve as an agency identifier, but represents the rate among an intake cohort. Furthermore, in the US, Morris et al. determined that the Facility Admission Profile, the conceptual cousin to the AIP, did not substantially improve the risk adjustment models and they did not recommend its use. This decision was not based on a fear of over adjustment, but rather a concern that the FAP added increased complexity without significantly improving the risk adjustment process.inappropriate risk adjustment where variance is due to poor practices  and "over adjustment" due to the excessive use of spurious risk adjusters resulting in suppression of variance. Another possible example of over adjustment is the use of individual level covariates that are too closely related to the quality indicator. For example, one might argue that using dressing of the upper body as a risk adjuster for a QI on dressing the lower body is a form of over adjustment because the dependent variable is represented on both sides of the regression equation.It is also important that to develop a clearer understanding of what is meant by "over adjustment". For example, it would seem reasonable to differentiate between instances of truly This area of research continues to present many challenges. Given the current results and those from the long-term care sector, it appears advantageous to undertake some form of risk adjustment, even though the definitive method may not yet be best understood. Given that these HCQIs are new and so little is known about the risk adjustment process, it seems appropriate that all three type of risk adjustment be considered by researchers and policy makers. The +AIP adjustment represents the most conservative approach and may therefore be most appropriate for public reporting of HCQI results.The assessment of quality of care, and ongoing refinement of risk adjustment methods, is ultimately intended to provide information to different audiences to assist in continuously improving the quality of care. To date, many initiatives in the US have taken place to provide additional information on the quality of LTC and home health services. SimilarSeveral important issues should be addressed prior to moving towards more public reporting of these types of quality data. For example, there needs to be a discussion of the relative importance of the HCQIs. The current project made no attempt to prioritize the indicators, although clearly some would have higher clinical priority than others. To some degree, individual agencies must determine their own priorities. However, a simple system based on prevalence, severity and modifiability may be useful in determining relative importance. For example, pain is a prevalent condition in this population, can be severely debilitating for clients and is clearly modifiable. One might argue that it is of higher importance to address than declining cognitive performance, which is less prevalent and in many cases not modifiable. On the other hand, cognitive decline can have extremely serious consequences for the individualIn addition, a decision is needed regarding the calculation of the HCQIs, in particular, which pairs of assessment are to be used in the calculation of failure to improve/incidence HCQIs and what time period in between assessments is acceptable . There will need to be further analysis to explore the relationship between the length of stay and triggering on the HCQIs. It is anticipated that clients who have been on service longer will show important clinical differences from short-stay clients and would therefore be expected to have differential HCQI rates.Tracking of the HCQI rates over time will be essential to determine the level of stability in the indicators. It would be useful to measure the amount of variation and to assess methods to summarize the rates over time in order to maximize their stability. For example, it may be important to report annual HCQI rates for a given organization or province if in fact the rates are found to be highly variable over shorter periods of observation. A shorter time frame may also lead to unstable rates if the number of observations is small. In this paper, a minimum of 20 observations was required and a decision would also be needed for the minimum sample size for public reporting of HCQIs.Finally, it will be essential to assess possible methods to create summary measures across the set of HCQIs. This is not a simple task and one for which little research has been conducted to date. Previous research has pointed to the lack of correlation between measures of quality, but has also suggested that summary measures would be useful for consumers who would likely prefer less complicated information.Knowledge about the home care sector in Canada remains rudimentary, and our knowledge and understanding of quality assessment in this sector is still in its infancy. The results from the current project serve to enhance the overall understanding of the issues involved in quality assessment and our ability, through the risk adjustment process, to create fair comparisons across providers.RAI, provides an important first step in assessing quality and the variable effects of different types of risk adjustment.Risk adjustment of quality indicators is important to enable fair comparisons across geographic regions or across home care providers. To date, little research has examined the quality of home care services. This project, using a set of HCQIs developed by interThe author(s) declare that they have no competing interests.DD participated in the coordination of the study, assisted in the conceptual design of the study, carried out the data analysis and took the lead in developing the manuscript. JH designed the original study, oversaw the development of the indicators, gave input into the data analysis and development of the draft manuscript. BF was the Co-Principal Investigator in the development of the indicators and provided feedback in terms of data analysis and choice of statistical methods. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:


Diagnosing chronic heart failure is difficult, especially in mild cases or early in the course of the disease, and guidelines are not easily implemented in everyday practice. The aim of this study was to investigate general practitioners' diagnostic reasoning about patients with suspected chronic heart failure in comparison with recommendations in European guidelines.Think-aloud technique was used. Fifteen general practitioners reasoned about six case vignettes, representing authentic patients with suspected chronic heart failure. Information about each case was added successively in five steps. The general practitioners said their thoughts aloud while reasoning about the probability of the patient having chronic heart failure, and tried to decide about the diagnosis. Arguments for and against chronic heart failure were analysed and compared to recommendations in guidelines.Information about ejection fraction was the most frequent diagnostic argument, followed by information about cardiac enlargement or pulmonary congestion on chest X-ray. However, in a third of the judgement situations, no information about echocardiography was utilized in the general practitioners' diagnostic reasoning. Only three of the 15 doctors used information about a normal electrocardiography as an argument against chronic heart failure. Information about other cardio-vascular diseases was frequently used as a diagnostic argument.The clinical information was not utilized to the extent recommended in guidelines. Some implications of our study are that 1) general practitioners need more information about how to utilize echocardiography when diagnosing chronic heart failure, 2) guidelines ought to give more importance to information about other cardio-vascular diseases in the diagnostic reasoning, and 3) guidelines ought to treat the topic of diastolic heart failure in a clearer way. Chronic heart failure (CHF) is a major cause of morbidity and mortality and it has a considerable impact on the health care system . In a reThe European Society of Cardiology adopted guidelines for diagnosing CHF in 1995, and these were revised in 2001 -9. Swedia) one or more typical symptoms (at rest or during exercise), b) objective evidence of cardiac dysfunction (at rest), and c) response to treatment directed towards CHF (in cases where diagnosis is in doubt). Criteria a and b should be fulfilled in all cases. Echocardiography (ECHO) is mentioned as the single most effective tool in widespread clinical use for objective assessment of cardiac dysfunction. In the algorithm for the diagnosis of CHF, a sequence of investigations is recommended: suspect CHF because of symptoms and signs; assess presence of cardiac disease by electrocardiography, X-ray or Natriuretic peptides (where available); imaging by echocardiography; assess aetiology, degree, precipitating factors and type of cardiac dysfunction; additional diagnostic tests where appropriate; choose therapy. Table The full versions of the above-mentioned guidelines are comprehensive documents, covering epidemiology, aetiology, pathophysiology and diagnostic methods, but may be difficult to apply to specific diagnostic situations . HoweverRelatively few studies on how patients suspected of having CHF are diagnosed have been performed in primary health care settings, and most of them report over-diagnosis ,4,14-16.Process-tracing techniques are used to study the cognitive processes involved in decision-making such as, for example, how judgements change over time as new information is presented, and which decision rules are used . A methoThe value of conclusions reached in such studies depends on the validity of the think-aloud method, and on the reliability of the coding process. Thinking aloud while performing a task often lengthens the time for completing the task, but does not seem to change the accuracy of task fulfilment or the cognitive processes . In a ren = 61) were listed and contacted in a random order. The distance from central Stockholm was chosen for practical reasons. In each health care centre the GPs were contacted in a random order by one of the authors (YS). Only one GP at each centre was included in the study, and this person had to be a specialist in family medicine. We contacted the GPs during their regular telephone hour, during the period October 2001 to October 2002. Our goal was to include 15 GPs in the study.All health care centres in northern Stockholm within a distance of 20–30 km from the city centre  years, they were on average 52.7 (range 42–62) years of age, and six of them were men. The non-participating GPs were on average 52.7 (range 35–62) years of age, and seven of them were men.Six case vignettes (CV), based on authentic patients, were presented to the participants. The information presented in the case vignettes was obtained from the patient records and included information about relevant diseases , lifestyle factors , symptoms, signs, electrocardiography (ECG), chest X-ray findings, and ECHO. Chest X-ray and ECHO results were presented in the same format as in the patient records. ECHO results could contain information about ejection fraction (EF), valvular disorders and ventricular wall motility. The diagnoses made by the attending cardiologists  were used as a reference standard when assessing the participants' diagnostic accomplishments.The six cases were selected to represent patients with various types of potential diagnostic problems: A "prototypical" CHF patient (CV2), a patient with both CHF and chronic obstructive pulmonary disease (COPD) (CV6), a patient with CHF, tachycardia and mitral valve insufficiency (CV3), an obese non-CHF patient with normal ECG and EF (CV5), a non-CHF patient with COPD (CV4) and a non-CHF patient with alcohol abuse and a metabolic syndrome (CV1). For one of the cases (CV3) there was a disagreement between the diagnosis according to the cardiologists and the diagnosis that could be deduced from a simplistic interpretation of the guidelines. This patient had typical clinical findings including gallop rhythm, cardiomegaly, and pulmonary congestion, but normal left ventricular function according to ECHO. It could therefore be categorized as not CHF according to the definition given in the guidelines. However, this patient also had a mitral valve insufficiency, which can give a "false normal" ejection fraction value: the left ventricle is emptied both forward (cardiac output) and backward .Before the sessions the GPs had received written information about the aim of the study  and about the method , but not about the kind of medical problems that would be presented to them. The study was conducted at the GPs' offices. All visits and recordings were made by one of the authors (YS). The participants were instructed that six authentic patients, suspected by GPs to have CHF, would be presented, and that their task was to say aloud their thoughts about the case, and to try to decide whether the patient had CHF or not.The order of the cases was the same for all participants. The order in which the information was presented was arranged to be as realistic as possible in relation to clinical practice . Each vignette was presented on a computer screen in five successive steps using QA software . All preThe doctors first got a test case (not recorded) in order to get acquainted with the think-aloud method, and then continued with the six study cases. The only intervention from the researcher during the think-aloud session was that a participant who was silent for more than about 15 seconds was reminded to say his or her thoughts aloud about the information presented . All sesAll information in the case vignettes that was of relevance for the diagnosis and that could take on different values was considered to be variables. Fifty variables were defined: 19 of them were included in all six vignettes , six in five, one in four, six in three, one in two, and 17 in one vignette . For each case vignette, the presented variables were coded for content and value Table .2) as an argument against CHF. For each participant, a specific evaluation of each variable value was only counted once for each case vignette in order not to give more weight to thoughtful repetitions of an argument than to a single, firm statement. However, if a participant used the same variable value as an argument both for and against the diagnosis of CHF, both evaluations were coded. Ten percent of the 90 case vignette protocols were selected at random and coded independently by two of the authors  to estimate the interrater agreement of the coding process. The rest of the protocols were coded by one of the authors (YS).For each participant, every mention of a variable was coded for how the GP seemed to use it: as an argument for the diagnosis of CHF, as an argument against CHF, or as not being of any explicit use for the diagnosis (mentioned only). "He has basal rales. This guy has CHF!" is an example of a participant using the variable "rales"  as an argument for CHF. "So I'm not really sure that he has got CHF. Just a moderate cardiac enlargement, no, I wouldn't think so" is an example of a participant using the variable "relative heart volume"  in three categories: CHF or probably CHF; uncertainty about diagnosis; probably not CHF or not CHF.The participants were not forced to express their diagnostic judgements in a specific format, and their free verbal statements therefore had to be interpreted and coded. Two of the authors  independently classified all the diagnostic judgements (Stata 8.0 was used for the statistics. Cohen's kappa test (κ) was used to determine interrater agreement regarding the coding of the think-aloud protocols and the classification of the diagnostic judgements. Kappa values are classified as follows: <0, worse than chance; 0 to 0.2, poor; 0.21 to 0.4, fair; 0.41 to 0.6, moderate; 0.61 to 0.8, good; and >0.8, very good .The research ethics committee of Huddinge University Hospital approved the study.The randomly selected test protocols contained 322 segments of propositions, 36 of which were excluded since they contained variables that were not going to be investigated in this study (e.g. treatment suggestions). There was disagreement between the two coders about the content of variables in 14 of the remaining 286 segments (4.8%). The remaining 272 segments were then tested for interrater agreement on argument values , which was 95% (κ 0.85).The interrater agreement was 92% (κ 0.85). For the few diagnostic judgements where there was initial disagreement, it was possible to agree upon an interpretation.The information that was used most frequently in diagnostic arguments was the ejection fraction value on ECHO, pulmonary congestion, and cardiac volume Figure . The mosSymptoms and signs were not often used as arguments in the GPs' diagnostic reasoning Figure . SymptomAccording to the guidelines, a normal ECG opposes the diagnosis of CHF Table . Only onAccording to the guidelines, information about cardiac enlargement or pulmonary congestion on chest X-ray gives some support for CHF if there are any pathological findings, and is of some importance as an argument against CHF if the findings are normal Table . Chest XEach of the 15 GPs judged 6 case vignettes, which resulted in 90 judgement situations. In 48 of them, the EF value (or the information about left ventricular function in CV3) was used as an argument for or against CHF . In three judgement situations, the final diagnosis was in the same direction as the X-ray argument (three CHF). In seven judgement situations, the GP was uncertain about the diagnosis.In our study, the GPs used other diseases as an argument in a total of 70 judgement situations, mostly as arguments for CHF (91%). Atrial fibrillation, emphysema, history of myocardial infarction, and hypertension were the diagnoses most commonly used in this way.For certain variables, the same information value was used by some GPs as an argument for and by others as an argument against CHF. The presence of emphysema was sometimes seen as increasing the risk of CHF , or as an alternative explanation for symptoms .Diabetes could also be seen as increasing the risk of CHF , or as an alternative explanation for symptoms .Advanced age could be seen as increasing the probability of CHF , or as an alternative explanation for symptoms . Age was used as a diagnostic argument only for the two patients over 80 years of age.For relative cardiac volume, the reasoning could be compatible with GPs using different threshold values in their reasoning. The two lowest values were only used as arguments against CHF, the two highest values only as arguments for it, and the two intermediate values were used as arguments in both directions.There was total agreement among the GPs only for the prototypical CHF case; otherwise there was a large variation among GPs regarding diagnoses. Case vignettes representing CHF patients were more likely to be correctly diagnosed than those representing non-CHF patients Table .When comparing the GPs' diagnostic reasoning with guidelines, we found that the clinical information in the case vignettes was not used to the extent recommended in the guidelines. It is true that information about the ejection fraction value on ECHO was the single most frequent diagnostic argument, and it was the most common argument against CHF. This is in line with the guidelines, which emphasize the need for objective evidence of cardiac dysfunction. However, in more than one third of the judgement situations, the information about ECHO that was presented was not used as an argument. Over-diagnosis of CHF in primary health care has been demonstrated in a number of studies, with ECHO findings as the gold standard ,4,15. LiIn the diagnostic algorithm, symptoms and signs are the entry criteria. However, the GPs did not seem to use them consistently in this way, except when diagnosing the prototypical CHF and non-CHF cases. One reason for this might be that most symptoms and signs considered typical for CHF are fairly non-specific as regards the diagnosis CHF.Information about other relevant diseases, which was important in the GPs' diagnostic reasoning, is not included in the list of assessments to be performed routinely Table  9]. How. How9]. The case vignettes represented authentic patients referred by GPs to a cardiology department for problems related to heart failure. This may have led to a selection of more complicated patients than the "typical" heart failure patients in primary health care. The reason we chose this group of patients was that we wanted to include patients who were thoroughly investigated, with a well-founded clinical diagnosis, and for whom information about all variables of interest could be found in the patient records. Selecting GPs only from health care centres in, or relatively close to, the city centre may have biased the results, since differences in catchment areas, working conditions, and access to echocardiography may influence GPs' diagnostic habits. This could make it difficult to generalize the results to other GPs. Only 50% of the GPs who were contacted agreed to take part in the study, which could bias the results. However, since the age distribution was the same in the two groups, it seems unlikely that the drop-out group would differ from the study group regarding clinical experience.The full version of the guidelines is difficult to apply to individual diagnostic situations and it is also difficult to use it for assessment of diagnostic behaviour . In thisGPs' tendency to over-diagnose CHF has been explained by their relying on symptoms, signs and less specific investigations such as chest X-ray, and by limited access to ECHO in the primary health care. However, this study indicates that a substantial minority of GPs seem to be less familiar with the use of ECHO and EF. Thus, access to ECHO ought to be accompanied by education about how to integrate this information better in the diagnostic reasoning.Guidelines ought to include search of information about other cardio-vascular diseases in the list of assessments to be performed routinely Table  and in tThe information in the case vignettes was underused as arguments for and against the possibility of CHF as compared with the guidelines. Information about the EF value was the single most frequently used argument for or against CHF; nevertheless, in one third of the diagnostic judgements the GPs did not consider any information about the ECHO in their diagnostic reasoning. Information about symptoms and signs were not used to to the extent suggested in the guidelines. Information about other relevant diseases was frequently used in the GPs' diagnostic reasoning, indicating that they often relied on illness scripts. Some implications of our study are that 1) GPs should be taught how to use ECHO information better in their diagnostic reasoning, 2) guidelines ought to give more importance to information about other cardio-vascular diseases in the diagnostic reasoning, and 3) guidelines ought to treat the topic of diastolic heart failure in a clearer way.CHF Chronic heart failureCOPD Chronic obstructive pulmonary diseaseECHO EchocardiographyECG ElectrocardiographyEF Ejection fractionGP General practitionerThe author(s) declare that they have no competing interests.All authors conceived of the study and participated in the design. YS carried out the data collection, performed the statistical analyses and drafted the manuscript. All authors participated in the interpretation of the results and the discussions of the drafts. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:Case vignette characteristics The additional file shows some important characteristics of the case vignettes. Format: Word-table.Click here for file


Current approaches for representing pathways are limited in their capacity to model molecular interactions in their spatial and temporal context. Moreover, the critical knowledge of cause-effect relationships among signaling events is not reflected by most conventional approaches for manipulating pathways.Substantial amounts of data on cell signaling, metabolic, gene regulatory and other biological pathways have been accumulated in literature and electronic databases. Conventionally, this information is stored in the form of pathway diagrams and can be characterized as highly "compartmental" (We have applied a semantic network (SN) approach to develop and implement a model for cell signaling pathways. The semantic model has mapped biological concepts to a set of semantic agents and relationships, and characterized cell signaling events and their participants in the hierarchical and spatial context. In particular, the available information on the behaviors and interactions of the PI3K enzyme family has been integrated into the SN environment and a cell signaling network in human macrophages has been constructed. A SN-application has been developed to manipulate the locations and the states of molecules and to observe their actions under different biological scenarios. The approach allowed qualitative simulation of cell signaling events involving PI3Ks and identified pathways of molecular interactions that led to known cellular responses as well as other potential responses during bacterial invasions in macrophages.We concluded from our results that the semantic network is an effective method to model cell signaling pathways. The semantic model allows proper representation and integration of information on biological structures and their interactions at different levels. The reconstruction of the cell signaling network in the macrophage allowed detailed investigation of connections among various essential molecules and reflected the cause-effect relationships among signaling events. The simulation demonstrated the dynamics of the semantic network, where a change of states on a molecule can alter its function and potentially cause a chain-reaction effect in the system. Interactions among genes, gene products and small molecules regulate all cellular processes involving cell survival, cell proliferation, and cell differentiation among others. Such interactions are organized into complex lattice structures conventionally divided into cell signaling, metabolic and gene regulatory networks in a cell . In receConventionally, this information is highly compartmental: various individual signaling pathways are mostly stored in separated and non-linked diagrams. Traditional pathway diagrams, where molecules are represented as nodes and their interactions are depicted as lines and arrows have significant limitations as they lack spatial and temporal context . MoreoveRecently, many models have been proposed for representing, storing and retrieving interactions among various biological entities. BIND  and IntARecently an artificial intelligence approach known as semantic networks (SN) have gained the attention of the biological community as a potentially powerful tool for organizing and integrating large amounts of biological information . For insA semantic network is a method to represent information or knowledge by nodes and edges in a graphic form, where a node represents a concept and an edge represents a relationship . A semanPreviously an application development environment known as Visual Knowledge (VK) has been created, and VK is capable of different formalizations and implementations of semantic networks for various knowledge domains . Visual Mycobacterium tuberculosis [M. tuberculosis and the formation of a phagosome within the macrophage. These processes are mediated by the class I phosphoinositide 3-kinase (PI3K) – one of the most well-characterized enzymes to date [M. tuberculosis can inhibit the activity of the class III PI3K, arresting phagosome maturation process, and ensuring the survival of M. tuberculosis inside the macrophage [It is the current knowledge that many pathogenic bacteria are capable of entering and surviving within mammalian macrophages by modulating the host signaling pathways . One welrculosis . Activat to date . The cla to date . The p11 to date . Activat to date . The for to date . Howevercrophage ,32.In addition to their role in phagocytosis, PI3Ks are essential proteins that regulate cell survival, cell growth, cell cycle and other cellular processes . AlthougIn the paper, the word "model" refers to a set of rules in two different but related contexts. In the context of the semantic network, the model refers to a set of rules that specify how a biological concept is mapped to one or multiple semantic agents/relationships. In the context of cell signaling pathways, the model is a set of rules that specify what, how, and when molecules interact with each other. The model has been formalized and implemented, using BioCAD software system, and it is presented in the following sections.Within the semantic network, all biological structures are modeled by semantic agents that are members in one of the 6 different prototypes. Table To create an adequate semantic model, we have postulated that structures of different levels in the cellular hierarchy can interact with one another. One example of such interactions is the movement of a molecule from one subcellular compartment (e.g. cytosol) to another (e.g. plasma membrane). This is referred to as a translocation event, and it is demonstrated on the left panel of Figure Table Interactions that occur by non-covalent or covalent forces have also been modeled as two distinct "event" prototypes as shown in Table All hypothetical spatial changes occurring in the three-dimensional structure of a given macromolecule have been modeled within the SN as switches in the corresponding conformational states, and the changes do not lead to the creation of new semantic agents. Domains or sites for every protein encoded into the SN model have been assigned to either "Functional" or "Non-functional" conformational states. The "Functional" state represents that a domain/site is currently in a conformation that enables a certain interaction. On the other hand, a "Non-functional" state implies a domain/site is in a conformation that prevents an interaction. To illustrate this construct we have graphed the semantic agents and their relationships created within the developed SN. It should be noted that within the SN, all semantic agents are visualized as icons, and their relationships are depicted as connecting arrows. In addition, all agents are related by pairs of reciprocal relationships, and for simplicity, only one direction of each pair of the relationships was visualized. The left panel of Figure In addition to the conformational states, a protein domain or site has been assigned one of the two binding states: "Bound" or "Not-bound". A "Bound" state implies that this domain/site currently associates with a domain/site of another molecule through a non-covalent interaction. On the other hand, a "Not Bound" state indicates such an association does not exist. Since ligand bindings can affect the conformation of a macromolecule through allosteric regulations, two types of such regulations have been implemented within the SN. A positive allosteric regulation event has been assigned to the scenario when a "Bound" binding state of a domain/site causes the conformational state of another domain/site to switch to "Functional". The right panel of Figure A more complex allosteric regulation event can be specified for mapping the binding states of multiple domains/sites (the condition or the input) to the conformational states of multiple domains/sites (the response or the output). Hence, a domain is switched to "functional" only if a certain combination of ligand bindings has occurred. The utilization of the states on domains/sites and allosteric regulation events in the SN has enabled us to express the cause-effect relationships among various signaling events explicitly.In the developed semantic model, any molecular complex formed due to non-covalent interaction has been treated as a transient state of these two molecules, and a complex was not represented by a new semantic agent. Instead, the existence of a protein complex is inferred from the non-covalent interaction event. Thus, Figure Conventionally, there is often some inconsistency between representing chemical modifications of small molecules in metabolic pathways and modifications of proteins in signaling pathways. In the developed model, two molecules that interacted by covalent forces have resulted the creation of distinct semantic agents within the SN. This rule has been implemented consistently for both macromolecules such as proteins and small molecules such as ATP. As one example, Figure In the semantic model, a molecule can participate in different sets of interactions in different locations. The translocation events define all possible localizations of molecules, and therefore, an interaction can only occur if the participating molecules can be present in the same location at the same time. Alternatively, an interaction  can directly associate with a subcellular compartment, and this interaction is only available to molecules in that location.In addition, all qualitative cellular responses such as cell survival and phagosome formation have been implemented within the SN under a distinct "event" prototype. They have been implemented in a way that the formation or the activation of certain signaling molecules such as PIP3 can trigger the occurrence of these cellular response events in a simulation.As it has been mentioned previously, the behavior of any semantic agent can be clearly defined by its relationships or connections to other agents. Thus, the formalization of the five types of events, which are translocations, non-covalent interactions, covalent interactions, allosteric regulations and cellular responses, has enabled us to model the behaviors of molecules depicted in conventional pathways and to reconstruct a cell signaling network of the human macrophage.The molecular composition of human macrophages and information of known intracellular interactions have been extracted from various research articles ,32,34-47A pathway diagram in the literature or an electronic database, in principle, represents some scenario of what may happen in a call if every depicted molecule is expressed in the correct location, at the correct time and with the correct states. Hence, the aggregation of multiple pathway diagrams describes some, if not all, possible molecular events that can potentially occur in a cell under the right conditions. To utilize such information and build a cell signaling network, we have decomposed conventional pathways into individual pieces of information such as subcellular localization of a protein, a pairwise protein binding, a chemical reaction or a cellular response. Then, using the sets of semantic rules described in the model, we have represented and integrated each piece of those information in the form of semantic agents and their relationships. Table The defined semantic agents have been connected in the semantic network and can be visualized at different levels. Figure Such detailed semantic reconstruction of the cell signaling network has allowed thorough investigation of biochemical relationships between essential proteins. One such example is presented on Figure Another example of successful SN reconstruction is the relationship between CD14 macrophage receptor and the class I PI3K; such a relationship was previously suspected but not clear . By incoIn the implemented semantic model, the "possible" behaviors of a molecule are defined through its relationships to other agents , and all instances of that prototypical molecule will inherit the same behaviors. However, the action of a molecule at any given time is affected by factors including its current states and its current location with respect to other molecules in the system. Hence, we have built an application that enabled us to produce instances of molecules in various locations and to observe the "current" action of a molecule qualitatively under different biological scenarios. We refer such scenario-play as simulation in this paper.The application or the SN-simulator allows the molecules to move among various locations, to interact with each other and to create events when the conditions are met. In addition, every instance of a molecule has been represented as an individual agent while every instance of a molecular interaction has also been implemented as an individual event agent. Thus, the simulator provides a traceable "trajectory" of all the events that have happened on every molecule during a simulation.As illustrated in Figure Because an Fcγ receptor was the only interacting molecule (for the IgG) present at plasma membrane in the simulation, it has bound to the IgG by a non-covalent interaction event, as illustrated in Figure M. tuberculosis was initially present in the plasma membrane, it would inhibit the class III PI3K and thus arrest the phagosome maturation response in the macrophage. Table M. tuberculosis has caused several known cellular responses as well as additional responses such as cell survival of the macrophage, cell cycle entry, increase of protein synthesis and increase of intracellular glucose level in the simulation. We suspect that some of these responses have not yet be appreciated in previous studies of bacterial invasions. Further experiments can be formalized to test the simulation results.Figure In this study, the SN-simulator has enabled us to "play" different scenarios and observed their effects in the macrophage. It allowed us to investigate how changes on one molecule caused changes of another molecule in the cell signaling network during bacterial invasions.In the present work we have developed a semantic model to represent the properties and the behaviors of molecules and their interactions in the context of cell signaling pathways. The proposed model offers some additional features, compared to other existing pathway models. Those features are essential for characterizing the complex behaviors of biological entities, and they include:The semantic model has specified the hierarchical relationships among the different biological structures, from cells to compartments, molecules and domains/sites. The hierarchy between subcellular compartments and molecules has allowed us to specify the spatial organization of molecules, model the translocation events and represent the effects of locations on the different interactions among molecules.The hierarchy between molecules and their domains/sites has enabled us to explicitly model the relationship between forms and functions for proteins. Through the allosteric regulation events, proteins have been modeled and implemented as integrating and logical devices in the semantic network, and their conformational states (outputs) are switched by the combination of non-covalent ligand bindings or covalent modifications (inputs).Through the prototyping system in the semantic network, any rule or interaction specified on a prototypical molecule automatically define the properties and behaviors of all its instances. As demonstrated by the simulator, the semantic network provided a direct communication from the interaction model to an application where the actions of molecules can be observed under different scenarios. Therefore, the semantic network is dynamic as a change of states on a molecule can alter its function and potentially cause a chain-reaction effect in the system.In addition, the current semantic model is different from the previous models in BioCAD. An essential difference is the representation of functional labels or roles on proteins. The meanings of functional descriptions or association words such as "enzyme", "activator/activates" or "inhibitor/inhibits", which are often used to characterize the behaviors of proteins in most pathway models, have been represented explicitly through events and relationships in the developed semantic network. For example, a protein acts as an "enzyme" if 1) the protein participates in a "covalent interaction event", 2) the presence of a "functional" catalytic domain on the protein is required for the occurrence of the event, and 3) the protein itself is not modified after the event. Similarly, a protein A "activates" a protein B if a non-covalent binding event from protein A turns on the "functional" state of a domain/site on protein B. Hence, the model has reduced the need for labels, which are often confusing or misleading on conventional pathway representation.The use of non-covalent and covalent events has enabled us to model protein-protein interactions and chemical modifications on molecules including proteins and metabolites. The next challenge is to model the complex interactions that govern gene regulations. The current construction of non-covalent interaction events can model the binding of an individual transcription factor to a particular site of a gene, and the covalent interaction event can represent the transcription process that leads to the production of an mRNA, and the translation process that produces a protein. However, a successful transcription in a eukaryotic cell requires the formation of a protein complex that involves more than one hundred subunits, and the complex may be assembled in various orders . We antiThe semantic network representation can be exploited for performing analysis of cell signaling pathways. The examples of Fcγ receptor, CR and the class I PI3K demonstrated that connections can be queried and analyzed among different biological entities. The semantic model is also compatible with other pathway models. Therefore, the number of biological entities and interactions in the semantic network can be greatly increased as pathway data from existing databases is integrated. Previous study has shown the value of combining gene expression profiles with protein-protein interaction networks for identifying active subnetworks . SimilarIn this study, the proposed semantic model has been applied to cell signaling pathways in the macrophage as a case study. The model is not limited to those pathways. The hierarchical classification of the biological structures and the events can model other cell signaling pathways for different cells and organisms. An interactive website is currently under development. We anticipate that through the web, researchers can utilize the semantic network approach for creating pathways in cells of their interest and for analyzing any existing pathways including the PI3K pathways of the human macrophage presented in the paper.In this study, we have developed a simple simulator to demonstrate the dynamics of the semantic network and to observe the actions of molecules qualitatively. In order to perform a realistic cellular simulation in the future, three components need to be improved. First, quantitative factors should be integrated into the model. For example binding affinity, directly associated with non-covalent events, will affect the probability and the duration of the binding of molecules. Reaction kinetics, associated with covalent events, will determine the rate of production.Second, the two parameters, the population of molecules and their localization, which influence the simulation outcome, could be initialized and supported by experimental data. For instance, gene expression data from microarrays supports the relative abundance of transcripts, and protein expression data supports the relative abundance of proteins. Computer algorithms such as PSORT  can assiThird, the proximity of molecules has been represented by subcellular compartments in the simulation. This approximation can be improved in two different ways. First, a compartment can be further divided into smaller sub-locations. Increasing the number of locations and reducing the size of each location will improve the accuracy of the simulation. Second, the occurrence of non-covalent events in the simulation has allowed us to identify molecular complexes and their members effectively. Hence, the proximity can be approximated through molecular complexes, such that molecules in a complex have higher probability to interact with members of the same complex.The simulator has demonstrated that a biological pathway can emerge from the creation of semantic agents and their relationships in the SN, and such a pathway represents a series of consecutive events resulting from the activation of a single molecule. It is anticipated that further development will improve our ability to track and visualize different instances of molecules participated in multiple pathways. Hence, the occurrence of a cellular response event can be triggered by the accumulation of certain molecular species with particular states.M. tuberculosis in the macrophage were investigated by the simulator. As a result, the simulation identified pathways of molecular interactions that led to known cellular responses as well as other potential responses during bacterial invasions.We concluded from our results that the semantic network is an effective method to model cell signaling pathways. Utilizing the semantic agents and the relationships in the model, information on biological structures and their interactions at different levels has been properly represented and integrated in the hierarchical and spatial context. The reconstruction of the cell signaling network in the macrophage has allowed qualitative investigation of connections among various essential molecules and reflected the cause-effect relationships among the events. The simulation demonstrated the dynamics of the semantic network, where actions of molecules are affected by their current states and locations, and the history of events can be traced and analyzed. In addition, changes caused by the invading Visual Knowledge (VK) is an application development environment, and its implementation has been influenced by the theory of semantic networks as well as other approaches including set theory, frame system, object-oriented modeling theory and systems based on networks of active software agents . DiffereBioCAD, a Visual Knowledge-based development environment, is developed by Upstream Biosciences, Inc. and customized to model biological systems . The BioThe semantic model was developed jointly by all authors and implemented by MH, JLB and CS. MH implemented the simulation, collected and analyzed data, constructed pathways in the macrophage and drafted the manuscript. JLB, CS, AC developed general concepts, provided scientific support, participated in the manuscript writing and coordinated the study. All authors read and approved the final version of the manuscript.


We describe a system of web applications designed to streamline the interdisciplinary collaboration in outcomes research.The outcomes research process can be described as a set of three interrelated phases: design and selection of data sources, analysis, and output. Each of these phases has inherent challenges that can be addressed by a group of five web applications developed by our group. QuestForm allows for the formulation of relevant and well-structured outcomes research questions; Research Manager facilitates the project management and electronic file exchange among researchers; Analysis Charts facilitate the communication of complex statistical techniques to clinicians with varying previous levels of statistical knowledge; Literature Matrices improve the efficiency of literature reviews. An outcomes research question is used to illustrate the use of the system.The system presents an alternative to streamline the interdisciplinary collaboration of clinicians, statisticians, programmers, and graduate students. In the last decade, the number of relevant data sources available for outcomes research has grown exponentially. In contrast, the number of individual researchers with clinical and statistical expertise required to explore these data sets increase at a much slower pace. As a result, an immense quantity of valuable clinical data are left untouched, never becoming clinical publications that could potentially improve health care.The disproportion between data volume and number of qualified researchers can be explained by the growing complexity involved in outcomes research projects using secondary data analyses. Researchers have to formulate of a clinically relevant and methodologically sound research question, find appropriate data sources, perform statistical analyses, and generate a final manuscript that will be submitted for peer-review. Frequently, individual researchers have the training and time to perform a few of these steps, but the integration of all tasks calls for an interdisciplinary systems approach ,2. This This article describes a suite of web applications developed to facilitate the process of converting outcome databases into clinical manuscripts, to streamline the interdisciplinary collaboration of researchers, and to connect all different steps of the outcomes research process. To illustrate its use, we will describe how a research project has been conducted using this system from its early phase of research question formulation to the completion of the final manuscript.. The following sections will describe each of the Web applications and their application in the answer of a real outcomes research question.The system of Web applications is composed by five different tools: QuestForm, Research Manager, Analysis Charts, and Literature Matrices. These tools were designed to assist researchers in each of the phases encountered in an outcomes research project involving secondary data analysis Figure . All too for an updated list). All databases have been de-identified and do not contain protected health information as specified by the Health Insurance Portability and Accountability Act . The application The application is built using Extensible HyperText Markup Language 1.0 (XHTML) [QuestForm, an acronym for "Question Formulation", is an application designed to assist researchers in the location of clinical databases and formulation of outcome research questions Figure . Clinica (XHTML) .QuestForm starts by presenting researchers with three main strategies to find research databases: use of pre-determined key words that describe the database as a unit Figure , user-deDr. Guller initiated the project searching for an existing database that would allow him to compare surgical outcomes between laparoscopic and open appendectomy procedures in the treatment of acute appendicitis. The outcomes were pre-specified as mortality and infection, although other existing outcomes would also be of interest. Although there are multiple single-institution studies attempting to answer this question ,7, few sAs a first step, Dr. Guller searched across previously formulated Question Diagrams to evaluate whether other studies could have used a similar research design. Since none was found in QuestForm, Dr. Guller searched across more than forty different databases for an existing database that would have the variables to answer his research question. After navigating through multiple data dictionaries, Dr. Guller found that the Nationwide Inpatient Sample (NIS) Release 6, 1997  presenteOnce the database was located, Dr. Guller selected the outcomes of interest , main predictors (laparoscopic versus open procedures), and confounders , inserting each of them into the research question fields in QuestForm. Using built-in search engines for ICD9 codes, Dr. Guller created the definition for each of the above-mentioned variables and defined the inclusion and exclusion criteria. The final research question was then saved as a Question Diagram and immediately submitted to Dr. Pietrobon for feasibility evaluation. Dr. Pietrobon judged that the project was feasible and could be completed using the database indicated by Dr. Guller. At this point, the project was initiated and a detailed project management plan was established using Research Manager.Research Manager is a Web application developed by our group designed to facilitate the project management of clinical research projects. Similar to QuestForm, Research Manager is licensed under the GNU Public License , which aResearch Manager provides multiple features to facilitate project management of clinical research projects. All projects are displayed by category  with a brief description. The internal content of all projects is password protected. All internal tasks within a project are assigned to individual researchers. Project administrators initially assign deadlines that can be modified by task leaders within three days. All participating members of the project receive weekly reports containing details about the activity and the latest electronic file within each task Figure . These fResearch Manager helps identifying such problems and enables their early elimination, thus avoiding delays in the project completion. Finally, since weekly reports are generated to all participating members, Research Manager also provides peer-incentive for project members to complete their tasks in a timely manner.Once the Question Diagram was evaluated by the clinical epidemiologist, this project was transferred to Research Manager. Contact information for each of the researchers involved and deadlines for completion of the main phases of the research process were set, including data extraction and cleaning, data analysis, literature review, and manuscript writing. Weekly reports were generated to update investigators on all tasks of the project, including different versions of the statistical analysis, modifications in the research question, and manuscript sections.With an established project plan, the statistician in charge selected the best methods for analysis. Since the database is a random sample of the United States and requires special survey analysis methods, it was necessary that all involved researchers understood the statistical approach by using Analysis Charts.Analysis Chart is a tool designed to enhance the understanding of statistical methods to a format that is understandable by clinical researchers with different previous levels of statistical knowledge. As such, it is important in the design as well as the analysis phases of a project Figure . The applayers of information. Layer 1 summarizes the general goal of the method. Layer 2 presents previous clinical applications so that the researcher can visualize situations in which the method may be realistically applied. Layer 3 describes the data requirements for the application of the statistical technique. Layer 4 describes the basic statistical underpinnings of the method, initially breaking down equations and then reassembling them. Finally, layer 5 presents a list of available software packages for the implementation of the technique as well as cases studies where all previous layers are applied to real data sets. Each layer ends with a section containing selected references that explain the topic in more detail.Analysis charts are composed by cascading links that display information about quantitative methods in progressive levels of complexity called "layers of information". Each layer explains the statistical method with an increasing level of complexity. In this manner, clinicians interested in simply understanding the method to evaluate whether it can be applied to a research project can simply read the first three layers. In contrast, researchers interested in a direct application of the method to available research data can follow all layers and their respective references. Most commonly, complex techniques are presented using five While deciding on the most appropriate analysis strategy for the Question Diagram, Drs. Pietrobon and Guller consulted the Analysis Chart searching for the most appropriate statistical methods of analysis. Given the nature of the research question and that the NIS database has a sample design, Drs. Guller and Pietrobon opted for an approach involving multiple and logistic regression models while adjusting for sampling weights, strata, and clusters. With a defined analysis protocol, the research question was then transferred to a statistical programmer trained in the translation of Question Diagrams into statistical code. This process was closely evaluated by Drs. Pietrobon and Guller, who scrutinized the statistical code and results from a clinical and statistical perspectives. Once the results were deemed to be accurate, Dr. Pietrobon started the literature review using a Literature Matrix.® or Open Office Calc [Literature matrices consist of a comprehensive but not necessarily exhaustive review of the literature focused on a narrow clinical topic Figure . Each arice Calc .The advantage of the Literature Matrix as implemented in the Web suite is its availability over the web to the whole outcomes research community. This allows Literature Matrices to be constantly updated, with authors receiving their due credit in a list of contributors. Literature Matrices also enable researchers to obtain a complete summary of the literature without going through the cumbersome process of copying and reading a manuscript for the first time. In contrast, researchers' time can be spent more efficiently in reviewing what has already been compiled and attempting to expand the Literature Matrix with other relevant bibliographic references.In order to evaluate the literature, a graduate student performed a thorough literature search. All relevant articles were copied, read, and the data extracted according to the established categories. Once the Literature Matrix had been completed, Drs. Guller compared the current project results with results published in the literature. The structured information in Literature Matrices also allowed Dr. Guller to compare the strengths and weaknesses of the current project in relation to previous publications. Once this phase was completed, Dr. Guller proceeded to the final writing of the manuscript using Output Templates.At the end of the project, Dr. Guller combined the Question Diagram, Analysis Chart, and Literature Matrices to write the final manuscript. While Analysis Charts provided the information concerning the statistical techniques used in this study, Literature Matrices provided the basis for comparison of the study results against previous publications. Although not included in the final manuscript, Dr. Guller also had access to multiple analysis files through Research Manager to orient him in each of the steps taken during the research question formulation, data analysis, and literature review.The use of web applications by individual clinical researchers can be summarized in Figure Since the concept of streamlining the interdisciplinary collaboration preceded the existence of the current suite of web applications, different versions of the central idea have been gradually applied since the second half of 2002. Although our usability, qualitative, and economic studies to evaluate this application are still ongoing, we have noticed a significant improvement in the number and quality of our publications as evidenced by the increasing acceptance rates of our manuscripts for publication in peer-reviewed journals. The Web applications are currently used in research projects involving Duke University and other universities in the United States and abroad, where they have been shown to facilitate inter-institutional collaborations.Improving the efficiency of an interdisciplinary approach to secondary data analyses has multiple potential benefits. These include an increase in the overall clinical significance of the final publication, decrease in the number of failed projects, decrease in the time for completion of individual projects, improvement in the education of future outcomes researchers, decrease in the cost-benefit ratio for individual outcomes research projects, and, perhaps most importantly, an automation of repetitive tasks. This last factor is crucial since eliminating repetitive tasks will allow researchers to concentrate on the design of innovative projects .Surprisingly, few systems and applications have been described to solve the problem of complex interdisciplinary collaboration between clinicians and statisticians. Isolated approaches have usually focused on specific portions of the outcomes research process, without attempting to integrate them into a cohesive system. For example, Marshall  has propAlthough this newly developed system of applications provides a significant improvement in the way secondary data analyses are conducted, it still has limitations. First, because of the lack of a formal evaluation of the effectiveness of this system, we are unable to quantify its real time and cost saving benefits. Second, the system is currently restricted to secondary analyses and does not allow for the planning of prospective data collection. Although one of the main advantages of formulating a research question based on existing data sets is the bounded nature of the process, future applications should attempt to create rules and algorithms that may guide prospective data collection.In summary, we have experienced that this system has significant advantages over the traditional manner of conducting outcomes research based on secondary data analyses. This tool may have important applications, not only resulting in an improvement in the overall efficiency of the outcomes research process, but also affecting the way new outcomes researchers are trained and introduced to a research environment.The Web application is available at XHTML Extensible HyperText Markup Language 1.0JAVA: by Sun MicrosystemsICD9-CM: International Classification of Diseases, Ninth Revision – Clinical ModificationNIS: Nationwide Inpatient SampleGNU: GNU's Not linux General Public LicenseWebMIRS: Web-based Medical Information Retrieval SystemNHANES: National Health and Examination SurveyUR: Uniform ResourceThe author(s) declare that they have no competing interestsRicardo Pietrobon – design, manuscript draftingUlrich Guller – design, manuscript revision for important intellectual contentHenrique Martins – design, manuscript revision for important intellectual content, software programmingAndreia P Menezes – design, manuscript revision for important intellectual contentLaurence D. Higgins – design, manuscript revision for important intellectual contentDanny O. Jacobs – design, manuscript revision for important intellectual contentThe pre-publication history for this paper can be accessed here:


Science published draft genome sequences for the two major subspecies of cultivated rice, Oryza sativa. The release of the rice genome—the first plant crop to be sequenced—was big news. Rice is a staple crop for more than half of the world's population, and it was hoped that the availability of the genome sequence might enable scientists to develop more productive rice strains or strains that are more environmentally friendly. Furthermore, the rice genome might provide the key to understanding the genetics of other major cereal crops, all of which have much larger genomes.In April 2002, indica and japonica subspecies of O. sativa, which are eaten in India and China, and Japan, respectively. Their analysis team, led by Gane Ka-Shu Wong, provides important insights into the evolution of rice.But the sequences published in 2002 were only draft genomes, containing many gaps and errors—works-in-progress rather than finished products. Now, a large group of scientists led by the Beijing Institute of Genomics is publishing a much improved, near-complete genome analysis of the indica by generating significantly more sequence data, and then they used better methods to assemble these data. In whole-genome shotgun sequencing, the entire genome is chopped into random fragments, each fragment is sequenced, and then powerful computer programs search for overlaps and put all these data in order. It's like putting a fiendishly difficult jigsaw puzzle together by looking for patches of matching color.First of all, the team improved their original whole-genome shotgun sequencing of indica or the japonica dataset, on one piece of DNA whose position along the chromosomes is well defined.The key to the improvement in the genome sequence analysis is that the researchers have used the combined DNA sequence data from the two subspecies to facilitate the sequence assembly. The result is a nearly 1,000-fold increase in contiguity for the two genome sequences. In other words, while the original draft was very fragmented, in the new version, 97.7% of the genes can be found, in either the The researchers have also used their improved genome sequence to investigate the evolutionary history of rice. Central to evolution is the development of new functions through mutation of existing genes. But when mutations occur in functional genes, the result is rarely beneficial, so it is thought that evolution is more likely to proceed first by duplicating existing genes and then experimenting on the “backup” copy of the gene. Wong and colleagues report that there is evidence in the rice DNA sequences for a whole-genome duplication event just before the grasses diverged from other flowering plants, about 55–70 million years ago. This genome duplication may have played a role in the origin of the grasses, which then spread rapidly across the world to provide important sources of food that, among other things, possibly influenced human evolution.Analysis of the rice genomes also indicates that a small chromosomal segment was duplicated about 21 million years ago and that there is massive ongoing duplication of individual genes. These individual gene duplications provide a continuous source of raw material for gene genesis and very likely contribute to the differences between members of the grass family. Now the challenge is to use the rice sequences as a basis for detailed genetic analyses of additional cereal crops and for the development of improved strains of not only rice, but wheat, maize, and other important food crops.


NCEAS -- an ecological synthesis center -- is changing the way ecological research is conducted by fostering new forms of collaboration and interdisciplinary research A substantial portion of research occurs in places where scholars congregate—in campus laboratories, in libraries, or in large specialized facilities, such as oceanographic ships, astronomical observatories, or accelerators. Under any of these circumstances, researchers can interact easily, exchanging ideas and information.Ecological research, especially the large component occurring in the field, takes place in disparate locations all over the world—in the air, on the surface, and below the oceans, lakes, and crust of the planet. Although there are many biological field stations where scientists and students gather, much of the research in ecology takes place in isolation. In addition to being highly dispersed geographically, ecology encompasses a disparate range of disciplines at scales from molecules to the globe, making the exchange of ideas and information even more difficult.Recognizing the benefits of interaction and collaboration, the ecological community began considering a synthesis center where researchers from many fields could meet to address important ecological questions. Several organizations held discussions about the nature of such a center, which culminated in two workshops hosted by the National Science Foundation (NSF) in the early 1990s, to set the scope. In 1994, NSF conducted an open competition for a synthesis center, eventually granting the University of California, Santa Barbara, the award for the National Center for Ecological Analysis and Synthesis (NCEAS). In addition to funding from NSF, the center is supported by the University of California system and its Santa Barbara campus and by several foundations.The center employs various types of research approaches. A primary approach is through Working Groups  where scNCEAS also supports up to six visiting Center Fellows  each year. These scientists often integrate their own research into a broader context or synthesize what is known about certain areas in ecology. Concurrently, the Center houses 15–18 Postdoctoral Associates for one to three years each. These postdoctoral positions are distinctive in that there are no permanent mentors for the larval scientists—rather, they interact with the other Associates, the resident Fellows, and the hundreds of individuals who annually visit the Center as part of Working Groups.The Center also conducts training activities, including a distributed graduate seminar program. In this approach, graduate students around the world become involved in seminars on specific topics using data from their region  and then representatives from each of the seminars are brought to NCEAS for a grand synthesis.As would be expected for a discipline as broad as ecology, the participants at NCEAS are extremely diverse. Over 3,000 individuals have visited NCEAS in just over eight years, representing 43 countries and 49 states in the United States. They come from over 800 institutions, many non-biology departments, and 397 non-academic organizations . An interesting measure of their breadth is that participants belong to more than 180 professional societies.Proposals are solicited twice a year and reviewed by a Science Advisory Board. The Board looks for topics that would benefit from synthesis and analysis and that would make significant contributions to our understanding of ecological relationships. While many proposals pertain to core ecological questions, others deal with economics or sociology . Approximately 40% of the projects have some applied component, many influencing resource management practices and conservation policies.http://www.ecoinformatics.org) have embarked on a major research program in developing tools to characterize data and make them available in standardized formats. The initial research effort, called the Knowledge Network for BioComplexity (KNB) is yielding tools to generate metadata (precise information about the data) and to make all the data available. The current research thrust, called Science Environment for Ecological Knowledge, will expand the capabilities of KNB by employing grid technology , semantic mediation, knowledge representation, and workflow models for analysis and synthesis.Because the Center is based on the use of existing data, access to highly dispersed and profoundly heterogeneous ecological information is essential, but also very difficult. Recognizing the need for open access to a wide variety of data—versus project-specific data solutions—NCEAS and several collaborators (see http://www.nceas.ucsb.edu). Furthermore, some of the projects have had direct influence on conservation and resource management. For example, scientists at NCEAS developed theories for the design of marine reserves that were soon thereafter applied to the placement of reserves directly off the coast of Santa Barbara.The Center has supported almost 200 projects, the results of which are published in top scientific journals (see project results at In addition to scientific results, NCEAS is changing the way we conduct ecological research through novel means of encouraging productive collaborations. Sociologists studying the NCEAS model of collaboration have identified several important factors in its success. These include a distant, neutral location facilitating periodic, highly focused opportunities to concentrate on the issues under consideration; logistic support that lowers the activation energy required to develop collaborations; and the proximity of scientists from many disciplines having the opportunity to interact in ways otherwise not possible.Many significant contributions to our understanding of the patterns and processes of the natural world have emerged from NCEAS research activities. In addition, the Center is fostering new forms of collaboration and interdisciplinary research by providing a place where scientists from many disciplines can productively interact and by working to make eclectic ecological data available to many users. This is an extremely simple model for the scientific enterprise—but one not captured in most existing institutional and organizational structures. As recognition of the success of NCEAS spreads, other institutions are attempting to incorporate some of the traits of the Center into their operations, and new centers are being proposed. For example, the NSF is in the midst of a review of proposals for an evolution synthesis center.It is clear that the complexity of ecological systems, as well as the importance of understanding and maintaining them, requires information and knowledge from many disciplines. This is true even at a time when disciplines are becoming more specialized and scientists have less time to concentrate on broader issues. By facilitating interactions among many scholars and practitioners, NCEAS provides both time when and a place where far-reaching topics can be addressed.