Background In the hypothesis that the increased collagen metabolism in the intestinal wall of patients affected by inflammatory bowel disease (IBD) is reflected in the systemic circulation, we aimed the study to evaluate serum level of procollagen III peptide (PIIIP) in peripheral and splanchnic circulation by a commercial radioimmunoassay of patients with different histories of disease.
Background
In the hypothesis that the increased collagen metabolism in the intestinal wall of patients affected by inflammatory bowel disease (IBD) is reflected in the systemic circulation, we aimed the study to evaluate serum level of procollagen III peptide (PIIIP) in peripheral and splanchnic circulation by a commercial radioimmunoassay of patients with different histories of disease.Methods Twenty-seven patients, 17 with Crohn and 10 with ulcerative colitis submitted to surgery were studied. Blood samples were obtained before surgery from a peripheral vein and during surgery from the mesenteric vein draining the affected intestinal segment. Fifteen healthy age and sex matched subjects were studied to determine normal range for peripheral PIIIP.
Methods
Twenty-seven patients, 17 with Crohn and 10 with ulcerative colitis submitted to surgery were studied. Blood samples were obtained before surgery from a peripheral vein and during surgery from the mesenteric vein draining the affected intestinal segment. Fifteen healthy age and sex matched subjects were studied to determine normal range for peripheral PIIIP.Results In IBD patients peripheral PIIIP level was significantly higher if compared with controls (5.0 ± 1.9 vs 2.7 ± 0.7 μg/l; p = 0.0001); splanchnic PIIIP level was 5.5 ± 2.6 μg/l showing a positive gradient between splanchnic and peripheral concentrations of PIIIP. No significant differences between groups nor correlations with patients' age and duration of disease were found.
Results
In IBD patients peripheral PIIIP level was significantly higher if compared with controls (5.0 ± 1.9 vs 2.7 ± 0.7 μg/l; p = 0.0001); splanchnic PIIIP level was 5.5 ± 2.6 μg/l showing a positive gradient between splanchnic and peripheral concentrations of PIIIP. No significant differences between groups nor correlations with patients' age and duration of disease were found.Conclusions We provide evidence that the increased local collagen metabolism in active IBD is reflected also in the systemic circulation irrespective of the history of the disease, suggesting that PIIIP should be considered more appropiately as a marker of the activity phases of IBD.
Conclusions
We provide evidence that the increased local collagen metabolism in active IBD is reflected also in the systemic circulation irrespective of the history of the disease, suggesting that PIIIP should be considered more appropiately as a marker of the activity phases of IBD.

Background Crohn's disease (CD) and Ulcerative Colitis (UC) are chronic inflammatory bowel diseases (IBD) of unknown origin of adolescent and young adulthood [ 1 ] where genetic polimorphisms [ 2 , 3 ], abnormal inflammation pathways activation [ 4 ], and environmental influences [ 5 ] seem to concur at different levels in the pathogenesis and the progression of IBD. These pathologic conditions are characterized by focal or diffuse inflammation of the alimentary tract, mucosal damage and epithelial destruction. IBD may be associated with an inability of the intestinal mucosa to protect itself from luminal challenges and inappropriate repair following intestinal injury [ 6 - 10 ]. CD differs from UC by the transmural granulomatous inflammation generally leading to fibrosis, strictures and fistulas [ 11 ]. Current opinions suggest that an increased synthesis of collagen type I, III, and V may play an important role in the pathophysiological mechanism leading to intestinal fibrosis [ 12 - 15 ]. An increased synthesis of collagen, namely an increased of procollagen type III, is well documented in fibrotic processes involving other organs such as liver, pancreas, and lung [ 16 - 18 ]. However, not all authors are in agreement regarding the increased serum levels of the aminoterminal propeptide (PIIIP) of collagen in peripheral and splanchnic circulation of patients with active IBD [ 13 , 14 ]. Below we present the results on the serum level of PIIIP in splanchnic and peripheral circulation in patients with active IBD submitted to surgery.
Background
Crohn's disease (CD) and Ulcerative Colitis (UC) are chronic inflammatory bowel diseases (IBD) of unknown origin of adolescent and young adulthood [ 1 ] where genetic polimorphisms [ 2 , 3 ], abnormal inflammation pathways activation [ 4 ], and environmental influences [ 5 ] seem to concur at different levels in the pathogenesis and the progression of IBD. These pathologic conditions are characterized by focal or diffuse inflammation of the alimentary tract, mucosal damage and epithelial destruction. IBD may be associated with an inability of the intestinal mucosa to protect itself from luminal challenges and inappropriate repair following intestinal injury [ 6 - 10 ]. CD differs from UC by the transmural granulomatous inflammation generally leading to fibrosis, strictures and fistulas [ 11 ].Current opinions suggest that an increased synthesis of collagen type I, III, and V may play an important role in the pathophysiological mechanism leading to intestinal fibrosis [ 12 - 15 ]. An increased synthesis of collagen, namely an increased of procollagen type III, is well documented in fibrotic processes involving other organs such as liver, pancreas, and lung [ 16 - 18 ].However, not all authors are in agreement regarding the increased serum levels of the aminoterminal propeptide (PIIIP) of collagen in peripheral and splanchnic circulation of patients with active IBD [ 13 , 14 ]. Below we present the results on the serum level of PIIIP in splanchnic and peripheral circulation in patients with active IBD submitted to surgery.Methods Twenty-seven patients affected by active IBD, 17 with CD (age 40.2 ± 13.1, yrs from diagnosis 9.2 ± 5.5), and 10 with UC (age 50.3 ± 15.6, yrs from diagnosis 9.8 ± 7.4) submitted to surgery, were enrolled in the study in a double blind fashion. The protocol was approved by local Ethical Committee and informed consent was obtained from all participants to the study. Three patients had CD in small bowel only, two in large bowel only, 12 had ileocolonic disease. Disease activity was assessed according to the Crohn's Disease Activity Index (CDAI) [ 19 ] and the Truelove-Witts index (TWI) [ 20 ] for CD and UC, respectively. According to CDAI, 2 patients were subclassified as having a moderate form of disease while 15 patients were subclassified as having a severe form of disease. According to TWI, 3 patients were classified as having moderate form of disease, 4 a mild form, and 3 a severe form of disease. Patients affected by CD were operated on for recurrent obstruction, whereas patients with UC were submitted to surgery because of refractory to medical therapy. The clinical diagnosis was confirmed by histology (Fig.  1 ); all cases under study fulfilled the histological criteria as follows: Figure 1 Histological images obtained from two IBD patients enrolled in the study affected by CD (panel a) and UC (panel b) with 12.0 and 10.3 μg/l splanchnic levels of PIIIP, respectively.  Panel a , CD: in the transmural section is clearly evident an ulceration (o) in the mucosa and submucosa with diffuse inflammatory infiltrations, pseudo-follicle nodules (arrow), and fibrosis of the intestinal wall.  Panel b, UC:  the inflammatory infiltration is more evident in the mucosa and submucosa with criptic abscesses (asterisks). A serpiginous linear ulcer is evident (arrow). - for CD: deep ulcers, marked proliferation of small lymphoid nodules involving all layers of intestinal wall sometime with sarcoid-type granulomas and serosal inflammation; - for UC: mucosal erosions and superficial ulcerations usually limited to the upper submucosa with cryptic abscesses and glandular destruction. The clinical profile of the studied patients is reported (Table  1 ). Two patients did not receive any medication, whereas other patients received two or three drugs for the treatment of IBD. Table  2  shows the treatment protocol for all the studied patients. A control group of 15 healthy age and gender matched subjects was also studied to determine normal range for peripheral PIIIP. Table 1 Clinical aspects of the studied patients Disease N° of patients Age Sex Years from diagnosis Activity index Crohn CDAI 1 34 F 12 Severe 2 35 M 8 Moderate 3 38 M 15 Severe 4 32 F 8 Severe 5 43 F 14 Severe 6 35 M 8 Severe 7 58 M 5 Severe 8 61 F 7 Severe 9 33 M 8 Severe 10 35 F 19 Severe 11 21 F 4 Severe 12 72 F 1 Severe 13 36 M 15 Moderate 14 42 M 7 Severe 15 25 F 6 Severe 16 51 F 18 Severe 17 34 M 1 Severe Ulcerative colitis TWI 1 68 F 15 Severe 2 70 F 1 Moderate 3 21 F 3 Mild 4 54 F 20 Moderate 5 52 M 20 Mild 6 38 M 4 Moderate 7 64 M 6 Mild 8 38 F 16 Severe 9 41 F 3 Severe 10 57 M 10 Mild CDAI: Crohn's Disease Activity Index; TWI: truelove-Witts index. Table 2 Frequency distribution for therapy Therapy N° of patients Crohn Ulcerative colitis No 2 1 1 Aminosalicydic acid 9 8 1 Cortisone 2 2 0 Aminosalicydic acid + Cortisone 14 6 8 Total 27 17 10 Collagen metabolism (PIIIP) Different kinds of collagen have been identified in humans. All of them derive from longer precursor molecules (procollagens). They are synthesized intracellularly and secreted in extracellular space where they are cleaved by aminoproteases [ 21 - 23 ]. Among the different kinds of precursors, type III is one of the most abundant interstitial procollagens. Since its aminoterminal propeptide, PIIIP, is formed in equimolar proportions to collagen, serum measurements of this fragment can provide an index of collagen synthesis [ 23 ]. The blood samples (two, 5-ml each) for PIIIP measurements were taken from the median cubital vein (p-PIIIP) before surgery after an overnight fast, during surgery from a mesenteric vein (s-PIIIP) draining the intestinal segment chosen for resection by the surgeon. Serum levels of PIIIP were assessed by commercial radioimmunoassay (Orion Diagnostics, Finland). The intra-assay and inter-assay variation were respectively 4% and 4.3%, mean 2.6 μg/l. Normal ranges of peripheral PIIIP concentrations assessed in the control group were 2.7 ± 0.7 μg/l. Statistical analysis Data were analyzed using a computer statistical software (SPSS-Rel 10; SPSS Inc., Chicago, Ill). All the quantitative variables were tested for Gaussian distribution with the Kolmogorov-Smirnov test. All that followed this distribution were presented as mean ± standard deviation. Differences at baseline in collagen parameters between IBD patients and controls were tested for significance using the analysis of variance with the Bonferroni correction. The relation between collagen parameters and the estimated duration of the disease and indices of disease were tested with regression analysis. In all cases, a p value less than 0.05 was considered significant.
Methods
Twenty-seven patients affected by active IBD, 17 with CD (age 40.2 ± 13.1, yrs from diagnosis 9.2 ± 5.5), and 10 with UC (age 50.3 ± 15.6, yrs from diagnosis 9.8 ± 7.4) submitted to surgery, were enrolled in the study in a double blind fashion. The protocol was approved by local Ethical Committee and informed consent was obtained from all participants to the study. Three patients had CD in small bowel only, two in large bowel only, 12 had ileocolonic disease. Disease activity was assessed according to the Crohn's Disease Activity Index (CDAI) [ 19 ] and the Truelove-Witts index (TWI) [ 20 ] for CD and UC, respectively. According to CDAI, 2 patients were subclassified as having a moderate form of disease while 15 patients were subclassified as having a severe form of disease. According to TWI, 3 patients were classified as having moderate form of disease, 4 a mild form, and 3 a severe form of disease.Patients affected by CD were operated on for recurrent obstruction, whereas patients with UC were submitted to surgery because of refractory to medical therapy.The clinical diagnosis was confirmed by histology (Fig.  1 ); all cases under study fulfilled the histological criteria as follows:- for CD: deep ulcers, marked proliferation of small lymphoid nodules involving all layers of intestinal wall sometime with sarcoid-type granulomas and serosal inflammation;- for UC: mucosal erosions and superficial ulcerations usually limited to the upper submucosa with cryptic abscesses and glandular destruction.The clinical profile of the studied patients is reported (Table  1 ). Two patients did not receive any medication, whereas other patients received two or three drugs for the treatment of IBD. Table  2  shows the treatment protocol for all the studied patients. A control group of 15 healthy age and gender matched subjects was also studied to determine normal range for peripheral PIIIP.Clinical aspects of the studied patientsDiseaseN° of patientsAgeSexYears from diagnosisActivity indexCrohnCDAIUlcerative colitisTWICDAI: Crohn's Disease Activity Index; TWI: truelove-Witts index.Frequency distribution for therapyTherapyN° of patientsCrohnUlcerative colitisCollagen metabolism (PIIIP) Different kinds of collagen have been identified in humans. All of them derive from longer precursor molecules (procollagens). They are synthesized intracellularly and secreted in extracellular space where they are cleaved by aminoproteases [ 21 - 23 ]. Among the different kinds of precursors, type III is one of the most abundant interstitial procollagens. Since its aminoterminal propeptide, PIIIP, is formed in equimolar proportions to collagen, serum measurements of this fragment can provide an index of collagen synthesis [ 23 ]. The blood samples (two, 5-ml each) for PIIIP measurements were taken from the median cubital vein (p-PIIIP) before surgery after an overnight fast, during surgery from a mesenteric vein (s-PIIIP) draining the intestinal segment chosen for resection by the surgeon. Serum levels of PIIIP were assessed by commercial radioimmunoassay (Orion Diagnostics, Finland). The intra-assay and inter-assay variation were respectively 4% and 4.3%, mean 2.6 μg/l. Normal ranges of peripheral PIIIP concentrations assessed in the control group were 2.7 ± 0.7 μg/l.
Collagen metabolism (PIIIP)
Different kinds of collagen have been identified in humans. All of them derive from longer precursor molecules (procollagens). They are synthesized intracellularly and secreted in extracellular space where they are cleaved by aminoproteases [ 21 - 23 ]. Among the different kinds of precursors, type III is one of the most abundant interstitial procollagens. Since its aminoterminal propeptide, PIIIP, is formed in equimolar proportions to collagen, serum measurements of this fragment can provide an index of collagen synthesis [ 23 ]. The blood samples (two, 5-ml each) for PIIIP measurements were taken from the median cubital vein (p-PIIIP) before surgery after an overnight fast, during surgery from a mesenteric vein (s-PIIIP) draining the intestinal segment chosen for resection by the surgeon. Serum levels of PIIIP were assessed by commercial radioimmunoassay (Orion Diagnostics, Finland). The intra-assay and inter-assay variation were respectively 4% and 4.3%, mean 2.6 μg/l. Normal ranges of peripheral PIIIP concentrations assessed in the control group were 2.7 ± 0.7 μg/l.Statistical analysis Data were analyzed using a computer statistical software (SPSS-Rel 10; SPSS Inc., Chicago, Ill). All the quantitative variables were tested for Gaussian distribution with the Kolmogorov-Smirnov test. All that followed this distribution were presented as mean ± standard deviation. Differences at baseline in collagen parameters between IBD patients and controls were tested for significance using the analysis of variance with the Bonferroni correction. The relation between collagen parameters and the estimated duration of the disease and indices of disease were tested with regression analysis. In all cases, a p value less than 0.05 was considered significant.
Statistical analysis
Data were analyzed using a computer statistical software (SPSS-Rel 10; SPSS Inc., Chicago, Ill). All the quantitative variables were tested for Gaussian distribution with the Kolmogorov-Smirnov test. All that followed this distribution were presented as mean ± standard deviation.Differences at baseline in collagen parameters between IBD patients and controls were tested for significance using the analysis of variance with the Bonferroni correction. The relation between collagen parameters and the estimated duration of the disease and indices of disease were tested with regression analysis. In all cases, a p value less than 0.05 was considered significant.Results Peripheral PIIIP assay At baseline, before surgery, serum p-PIIIP in IBD patients were significantly higher if compared with healthy controls (5.0 ± 1.9 vs 2.7 ± 0.7 μg/l, respectively; p = 0.0001) (fig  2a ). No significant differences were found when comparing CD and UC subgroups (5.0 ± 1.6 vs 4.9 ± 2.4 μg/l, respectively; p = ns) (fig  2b ). Figure 2 Panel a:  Differences in baseline p-PIIIP values in Controls and IBD patients.  Panel b:  No significant differences in p-PIIIP values between CD and UC subgroups.  Panel c:  Differences between splancnic and periferic values of PIIIP, without significant differences in CD and UC sbgroups. p-PIIIP: periferic (median cubital vein) PIIIP; s-PIIIP: splancnic (mesenteric vein) PIIIP; Δ-PIIIP: differences between s- and p-PIIIP in IBD patients; IBD: inflammatory bowel diseases; CD: Crohn's Disease; UC: ulcerative colitis; Splanchnic PIIIP assay During surgery, serum s-PIIIP in IBD patients was 5.5 ± 2.6 μg/l. No significant differences were found when comparing CD and UC subgroups (5.4 ± 2.3 vs 5.7 ± 3.1 μg/l, respectively; p = ns). A positive gradient was found in IBD patients between splanchnic and peripheral serum concentrations of PIIIP (0.7 ± 1.9 μg/l). This gradient was confirmed when separately considering each disease, without significant differences between the two subgroups (CD 0.3 ± 1.3 vs UC 1.3 ± 2.6 μg/l; p = ns) (fig  2c ). Other variables and PIIIP levels No significant correlation was found between peripheral and splanchnic levels of PIIIP and the age of the patients and the estimated duration of the disease. Regarding the activity indices, the number of patients belonging to each class was not enough to perform a statistical analysis. Notwithstanding, for the TWI in UC patients a significant difference in PIIIP levels was found between mild and severe form of the disease (Table  3 ). Finally, no significant differences were found in PIIIP levels between patients treated with glucocorticoids compared with patients not receiving this treatment. Table 3 Baseline p-PIIIP levels in UC patients TWI Activity Index Mean ± SD Mild 7.05 ± 2.25 Moderate 3.9 ± 1.11 Severe 3.13 ± 1.57* * p = 0.02 vs mild
Results
Peripheral PIIIP assay At baseline, before surgery, serum p-PIIIP in IBD patients were significantly higher if compared with healthy controls (5.0 ± 1.9 vs 2.7 ± 0.7 μg/l, respectively; p = 0.0001) (fig  2a ). No significant differences were found when comparing CD and UC subgroups (5.0 ± 1.6 vs 4.9 ± 2.4 μg/l, respectively; p = ns) (fig  2b ). Figure 2 Panel a:  Differences in baseline p-PIIIP values in Controls and IBD patients.  Panel b:  No significant differences in p-PIIIP values between CD and UC subgroups.  Panel c:  Differences between splancnic and periferic values of PIIIP, without significant differences in CD and UC sbgroups. p-PIIIP: periferic (median cubital vein) PIIIP; s-PIIIP: splancnic (mesenteric vein) PIIIP; Δ-PIIIP: differences between s- and p-PIIIP in IBD patients; IBD: inflammatory bowel diseases; CD: Crohn's Disease; UC: ulcerative colitis;
Peripheral PIIIP assay
At baseline, before surgery, serum p-PIIIP in IBD patients were significantly higher if compared with healthy controls (5.0 ± 1.9 vs 2.7 ± 0.7 μg/l, respectively; p = 0.0001) (fig  2a ). No significant differences were found when comparing CD and UC subgroups (5.0 ± 1.6 vs 4.9 ± 2.4 μg/l, respectively; p = ns) (fig  2b ).Splanchnic PIIIP assay During surgery, serum s-PIIIP in IBD patients was 5.5 ± 2.6 μg/l. No significant differences were found when comparing CD and UC subgroups (5.4 ± 2.3 vs 5.7 ± 3.1 μg/l, respectively; p = ns). A positive gradient was found in IBD patients between splanchnic and peripheral serum concentrations of PIIIP (0.7 ± 1.9 μg/l). This gradient was confirmed when separately considering each disease, without significant differences between the two subgroups (CD 0.3 ± 1.3 vs UC 1.3 ± 2.6 μg/l; p = ns) (fig  2c ).
Splanchnic PIIIP assay
During surgery, serum s-PIIIP in IBD patients was 5.5 ± 2.6 μg/l. No significant differences were found when comparing CD and UC subgroups (5.4 ± 2.3 vs 5.7 ± 3.1 μg/l, respectively; p = ns). A positive gradient was found in IBD patients between splanchnic and peripheral serum concentrations of PIIIP (0.7 ± 1.9 μg/l). This gradient was confirmed when separately considering each disease, without significant differences between the two subgroups (CD 0.3 ± 1.3 vs UC 1.3 ± 2.6 μg/l; p = ns) (fig  2c ).Other variables and PIIIP levels No significant correlation was found between peripheral and splanchnic levels of PIIIP and the age of the patients and the estimated duration of the disease. Regarding the activity indices, the number of patients belonging to each class was not enough to perform a statistical analysis. Notwithstanding, for the TWI in UC patients a significant difference in PIIIP levels was found between mild and severe form of the disease (Table  3 ). Finally, no significant differences were found in PIIIP levels between patients treated with glucocorticoids compared with patients not receiving this treatment. Table 3 Baseline p-PIIIP levels in UC patients TWI Activity Index Mean ± SD Mild 7.05 ± 2.25 Moderate 3.9 ± 1.11 Severe 3.13 ± 1.57* * p = 0.02 vs mild
Other variables and PIIIP levels
No significant correlation was found between peripheral and splanchnic levels of PIIIP and the age of the patients and the estimated duration of the disease. Regarding the activity indices, the number of patients belonging to each class was not enough to perform a statistical analysis. Notwithstanding, for the TWI in UC patients a significant difference in PIIIP levels was found between mild and severe form of the disease (Table  3 ). Finally, no significant differences were found in PIIIP levels between patients treated with glucocorticoids compared with patients not receiving this treatment.Baseline p-PIIIP levels in UC patientsTWI Activity IndexMean ± SDMildModerateSevere* p = 0.02 vs mildDiscussion CD and UC are chronic pathologies characterized by an early onset followed by sporadic episodes of acute symptoms during lifetime, debilitating the affected patients to perform their daily functions [ 24 ]. Until now controversial theories exist about the synthesis and degradation of PIIIP, its level on systemic circulation, and its deposition far for main target organ [ 12 , 13 , 15 , 25 ]. In the present study we have found that intestinal collagen metabolism in IBD patients was increased and that it is reflected in local and systemic circulation. Differently from some experiences [ 12 , 13 ], we have found that serum PIIIP levels in IBD patients was significantly higher if compared with healthy subjects. No significant differences were found in peripheral and splanchnic circulation between patients affected by UC and CD. We also found a positive gradient between serum s-PIIIP and p-PIIIP levels in IBD patients. This gradient was confirmed when considering serum s-PIIIP and p-PIIIP in UC and CD separately, even if the differences between the two subgroups were not statistically significant. In our experience no significant differences were found when considering the age of the patients, the duration of the disease, and the activity indices. This fact implies that serum PIIIP should not be considered a long-term marker of the disease, probably reflecting the short-term fluctuation in the activity phases of the remodeling processes. When comparing the mild with the severe form of the disease, a significant difference in PIIIP levels was found only in patients affected by UC. This data will probably be confirmed when the number of patients enrolled in each disease-related activity categories is extended as presently in our series the majority of the patients were classified as severe. The effect of glucocorticoids on collagen synthesis, collagenase, and collagen degradation has not yet fully been clarified [ 25 ]. In our study the cortisone therapy did not have influence on the PIIIP levels, but the number of patients was too small and it was not possible to speculate on this regard.
Discussion
CD and UC are chronic pathologies characterized by an early onset followed by sporadic episodes of acute symptoms during lifetime, debilitating the affected patients to perform their daily functions [ 24 ]. Until now controversial theories exist about the synthesis and degradation of PIIIP, its level on systemic circulation, and its deposition far for main target organ [ 12 , 13 , 15 , 25 ].In the present study we have found that intestinal collagen metabolism in IBD patients was increased and that it is reflected in local and systemic circulation. Differently from some experiences [ 12 , 13 ], we have found that serum PIIIP levels in IBD patients was significantly higher if compared with healthy subjects. No significant differences were found in peripheral and splanchnic circulation between patients affected by UC and CD. We also found a positive gradient between serum s-PIIIP and p-PIIIP levels in IBD patients. This gradient was confirmed when considering serum s-PIIIP and p-PIIIP in UC and CD separately, even if the differences between the two subgroups were not statistically significant. In our experience no significant differences were found when considering the age of the patients, the duration of the disease, and the activity indices. This fact implies that serum PIIIP should not be considered a long-term marker of the disease, probably reflecting the short-term fluctuation in the activity phases of the remodeling processes.When comparing the mild with the severe form of the disease, a significant difference in PIIIP levels was found only in patients affected by UC. This data will probably be confirmed when the number of patients enrolled in each disease-related activity categories is extended as presently in our series the majority of the patients were classified as severe.The effect of glucocorticoids on collagen synthesis, collagenase, and collagen degradation has not yet fully been clarified [ 25 ]. In our study the cortisone therapy did not have influence on the PIIIP levels, but the number of patients was too small and it was not possible to speculate on this regard.Conclusions In conclusion we provide evidence that collagen metabolism in IBD is reflected in the systemic and local circulation, without any differences between UC and CD, irrespective of the age of the patients and the duration of the disease. Therefore, this marker may give further information on the activity phases rather than on the entire history of the disease. Further data on the possible use of PIIIP as useful marker of choice for surgical option are attended from the follow-up at 6 and 12 months, which is still on-going [ 26 ].
Conclusions
In conclusion we provide evidence that collagen metabolism in IBD is reflected in the systemic and local circulation, without any differences between UC and CD, irrespective of the age of the patients and the duration of the disease. Therefore, this marker may give further information on the activity phases rather than on the entire history of the disease. Further data on the possible use of PIIIP as useful marker of choice for surgical option are attended from the follow-up at 6 and 12 months, which is still on-going [ 26 ].List of abbreviations Inflammatory bowel diseases = IBD Procollagen III propeptide = PIIIP Peripheral Procollagen III propeptide = p-PIIIP Splanchnic Procollagen III propeptide = s-PIIIP Crohn's disease = CD Ulcerative Colitis = UC Crohn's Disease Activity Index = CDAI Truelove-Witts index = TWI
List of abbreviations
Inflammatory bowel diseases = IBDProcollagen III propeptide = PIIIPPeripheral Procollagen III propeptide = p-PIIIPSplanchnic Procollagen III propeptide = s-PIIIPCrohn's disease = CDUlcerative Colitis = UCCrohn's Disease Activity Index = CDAITruelove-Witts index = TWICompeting interests The author(s) declare that they have no competing interests.
Competing interests
The author(s) declare that they have no competing interests.Authors' contributions UC conception and design, interpretation of data, drafting the article MDS conception and design, interpretation of data, drafting the article ECA performed surgical operations, critical revision of the article, final approval of the version BO patients' enrollement, blood samples collection RP statistical analysis, interpretation of data, drafting the article AP echocardiographic studies GB radioimmunoassays EO radioimmunoassays SF histological examinations FM interpretation of data, critical revision of the article, final approval of the version MMC conception and design, interpretation of data, drafting the article All Authors read and approved the final manuscript.
Authors' contributions
UC conception and design, interpretation of data, drafting the articleMDS conception and design, interpretation of data, drafting the articleECA performed surgical operations, critical revision of the article, final approval of the versionBO patients' enrollement, blood samples collectionRP statistical analysis, interpretation of data, drafting the articleAP echocardiographic studiesGB radioimmunoassaysEO radioimmunoassaysSF histological examinationsFM interpretation of data, critical revision of the article, final approval of the versionMMC conception and design, interpretation of data, drafting the articleAll Authors read and approved the final manuscript.Pre-publication history The pre-publication history for this paper can be accessed here:
Pre-publication history
The pre-publication history for this paper can be accessed here:


Background Balanced decisions about health care interventions require reliable evidence on harms as well as benefits. Most systematic reviews focus on efficacy and randomised trials, for which the methodology is well established. Methods to systematically review harmful effects are less well developed and there are few sources of guidance for researchers. We present our own recent experience of conducting systematic reviews of harmful effects and make suggestions for future practice and further research.
Background
Balanced decisions about health care interventions require reliable evidence on harms as well as benefits. Most systematic reviews focus on efficacy and randomised trials, for which the methodology is well established. Methods to systematically review harmful effects are less well developed and there are few sources of guidance for researchers. We present our own recent experience of conducting systematic reviews of harmful effects and make suggestions for future practice and further research.Methods We described and compared the methods used in three systematic reviews. Our evaluation focused on the review question, study designs and quality assessment.
Methods
We described and compared the methods used in three systematic reviews. Our evaluation focused on the review question, study designs and quality assessment.Results One review question focused on providing information on specific harmful effects to furnish an economic model, the other two addressed much broader questions. All three reviews included randomised and observational data, although each defined the inclusion criteria differently. Standard methods were used to assess study quality. Various practical problems were encountered in applying the study design inclusion criteria and assessing quality, mainly because of poor study design, inadequate reporting and the limitations of existing tools. All three reviews generated a large volume of work that did not yield much useful information for health care decision makers. The key areas for improvement we identified were focusing the review question and developing methods for quality assessment of studies of harmful effects.
Results
One review question focused on providing information on specific harmful effects to furnish an economic model, the other two addressed much broader questions. All three reviews included randomised and observational data, although each defined the inclusion criteria differently. Standard methods were used to assess study quality. Various practical problems were encountered in applying the study design inclusion criteria and assessing quality, mainly because of poor study design, inadequate reporting and the limitations of existing tools. All three reviews generated a large volume of work that did not yield much useful information for health care decision makers. The key areas for improvement we identified were focusing the review question and developing methods for quality assessment of studies of harmful effects.Conclusions Systematic reviews of harmful effects are more likely to yield information pertinent to clinical decision-making if they address a focused question. This will enable clear decisions to be made about the type of research to include in the review. The methodology for assessing the quality of harmful effects data in systematic reviews requires further development.
Conclusions
Systematic reviews of harmful effects are more likely to yield information pertinent to clinical decision-making if they address a focused question. This will enable clear decisions to be made about the type of research to include in the review. The methodology for assessing the quality of harmful effects data in systematic reviews requires further development.

Background Systematic reviews are important tools for evidence-based health care. They are certainly one of the reasons for the progress that has been made in obtaining reliable evidence on the beneficial effects of interventions. A recent study of the medical literature, using Medline and the Cochrane Library, showed that the number of systematic reviews published has increased dramatically, from a single publication in the years 1966 to 1970, to 23 in 1981 to 1985, and 2467 in 1996 to 2000 [ 1 ]. Most of the systematic reviews focused on efficacy or effectiveness. However, to make a balanced decision about any intervention it is essential to have reliable evidence on the harms as well as the benefits. Although the coverage of harmful effects has increased over time, only 27% of the reviews published between 1996 and 2000 included any information about safety, and only 4% focused primarily on the safety of the intervention reviewed [ 1 ]. This is perhaps unsurprising as many authors of systematic reviews restrict inclusion to randomised controlled trials (RCTs) to minimise bias, and harmful effects are often inadequately assessed and/or reported in RCTs [ 2 , 3 ]. Another important reason for the relative lack of reliable evidence on harmful effects is that RCTs are not always suitable to evaluate them and other types of study design need to be considered [ 4 ]. The methodology for conducting systematic reviews of beneficial effects from RCTs is well established, whereas the methods for systematically reviewing randomised or observational data on harmful effects are less well developed and less often used. Only 1.25% of 3604 publications cited in the 2001 edition of Side Effects of Drugs (SEDA-24) were systematic reviews [ 5 ]. At present researchers, like us, who conduct systematic reviews have limited sources of guidance, such as the suggestions offered by the Cochrane Collaboration [ 6 ]. Fortunately, research into the methodology of incorporating harmful effects data in systematic reviews is on the increase, from which we expect more sources of guidance to emerge. It is not uncommon, even among experienced reviewers, to assume that the objective of a systematic review of harmful effects should encompass all known and previously unrecognised harmful effects and that data from all types of study design should be sought. We have re-visited three systematic reviews of drug interventions in which we had reviewed harmful effects, to evaluate our own recent experience, identify areas for improvement and to share our ideas with other researchers undertaking reviews.
Background
Systematic reviews are important tools for evidence-based health care. They are certainly one of the reasons for the progress that has been made in obtaining reliable evidence on the beneficial effects of interventions. A recent study of the medical literature, using Medline and the Cochrane Library, showed that the number of systematic reviews published has increased dramatically, from a single publication in the years 1966 to 1970, to 23 in 1981 to 1985, and 2467 in 1996 to 2000 [ 1 ]. Most of the systematic reviews focused on efficacy or effectiveness. However, to make a balanced decision about any intervention it is essential to have reliable evidence on the harms as well as the benefits. Although the coverage of harmful effects has increased over time, only 27% of the reviews published between 1996 and 2000 included any information about safety, and only 4% focused primarily on the safety of the intervention reviewed [ 1 ]. This is perhaps unsurprising as many authors of systematic reviews restrict inclusion to randomised controlled trials (RCTs) to minimise bias, and harmful effects are often inadequately assessed and/or reported in RCTs [ 2 , 3 ]. Another important reason for the relative lack of reliable evidence on harmful effects is that RCTs are not always suitable to evaluate them and other types of study design need to be considered [ 4 ].The methodology for conducting systematic reviews of beneficial effects from RCTs is well established, whereas the methods for systematically reviewing randomised or observational data on harmful effects are less well developed and less often used. Only 1.25% of 3604 publications cited in the 2001 edition of Side Effects of Drugs (SEDA-24) were systematic reviews [ 5 ]. At present researchers, like us, who conduct systematic reviews have limited sources of guidance, such as the suggestions offered by the Cochrane Collaboration [ 6 ]. Fortunately, research into the methodology of incorporating harmful effects data in systematic reviews is on the increase, from which we expect more sources of guidance to emerge.It is not uncommon, even among experienced reviewers, to assume that the objective of a systematic review of harmful effects should encompass all known and previously unrecognised harmful effects and that data from all types of study design should be sought. We have re-visited three systematic reviews of drug interventions in which we had reviewed harmful effects, to evaluate our own recent experience, identify areas for improvement and to share our ideas with other researchers undertaking reviews.Methods We used three reviews for this study on the basis that they had been completed recently (between 2001 and 2003) and that one of us had been the lead reviewer of harmful effects in each review. The reviews were conducted as Health Technology Assessments for the National Coordinating Centre for Health Technology Assessment (NCCHTA) on behalf of the National Institute for Clinical Excellence (NICE). The reviews, in order of completion, were: nicotine replacement therapy (NRT) and bupropion sustained release (SR) for aiding smoking cessation [ 7 ], atypical antipsychotics for schizophrenia [ 8 ], and newer antiepileptic drugs for epilepsy in adults [ 9 ]. We described and compared the methods used in each review and the problems we encountered in applying those methods. We focused our evaluation on the review objectives, the inclusion criteria for study design and the quality assessment of the primary studies. We do not report on the matter of searching for studies about harmful effects which presents another challenge to those who conduct systematic reviews [ 10 , 11 ], because exploratory work following from the reviews described here is underway and preliminary results are reported elsewhere [ 12 , 13 ].
Methods
We used three reviews for this study on the basis that they had been completed recently (between 2001 and 2003) and that one of us had been the lead reviewer of harmful effects in each review. The reviews were conducted as Health Technology Assessments for the National Coordinating Centre for Health Technology Assessment (NCCHTA) on behalf of the National Institute for Clinical Excellence (NICE). The reviews, in order of completion, were: nicotine replacement therapy (NRT) and bupropion sustained release (SR) for aiding smoking cessation [ 7 ], atypical antipsychotics for schizophrenia [ 8 ], and newer antiepileptic drugs for epilepsy in adults [ 9 ].We described and compared the methods used in each review and the problems we encountered in applying those methods. We focused our evaluation on the review objectives, the inclusion criteria for study design and the quality assessment of the primary studies. We do not report on the matter of searching for studies about harmful effects which presents another challenge to those who conduct systematic reviews [ 10 , 11 ], because exploratory work following from the reviews described here is underway and preliminary results are reported elsewhere [ 12 , 13 ].Results The main components of the three systematic reviews of harmful effects are described in Table  1 . Our evaluation highlighted the following aspects of the methodology that could have been improved on and others that require further development. Table 1 Description of the assessment of harmful effects in the three systematic reviews Review Schizophrenia Smoking cessation Epilepsy Intervention evaluated 8 atypical antipsychotics. NRT and bupropion SR. 7 newer antiepileptic drugs. Objective / Scope Review commissioned by the HTA programme and an update of the HTA report commissioned by NICE. The objective regarding harmful effects was to determine the incidence of specific rare adverse events to populate an economic model. Scope provided by NICE: To review all known or unknown harmful effects that might be associated with the interventions. Scope provided by NICE: To include adverse effects in a review of RCTs of clinical effectiveness in adults with epilepsy. The reviewers undertook a supplementary review of serious, rare and long-term harmful effects. Serious was defined by WHO criteria [25], long-term as longer than 6 months, and rare as defined by the authors of primary studies. Study designs included Randomised trials RCTs of atypical antipsychotics versus alternative drug treatment or placebo in schizophrenia. An existing Cochrane review was used as a source of summary data on adverse effects from RCTs of effectiveness [26]. Studies that assessed safety as the primary objective were included in the review of primary studies of harmful effects. This included RCTs that investigated aspects of clinical pharmacology that might impact on the drugs' tolerability and safety. The five most commonly reported adverse effects were extracted from RCTs as part of the review of clinical effectiveness in epilepsy. RCTs in indications other than epilepsy and dose comparisons were eligible for inclusion in the supplementary review of harmful effects. Non-randomised studies Cohort studies and case series with 2000 or more participants or at least 2 years follow-up, and case-control studies of any size or duration. Uncontrolled trials, prospective and retrospective observational studies, data from adverse events monitoring systems (e.g. UK yellow card scheme) and case reports. Non-randomised controlled trials, cohort and case-control studies, prospective case series and other uncontrolled trials, and open-label extension phases of trials. More than 300 participants had to be exposed or follow-up more than 6 months unless the study objective was to investigate a specific adverse effect. Prescription event monitoring [27], and post-marketing surveillance reports were also included. Studies identified 6477 items screened, 924 articles retrieved, and 223 studies included:  171 RCTs, 13 cohort studies, 1 case-control study, 38 case series. 1280 items screened, 353 articles retrieved, and 123 studies included:  25 RCTs, 4 non-randomised controlled trials, 30 uncontrolled trials, before/after studies or cohort studies, 1 case-control study, 9 surveillance studies, 1 survey, 53 case reports or case series. 108 RCTs were included in the review of effectiveness, selected from 4211 items screened and 887 articles retrieved. In the supplementary review of harmful effects 3884 items were screened, 227 articles retrieved, and 77 studies included: 2 RCTs, 2 non-randomised controlled trials, 26 uncontrolled trials, 14 open-label phases, 25 cohort studies, 1 case-control study, 4 prescription event monitoring studies, 3 post-marketing surveillance studies. Quality assessment Quality checklists for various study designs provided in CRD Report 4 were used [28]. The quality checklist for RCTs provided in CRD Report 4 [28], and checklists for the other study designs published elsewhere [29], were used. Published checklists were used as a starting point. Questions were amended and others added to capture information specifically on the reliability of harmful effects data. Findings of review Very few studies with useful data were found, so the economic model could not be populated with incidence rates of the adverse events of interest. Primarily the findings merely reflected the accepted side-effect profiles for NRT and bupropion SR. The review did not identify any previously unknown harmful effects. The supplementary review of harmful effects did identify reports of potential adverse effects not reported in the RCTs of clinical effectiveness. However, these were mostly effects already documented in tertiary sources. There was insufficient evidence to attribute causality of other reported effects to the test drugs. Review objectives The schizophrenia review objective appeared to be appropriate in seeking to determine the incidence of named outcomes that were considered by health economists to be most likely to lead to a change in prescribed treatment [ 14 ]. The objectives of the smoking cessation and epilepsy reviews were very broad in comparison. Given that the side-effect profiles of the drugs for smoking cessation were well established, with details available in various published standard reference texts [ 15 , 16 ], it would have been more efficient to focus the review effort on a clear question, such as the significance of seizures for bupropion SR and the cardiovascular effects of nicotine in NRT. The objective of the review of harmful effects of the antiepileptic drugs did not target clinical decision-making; the supplementary review of harmful effects might have been of real use to decision makers if we had focused on a crucial clinical question such as the safety of the drugs in pregnancy. Study designs All three reviews included study designs other than RCTs to assess harmful effects. The types of non-randomised studies included for each review reflected differences in the reviews' objectives, our judgment as reviewers as to where the most useful data were likely to be found, and was to some extent pragmatic in terms of the time available to complete the reviews. The reviews with the broad objectives included more non-randomised studies and more diverse study designs. The schizophrenia and epilepsy reviews specified a minimum size and duration of study to be included (see table) in an attempt to add data over and above what was available from the largest and longest RCTs. Doing this did involve some indeterminable risk of missing important information. The review of observational studies carried out in the schizophrenia review was necessary because the pre-determined harmful effects of interest were known to be under-reported in RCTs [ 8 ]. The inclusion of non-randomised studies in the smoking cessation review might have targeted observational data on specific questions about harmful effects had we reviewed beforehand the RCTs that were summarised briefly in the Cochrane review. Similarly in the epilepsy review all the adverse events (not just the most common) reported in the RCTs of clinical effectiveness should have been reviewed before moving on to observational studies. Applying the inclusion criteria Once the inclusion criteria for study design had been defined, applying them was problematic. Reports of primary studies rarely described the study design in sufficient detail. Many of the studies included in the schizophrenia review purported to be cohort studies but on closer examination were in fact large case series involving more than one intervention. Some of the 'cohort study' data on bupropion SR included in the smoking cessation review had actually been derived retrospectively from RCTs. How exactly the 'cohorts' had been established in studies of epilepsy was often unclear in terms of the source population, eligibility criteria, and selection, or was simply not reported. Had we, in all three reviews, only included reports of studies fitting textbook definitions of particular study designs, virtually all of the primary study reports we identified would have been excluded. The inclusive approach we took turned out to be unrewarding. In the smoking cessation review, in addition to difficulties with the study design inclusion criteria, application of the criterion to only include studies in which assessment of adverse events was the primary objective was problematic because it involved a high degree of subjective judgment. Quality assessment We encountered problems when applying published checklists in our reviews of harmful effects. The response to some questions depended on the outcome of interest, for example, follow-up may have been adequate for the assessment of the primary (usually a beneficial) outcome of the study but not for the collection of data on harmful effects. We also found that published checklists omit key features such as how harmful effects data were recorded. In the epilepsy review we were in a position to learn from the earlier reviews and spent time clarifying the questions in the checklists so that they would provide information relevant to the reliability of the harmful effects data. We also added items pertinent to reports of harmful effects such as how and when events were recorded and whether the time at which they occurred during the study was reported. Although this informed approach was a step in the right direction, the major hindrance to applying checklists in all three reviews was inadequate reporting of the basic design features of the primary studies. Once the quality criteria had been applied there remained the challenge of interpreting the results. In our reviews we described the evidence identified and tabulated the response to each checklist question for each primary study. This generated lengthy summaries that had limited utility. Even comparing validity within study designs (not across them) we found it impossible to synthesise the information as all the included studies had methodological flaws and features that could not be assessed due to inadequate reporting. Reaching a decision about which studies were likely to give the most reliable results was not straightforward.
Results
The main components of the three systematic reviews of harmful effects are described in Table  1 . Our evaluation highlighted the following aspects of the methodology that could have been improved on and others that require further development.Description of the assessment of harmful effects in the three systematic reviewsReviewSchizophreniaSmoking cessationEpilepsyIntervention evaluatedObjective / ScopeStudy designs includedRandomised trialsRandomised trialsNon-randomised studiesNon-randomised studiesStudies identifiedQuality assessmentFindings of reviewReview objectives The schizophrenia review objective appeared to be appropriate in seeking to determine the incidence of named outcomes that were considered by health economists to be most likely to lead to a change in prescribed treatment [ 14 ]. The objectives of the smoking cessation and epilepsy reviews were very broad in comparison. Given that the side-effect profiles of the drugs for smoking cessation were well established, with details available in various published standard reference texts [ 15 , 16 ], it would have been more efficient to focus the review effort on a clear question, such as the significance of seizures for bupropion SR and the cardiovascular effects of nicotine in NRT. The objective of the review of harmful effects of the antiepileptic drugs did not target clinical decision-making; the supplementary review of harmful effects might have been of real use to decision makers if we had focused on a crucial clinical question such as the safety of the drugs in pregnancy.
Review objectives
The schizophrenia review objective appeared to be appropriate in seeking to determine the incidence of named outcomes that were considered by health economists to be most likely to lead to a change in prescribed treatment [ 14 ]. The objectives of the smoking cessation and epilepsy reviews were very broad in comparison. Given that the side-effect profiles of the drugs for smoking cessation were well established, with details available in various published standard reference texts [ 15 , 16 ], it would have been more efficient to focus the review effort on a clear question, such as the significance of seizures for bupropion SR and the cardiovascular effects of nicotine in NRT. The objective of the review of harmful effects of the antiepileptic drugs did not target clinical decision-making; the supplementary review of harmful effects might have been of real use to decision makers if we had focused on a crucial clinical question such as the safety of the drugs in pregnancy.Study designs All three reviews included study designs other than RCTs to assess harmful effects. The types of non-randomised studies included for each review reflected differences in the reviews' objectives, our judgment as reviewers as to where the most useful data were likely to be found, and was to some extent pragmatic in terms of the time available to complete the reviews. The reviews with the broad objectives included more non-randomised studies and more diverse study designs. The schizophrenia and epilepsy reviews specified a minimum size and duration of study to be included (see table) in an attempt to add data over and above what was available from the largest and longest RCTs. Doing this did involve some indeterminable risk of missing important information. The review of observational studies carried out in the schizophrenia review was necessary because the pre-determined harmful effects of interest were known to be under-reported in RCTs [ 8 ]. The inclusion of non-randomised studies in the smoking cessation review might have targeted observational data on specific questions about harmful effects had we reviewed beforehand the RCTs that were summarised briefly in the Cochrane review. Similarly in the epilepsy review all the adverse events (not just the most common) reported in the RCTs of clinical effectiveness should have been reviewed before moving on to observational studies.
Study designs
All three reviews included study designs other than RCTs to assess harmful effects. The types of non-randomised studies included for each review reflected differences in the reviews' objectives, our judgment as reviewers as to where the most useful data were likely to be found, and was to some extent pragmatic in terms of the time available to complete the reviews. The reviews with the broad objectives included more non-randomised studies and more diverse study designs. The schizophrenia and epilepsy reviews specified a minimum size and duration of study to be included (see table) in an attempt to add data over and above what was available from the largest and longest RCTs. Doing this did involve some indeterminable risk of missing important information.The review of observational studies carried out in the schizophrenia review was necessary because the pre-determined harmful effects of interest were known to be under-reported in RCTs [ 8 ]. The inclusion of non-randomised studies in the smoking cessation review might have targeted observational data on specific questions about harmful effects had we reviewed beforehand the RCTs that were summarised briefly in the Cochrane review. Similarly in the epilepsy review all the adverse events (not just the most common) reported in the RCTs of clinical effectiveness should have been reviewed before moving on to observational studies.Applying the inclusion criteria Once the inclusion criteria for study design had been defined, applying them was problematic. Reports of primary studies rarely described the study design in sufficient detail. Many of the studies included in the schizophrenia review purported to be cohort studies but on closer examination were in fact large case series involving more than one intervention. Some of the 'cohort study' data on bupropion SR included in the smoking cessation review had actually been derived retrospectively from RCTs. How exactly the 'cohorts' had been established in studies of epilepsy was often unclear in terms of the source population, eligibility criteria, and selection, or was simply not reported. Had we, in all three reviews, only included reports of studies fitting textbook definitions of particular study designs, virtually all of the primary study reports we identified would have been excluded. The inclusive approach we took turned out to be unrewarding. In the smoking cessation review, in addition to difficulties with the study design inclusion criteria, application of the criterion to only include studies in which assessment of adverse events was the primary objective was problematic because it involved a high degree of subjective judgment.
Applying the inclusion criteria
Once the inclusion criteria for study design had been defined, applying them was problematic. Reports of primary studies rarely described the study design in sufficient detail. Many of the studies included in the schizophrenia review purported to be cohort studies but on closer examination were in fact large case series involving more than one intervention. Some of the 'cohort study' data on bupropion SR included in the smoking cessation review had actually been derived retrospectively from RCTs. How exactly the 'cohorts' had been established in studies of epilepsy was often unclear in terms of the source population, eligibility criteria, and selection, or was simply not reported. Had we, in all three reviews, only included reports of studies fitting textbook definitions of particular study designs, virtually all of the primary study reports we identified would have been excluded. The inclusive approach we took turned out to be unrewarding.In the smoking cessation review, in addition to difficulties with the study design inclusion criteria, application of the criterion to only include studies in which assessment of adverse events was the primary objective was problematic because it involved a high degree of subjective judgment.Quality assessment We encountered problems when applying published checklists in our reviews of harmful effects. The response to some questions depended on the outcome of interest, for example, follow-up may have been adequate for the assessment of the primary (usually a beneficial) outcome of the study but not for the collection of data on harmful effects. We also found that published checklists omit key features such as how harmful effects data were recorded. In the epilepsy review we were in a position to learn from the earlier reviews and spent time clarifying the questions in the checklists so that they would provide information relevant to the reliability of the harmful effects data. We also added items pertinent to reports of harmful effects such as how and when events were recorded and whether the time at which they occurred during the study was reported. Although this informed approach was a step in the right direction, the major hindrance to applying checklists in all three reviews was inadequate reporting of the basic design features of the primary studies. Once the quality criteria had been applied there remained the challenge of interpreting the results. In our reviews we described the evidence identified and tabulated the response to each checklist question for each primary study. This generated lengthy summaries that had limited utility. Even comparing validity within study designs (not across them) we found it impossible to synthesise the information as all the included studies had methodological flaws and features that could not be assessed due to inadequate reporting. Reaching a decision about which studies were likely to give the most reliable results was not straightforward.
Quality assessment
We encountered problems when applying published checklists in our reviews of harmful effects. The response to some questions depended on the outcome of interest, for example, follow-up may have been adequate for the assessment of the primary (usually a beneficial) outcome of the study but not for the collection of data on harmful effects. We also found that published checklists omit key features such as how harmful effects data were recorded. In the epilepsy review we were in a position to learn from the earlier reviews and spent time clarifying the questions in the checklists so that they would provide information relevant to the reliability of the harmful effects data. We also added items pertinent to reports of harmful effects such as how and when events were recorded and whether the time at which they occurred during the study was reported. Although this informed approach was a step in the right direction, the major hindrance to applying checklists in all three reviews was inadequate reporting of the basic design features of the primary studies.Once the quality criteria had been applied there remained the challenge of interpreting the results. In our reviews we described the evidence identified and tabulated the response to each checklist question for each primary study. This generated lengthy summaries that had limited utility. Even comparing validity within study designs (not across them) we found it impossible to synthesise the information as all the included studies had methodological flaws and features that could not be assessed due to inadequate reporting. Reaching a decision about which studies were likely to give the most reliable results was not straightforward.Discussion Our experience of reviewing harmful effects mirrors that of other researchers in that a significant investment of effort failed to yield significant new information [ 6 , 17 ]. A focused review question is standard practice for assessing beneficial outcomes in systematic reviews and should also be so when reviewing harms. Researchers conducting reviews need to make sure that they address a well-formulated question about harms that are likely to impact on clinical decisions. Focusing a review question about harmful effects will not necessarily mean restricting it to specific adverse events but may mean, for example, addressing a particular issue such as long-term effects, drug interactions, or the incidence of mild effects of importance to patients. If the aim of the research is to look for previously unrecognised harmful effects, analysis of primary surveillance data may be more appropriate than a systematic review [ 18 ]. Researchers also need to be aware that scopes set by external commissioning bodies, despite having consulted with national professional and patient organisations, may not be a suitable question to address in a systematic review. The wisdom of broad and non-specific questions about harmful effects should be questioned because the resources, especially time, needed to do this comprehensively are usually insufficient. It is important to realise that an unquestioning belief that observational studies are the best source of harmful effects data simply because they are not RCTs can be a pitfall. It is essential to think carefully about the review question before widening the inclusion criteria to include non-randomised study designs. Some harmful effects, such as very rare events or those emerging in the long-term, are unlikely to be addressed adequately in RCTs. But, even if observational studies are appropriate to the review question researchers should be prepared for the difficulty of interpreting observational study data outweighing the anticipated benefits. The importance of quality assessment of RCTs in systematic reviews of effectiveness is well established [ 19 ], but debate continues over the usefulness of checklists and scales. Quality assessment of other study designs in systematic reviews is far less well developed [ 20 ]. Although the feasibility of creating one quality checklist to apply to various study designs has been explored [ 21 ], and research has gone into developing an instrument to measure the methodological quality of observational studies [ 22 ], and a scale to assess the quality of observational studies in meta-analyses [ 23 ], there is as yet no consensus on how to synthesise information about quality from a range of study designs within a systematic review. Our appraisal of our reviews has shown that these difficulties are compounded when reviewing data on harms. It is essential that quality assessment is able to discriminate poor from better quality studies of harmful effects. Levels of evidence hierarchies have several shortcomings. The hierarchy of evidence is not always the same for all harmful or beneficial outcomes. For example, an RCT with adequate internal validity but limited sample size or follow-up may be a less reliable source of information about relatively uncommon harmful effects emerging in the long-term than a large well-conducted cohort study with many years of follow-up. Another problem with ranking evidence in a hierarchy is that different dimensions of quality get condensed into a single grade, resulting in a loss of information. Furthermore, the dimensions included in current hierarchies may not be the most important in terms of reflecting the reliability of a particular study's findings [ 24 ]. Researchers need to clarify  a priori  what exactly they need to glean from their quality assessment of the primary studies in their own review of harmful effects and it may be necessary to differentiate clearly between internal and external validity. We suggest that further research is needed to collate, assimilate and build on the existing information relevant to systematically reviewing primary studies for harmful effects of health care interventions. This should include a review of the literature pertinent to the methodology of incorporating evidence of harmful effects in systematic reviews; a description and categorisation of the methods used in systematic reviews published to date, and any evidence from methodological research on which they are based; and the development of quality assessment methods.
Discussion
Our experience of reviewing harmful effects mirrors that of other researchers in that a significant investment of effort failed to yield significant new information [ 6 , 17 ].A focused review question is standard practice for assessing beneficial outcomes in systematic reviews and should also be so when reviewing harms. Researchers conducting reviews need to make sure that they address a well-formulated question about harms that are likely to impact on clinical decisions. Focusing a review question about harmful effects will not necessarily mean restricting it to specific adverse events but may mean, for example, addressing a particular issue such as long-term effects, drug interactions, or the incidence of mild effects of importance to patients. If the aim of the research is to look for previously unrecognised harmful effects, analysis of primary surveillance data may be more appropriate than a systematic review [ 18 ]. Researchers also need to be aware that scopes set by external commissioning bodies, despite having consulted with national professional and patient organisations, may not be a suitable question to address in a systematic review. The wisdom of broad and non-specific questions about harmful effects should be questioned because the resources, especially time, needed to do this comprehensively are usually insufficient.It is important to realise that an unquestioning belief that observational studies are the best source of harmful effects data simply because they are not RCTs can be a pitfall. It is essential to think carefully about the review question before widening the inclusion criteria to include non-randomised study designs. Some harmful effects, such as very rare events or those emerging in the long-term, are unlikely to be addressed adequately in RCTs. But, even if observational studies are appropriate to the review question researchers should be prepared for the difficulty of interpreting observational study data outweighing the anticipated benefits.The importance of quality assessment of RCTs in systematic reviews of effectiveness is well established [ 19 ], but debate continues over the usefulness of checklists and scales. Quality assessment of other study designs in systematic reviews is far less well developed [ 20 ]. Although the feasibility of creating one quality checklist to apply to various study designs has been explored [ 21 ], and research has gone into developing an instrument to measure the methodological quality of observational studies [ 22 ], and a scale to assess the quality of observational studies in meta-analyses [ 23 ], there is as yet no consensus on how to synthesise information about quality from a range of study designs within a systematic review. Our appraisal of our reviews has shown that these difficulties are compounded when reviewing data on harms.It is essential that quality assessment is able to discriminate poor from better quality studies of harmful effects. Levels of evidence hierarchies have several shortcomings. The hierarchy of evidence is not always the same for all harmful or beneficial outcomes. For example, an RCT with adequate internal validity but limited sample size or follow-up may be a less reliable source of information about relatively uncommon harmful effects emerging in the long-term than a large well-conducted cohort study with many years of follow-up. Another problem with ranking evidence in a hierarchy is that different dimensions of quality get condensed into a single grade, resulting in a loss of information. Furthermore, the dimensions included in current hierarchies may not be the most important in terms of reflecting the reliability of a particular study's findings [ 24 ]. Researchers need to clarify  a priori  what exactly they need to glean from their quality assessment of the primary studies in their own review of harmful effects and it may be necessary to differentiate clearly between internal and external validity.a prioriWe suggest that further research is needed to collate, assimilate and build on the existing information relevant to systematically reviewing primary studies for harmful effects of health care interventions. This should include a review of the literature pertinent to the methodology of incorporating evidence of harmful effects in systematic reviews; a description and categorisation of the methods used in systematic reviews published to date, and any evidence from methodological research on which they are based; and the development of quality assessment methods.Conclusions Appraisal of our recent experience highlighted some of the problems inherent in conducting systematic reviews of harmful effects of health care interventions. Such reviews need to address a well-formulated question to facilitate clear decisions about the type of research to include and how best to summarise it, and to avoid repeating what is already known. The review question about harmful effects needs to be relevant to clinical decision-making. A systematic review of the methodology pertinent to systematic reviews of harmful effects is warranted.
Conclusions
Appraisal of our recent experience highlighted some of the problems inherent in conducting systematic reviews of harmful effects of health care interventions. Such reviews need to address a well-formulated question to facilitate clear decisions about the type of research to include and how best to summarise it, and to avoid repeating what is already known. The review question about harmful effects needs to be relevant to clinical decision-making. A systematic review of the methodology pertinent to systematic reviews of harmful effects is warranted.Competing interests None declared.
Competing interests
None declared.Authors' contributions AMB, HMM and NFW conducted the review work described. NFW conceived of the study. NFW and HMM drafted the manuscript. All authors contributed to, read and approved the final manuscript.
Authors' contributions
AMB, HMM and NFW conducted the review work described. NFW conceived of the study. NFW and HMM drafted the manuscript. All authors contributed to, read and approved the final manuscript.Pre-publication history The pre-publication history for this paper can be accessed here:
Pre-publication history
The pre-publication history for this paper can be accessed here: