Background Protein expression vectors that utilize the bacteriophage T7 polymerase/promoter system are capable of very high levels of protein production. Frequently, however, expression from these vectors does not reliably achieve optimal levels of protein production. Strategies have been proposed previously that successfully maintain high expression levels, however we sought to determine the cause of induction failure.
Background
Protein expression vectors that utilize the bacteriophage T7 polymerase/promoter system are capable of very high levels of protein production. Frequently, however, expression from these vectors does not reliably achieve optimal levels of protein production. Strategies have been proposed previously that successfully maintain high expression levels, however we sought to determine the cause of induction failure.Results We demonstrated that decreases in protein overproduction levels are not due to significant plasmid loss nor to mutations arising on the plasmid, but instead largely are attributable to chromosomal mutations that diminish the level of functional T7 RNA polymerase, resulting in decreased expression from the plasmid. Isolation of plasmid DNA from non-expressing strains and reintroduction of the plasmid into a T7 RNA polymerase-producing strain such as BL21(λDE3) reproducibly restored high level protein production.
Results
We demonstrated that decreases in protein overproduction levels are not due to significant plasmid loss nor to mutations arising on the plasmid, but instead largely are attributable to chromosomal mutations that diminish the level of functional T7 RNA polymerase, resulting in decreased expression from the plasmid. Isolation of plasmid DNA from non-expressing strains and reintroduction of the plasmid into a T7 RNA polymerase-producing strain such as BL21(λDE3) reproducibly restored high level protein production.Conclusions Our results suggest that a major contributing factor to decreased expression levels in T7 based systems is chromosomal mutation resulting in loss of functional T7 RNA polymerase. Consistent with this hypothesis, we found that optimal protein overproduction was obtained reproducibly from T7 promoters using freshly transformed cells that had not been subjected to outgrowth during which mutations could accumulate.
Conclusions
Our results suggest that a major contributing factor to decreased expression levels in T7 based systems is chromosomal mutation resulting in loss of functional T7 RNA polymerase. Consistent with this hypothesis, we found that optimal protein overproduction was obtained reproducibly from T7 promoters using freshly transformed cells that had not been subjected to outgrowth during which mutations could accumulate.

Background Often the first step in protein purification is production of the protein of interest in  Escherichia coli  using a gene expression system that allows selective overexpression of a cloned gene [ 1 ]. In the most desirable situation, expression of the cloned gene will consume the majority of the cellular resources, such that at the time of harvest the target protein makes up the bulk of the cell content. Complete purification of the protein then requires fewer selective extraction procedures and results in a more concentrated product. It is clear that optimizing the overexpression step is an effective method for maximizing and facilitating protein purification. One commonly used system for achieving high levels of target protein production depends on the extremely selective nature of the bacteriophage T7 RNA polymerase for specific promoters [ 2 - 4 ]. In the pET system, the gene encoding T7 RNA polymerase is usually supplied by the host bacterial cell in the form of a λ lysogen which expresses the polymerase gene under control of the  lacUV5  promoter [ 2 , 3 ]. The target protein is encoded on a plasmid in which the gene has been cloned so that transcription will be driven by the T7 RNA polymerase. Thus, expression of the T7 RNA polymerase gene is controlled by addition of IPTG to the growth media, and in turn, production of the polymerase controls expression of the plasmid-borne target gene. The T7 RNA polymerase system is extremely effective, resulting in very high levels of target protein production, and the controllable promoters allow even the cloning and expression of genes encoding highly toxic proteins in  E. coli  [ 2 , 3 ]. However, anecdotal evidence has suggested that many investigators observe less than optimal expression from this system and even the lack of detectable protein overproduction. In our hands, the level of synthesis of proteins varied, not only with the specific protein encoded on the plasmid but also from sample to sample of the same protein. Although poor protein synthesis, or complete lack of protein production, appears to be a common problem, strategies to cope with it are varied. Some protocols suggest "pre-induction" tests to determine specific colonies that will result in overproduction [ 5 ]. We found that this test, although simple, is tedious and can fail to identify expressor colonies. However, there are a number of effective strategies that have been proposed over the years [ 2 , 3 , 6 - 8 ]. While we support these recommendations, in this work we sought to determine the underlying cause of reduced expression from pET vectors in order to more effectively prevent problems with poor protein production levels.
Background
Often the first step in protein purification is production of the protein of interest in  Escherichia coli  using a gene expression system that allows selective overexpression of a cloned gene [ 1 ]. In the most desirable situation, expression of the cloned gene will consume the majority of the cellular resources, such that at the time of harvest the target protein makes up the bulk of the cell content. Complete purification of the protein then requires fewer selective extraction procedures and results in a more concentrated product. It is clear that optimizing the overexpression step is an effective method for maximizing and facilitating protein purification.Escherichia coliOne commonly used system for achieving high levels of target protein production depends on the extremely selective nature of the bacteriophage T7 RNA polymerase for specific promoters [ 2 - 4 ]. In the pET system, the gene encoding T7 RNA polymerase is usually supplied by the host bacterial cell in the form of a λ lysogen which expresses the polymerase gene under control of the  lacUV5  promoter [ 2 , 3 ]. The target protein is encoded on a plasmid in which the gene has been cloned so that transcription will be driven by the T7 RNA polymerase. Thus, expression of the T7 RNA polymerase gene is controlled by addition of IPTG to the growth media, and in turn, production of the polymerase controls expression of the plasmid-borne target gene.lacUV5The T7 RNA polymerase system is extremely effective, resulting in very high levels of target protein production, and the controllable promoters allow even the cloning and expression of genes encoding highly toxic proteins in  E. coli  [ 2 , 3 ]. However, anecdotal evidence has suggested that many investigators observe less than optimal expression from this system and even the lack of detectable protein overproduction. In our hands, the level of synthesis of proteins varied, not only with the specific protein encoded on the plasmid but also from sample to sample of the same protein.E. coliAlthough poor protein synthesis, or complete lack of protein production, appears to be a common problem, strategies to cope with it are varied. Some protocols suggest "pre-induction" tests to determine specific colonies that will result in overproduction [ 5 ]. We found that this test, although simple, is tedious and can fail to identify expressor colonies. However, there are a number of effective strategies that have been proposed over the years [ 2 , 3 , 6 - 8 ]. While we support these recommendations, in this work we sought to determine the underlying cause of reduced expression from pET vectors in order to more effectively prevent problems with poor protein production levels.Results We initially examined the conditions required for optimal target protein production from pET plasmids because we observed inconsistent results upon induction of plasmid-borne genes. This inconsistency was most apparent with the plasmid that encoded SecB, pJW25. Following the protocol provided [ 5 ], we streaked a frozen glycerol stock of BL21(λDE3) containing pJW25 for single colonies on LB plates with ampicillin, picked 4 colonies and grew small cultures to analyze overproduction of SecB. Only 1 of the 4 cultures demonstrated detectable overproduction of the SecB protein, and the level of overproduction from that one was not remarkable. Random selection of a large number of additional colonies (greater than 100 total) resulted in only 25–30% that synthesized levels of SecB that were clearly greater than the uninduced control. Further, if a culture that was determined initially to overproduce SecB was inoculated into a larger volume (500 ml), overproduction of SecB in the larger culture was not always detected. As we intended to grow strains for overproduction in a 5 liter fermentor, it was critical that the culture used reproducibly synthesize high levels of target protein. While overproduction of SecB was unreliable from frozen stocks, we found that BL21(λDE3) that had been newly transformed with pJW25 overproduced SecB to a much higher level and that every colony examined synthesized SecB to a similar extent. Continued subculturing, however, resulted in a decrease or loss of overproduction (Figure  1A ). These results suggested that with time the bacteria were losing the ability to overproduce SecB. To verify that our observations were not unique to SecB, we examined a variety of other plasmids for overproduction of  E. coli  proteins in BL21(λDE3); the plasmid pJGGV encodes the secretory protein, proOmpA, pT7SecA produces SecA, the translocation ATPase [ 9 ], and pET11Tus encodes the Tus DNA binding protein [ 10 ]. Similar results were obtained with each of these plasmids. In all cases, newly transformed BL21(λDE3) gave rise to high levels of target protein production, while repeated subculturing decreased the level of expression, eventually to undetectable levels. The initial level of synthesis varied for each protein as did the severity of the effect of subculturing on overproduction. In all cases, however, the degree of overproduction was optimal immediately after transformation. Results are shown for data obtained with the pJW25 (SecB) and pJGGV (proOmpA) plasmids (Figure  1 ). To more accurately determine the effect of subculturing on target protein production, protein levels were monitored as the strains were allowed to grow. Newly transformed BL21(λDE3) were grown as overnight cultures, then repeatedly subcultured as described in Methods. After each subculturing, when cultures reached early log phase (A 600  = 0.4), a sample was removed and tested for the ability to overproduce plasmid-encoded protein. The amount of SecB synthesized by pJW25 decreased gradually with each subculturing, so that by the eighth subculture SecB protein production was minimal (Figure  1A ). At this point, four of eight cultures no longer made detectable levels of SecB; the other four were greatly reduced in  secB  expression (data not shown). The synthesis of proOmpA was even more dramatically affected by subculturing. Induction was abolished very quickly as there was no detectable overproduction in 15 out of 16 cultures after 3 subculturings (Figure  1B ). SecA retained reasonable, but low, protein production levels through 11 subculturings, while Tus synthesis was nearly abolished after only 3 subculturings (data not shown). In all cases, cultures that continued to overproduce protein did so at greatly reduced levels. These results indicated that some proteins are more deleterious to the cell and that continued growth, even without induction, results in alterations to the strain such that overproduction is no longer possible. We considered the following possible explanations for decreased synthesis of target protein with time: 1) the plasmid was not stably maintained in all bacteria, even with the constant presence of selective agent, 2) the plasmid was accumulating mutations that decreased expression levels, or 3) the bacterial strain accumulated mutations that inhibited production of functional T7 RNA polymerase, resulting in decreased or no expression from the plasmid. We examined each of these possibilities. Previous protocols highly recommend that strains be examined for loss of plasmid prior to induction, and suggest that this is the major contributor to decreased expression levels [ 2 , 3 ]. However, our strains were grown with constant selective pressure for plasmid maintainence and subculturings were always made at high dilutions (1:1000) to minimize transfer of any β-lactamase from lysed cells to the fresh culture. Therefore, we felt that, while plasmid loss may be occurring, it probably would not be sufficient to account for the dramatic decreases observed in protein levels. To address this point, we prepared plasmid DNA from strains that had been newly transformed and overproduced proteins at high levels and also from the same strains that had been subcultured repeatedly and no longer synthesized high levels of target protein. We found that the amount of DNA present as observed by agarose gel electrophoresis was quite similar in both strains, regardless of the plasmid examined (data not shown). While the results we obtained are not quantitative, they indicated that plasmid loss alone could not account for the reduced level of expression. To address possible plasmid loss more directly, we compared newly transformed strains and ones that had been subcultured repeatedly for their ability to form individual colonies on LB agar or LB agar containing ampicillin. As β-lactamase is a periplasmic enzyme, it is possible that sufficient leakage of the enzyme occurred to inactivate the ampicillin in liquid culture and allow growth of bacteria that no longer retained plasmid DNA despite the high dilution (1:1000) upon subculturing. If that were the case, we would expect the strain that had been subcultured many times to contain fewer plasmid-containing cells and therefore to form fewer colonies on LB+ampicillin. If no significant plasmid loss were occurring, we would expect the number of colonies to be equal whether or not ampicillin was present. We found that the pJW25-containing strain formed equivalent numbers of colonies on LB and on LB-ampicillin, even after subculturing eight times to result in a culture that did not overproduce SecB (data not shown). Thus, plasmid loss is not sufficient to account for the observed decrease in protein overproduction in these strains. The second hypothesis was that the plasmids had suffered mutations that rendered them unable to overexpress target genes, for example mutations to the T7 RNA polymerase binding site or to the Shine Dalgarno region. To observe such a great decrease in protein production, a mutated plasmid must confer a growth advantage that results in cells containing the mutated plasmid to overtake the culture. Therefore, strains that no longer overproduce significant amounts of target protein should contain mutated plasmid. To test this possibility, we isolated plasmid DNA from strains that no longer synthesized significant amounts of target protein and retransformed BL21(λDE3) with the plasmid DNAs. If plasmid mutations were occurring, we would expect a large number of the transformants to be poor expressors. On the other hand, if plasmid mutations were not responsible for decreased expression, then we would predict that the new transformants would produce high levels of protein. Indeed, this was the result we observed. Every colony examined, from every plasmid tested, synthesized target protein efficiently after retransformation. Results with pJW25 and pJGGV are shown in Figure  1 . The levels of protein production in newly transformed cells were similar to that of the original strain. This result indicated that plasmid mutation was not a significant cause of decreased target protein production from these plasmids. The third hypothesis was that mutation of the bacterial chromosome might lead to decreased production of active T7 RNA polymerase, in turn decreasing expression from the plasmid. We felt this explanation was probable as a single chromosomal mutation that diminished T7 RNA polymerase expression could be sufficient to abolish protein production from all resident plasmids and this mutant could quite likely overtake the culture in a fairly short period of time, even in the presence of antibiotic to maintain the plasmid. We examined this possibility by titering a mutant T7 phage, Δ4107, that requires host production of T7 RNA polymerase for growth [ 2 ]. T7 Δ4107 produced very large, clear plaques when grown on BL21(λDE3) (Figure  2A ). When the host strain was BL21(λDE3) that had been newly transformed with pJW25, the titer decreased to about 25% the original, and the plaques produced were slightly smaller (Figure  2B ). Furthermore, resistant colonies could be seen emerging within the plaques. After BL21(λDE3) containing pJW25 had been subcultured 10 times so that SecB was no longer detectably overproduced, the phage was again titered. The titer was about 25% of the original, but the plaques produced on this strain were very small and cloudy (Figure  2C ). These results are consistent with loss of functional T7 RNA polymerase. We would not predict complete loss of plaque forming ability as the strain culture probably consists of a mixed population at the time of plating; that is, while most cells in the population would be derived from a mutant in which there is little or no functional T7 RNA polymerase, there may be some cells that have not suffered a mutation and would therefore support growth of the phage. We conclude therefore, that a major cause of decreased production of proteins from pET plasmids may be a decreased level of functional T7 RNA polymerase due to mutations in the BL21(λDE3) chromosome. The loss of T7 RNA polymerase activity could be due either to mutation of the polymerase gene or excision of the λ prophage. To distinguish between these possibilities, we performed PCR using primers that amplify an 1100 base pair fragment of the T7 RNA polymerase gene. As template DNA, we used either BL21(λDE3), BL21(λDE3) freshly transformed with pJW25, or BL21(λDE3) containing pJW25 that had been subcultured eight times and no longer produced significant amounts of SecB. In all cases, a DNA product of the correct size was observed, indicating that prophage excision was not the major cause for decreased T7 RNA polymerase activity (data not shown). Therefore, our results suggest that mutation to the prophage, resulting in decreased levels of functional T7 RNA polymerase, was the predominant contributory factor for decreased production of target proteins.
Results
We initially examined the conditions required for optimal target protein production from pET plasmids because we observed inconsistent results upon induction of plasmid-borne genes. This inconsistency was most apparent with the plasmid that encoded SecB, pJW25. Following the protocol provided [ 5 ], we streaked a frozen glycerol stock of BL21(λDE3) containing pJW25 for single colonies on LB plates with ampicillin, picked 4 colonies and grew small cultures to analyze overproduction of SecB. Only 1 of the 4 cultures demonstrated detectable overproduction of the SecB protein, and the level of overproduction from that one was not remarkable. Random selection of a large number of additional colonies (greater than 100 total) resulted in only 25–30% that synthesized levels of SecB that were clearly greater than the uninduced control. Further, if a culture that was determined initially to overproduce SecB was inoculated into a larger volume (500 ml), overproduction of SecB in the larger culture was not always detected. As we intended to grow strains for overproduction in a 5 liter fermentor, it was critical that the culture used reproducibly synthesize high levels of target protein.While overproduction of SecB was unreliable from frozen stocks, we found that BL21(λDE3) that had been newly transformed with pJW25 overproduced SecB to a much higher level and that every colony examined synthesized SecB to a similar extent. Continued subculturing, however, resulted in a decrease or loss of overproduction (Figure  1A ). These results suggested that with time the bacteria were losing the ability to overproduce SecB.To verify that our observations were not unique to SecB, we examined a variety of other plasmids for overproduction of  E. coli  proteins in BL21(λDE3); the plasmid pJGGV encodes the secretory protein, proOmpA, pT7SecA produces SecA, the translocation ATPase [ 9 ], and pET11Tus encodes the Tus DNA binding protein [ 10 ]. Similar results were obtained with each of these plasmids. In all cases, newly transformed BL21(λDE3) gave rise to high levels of target protein production, while repeated subculturing decreased the level of expression, eventually to undetectable levels. The initial level of synthesis varied for each protein as did the severity of the effect of subculturing on overproduction. In all cases, however, the degree of overproduction was optimal immediately after transformation. Results are shown for data obtained with the pJW25 (SecB) and pJGGV (proOmpA) plasmids (Figure  1 ).E. coliTo more accurately determine the effect of subculturing on target protein production, protein levels were monitored as the strains were allowed to grow. Newly transformed BL21(λDE3) were grown as overnight cultures, then repeatedly subcultured as described in Methods. After each subculturing, when cultures reached early log phase (A 600  = 0.4), a sample was removed and tested for the ability to overproduce plasmid-encoded protein. The amount of SecB synthesized by pJW25 decreased gradually with each subculturing, so that by the eighth subculture SecB protein production was minimal (Figure  1A ). At this point, four of eight cultures no longer made detectable levels of SecB; the other four were greatly reduced in  secB  expression (data not shown).secBThe synthesis of proOmpA was even more dramatically affected by subculturing. Induction was abolished very quickly as there was no detectable overproduction in 15 out of 16 cultures after 3 subculturings (Figure  1B ). SecA retained reasonable, but low, protein production levels through 11 subculturings, while Tus synthesis was nearly abolished after only 3 subculturings (data not shown). In all cases, cultures that continued to overproduce protein did so at greatly reduced levels. These results indicated that some proteins are more deleterious to the cell and that continued growth, even without induction, results in alterations to the strain such that overproduction is no longer possible.We considered the following possible explanations for decreased synthesis of target protein with time: 1) the plasmid was not stably maintained in all bacteria, even with the constant presence of selective agent, 2) the plasmid was accumulating mutations that decreased expression levels, or 3) the bacterial strain accumulated mutations that inhibited production of functional T7 RNA polymerase, resulting in decreased or no expression from the plasmid. We examined each of these possibilities.Previous protocols highly recommend that strains be examined for loss of plasmid prior to induction, and suggest that this is the major contributor to decreased expression levels [ 2 , 3 ]. However, our strains were grown with constant selective pressure for plasmid maintainence and subculturings were always made at high dilutions (1:1000) to minimize transfer of any β-lactamase from lysed cells to the fresh culture. Therefore, we felt that, while plasmid loss may be occurring, it probably would not be sufficient to account for the dramatic decreases observed in protein levels. To address this point, we prepared plasmid DNA from strains that had been newly transformed and overproduced proteins at high levels and also from the same strains that had been subcultured repeatedly and no longer synthesized high levels of target protein. We found that the amount of DNA present as observed by agarose gel electrophoresis was quite similar in both strains, regardless of the plasmid examined (data not shown). While the results we obtained are not quantitative, they indicated that plasmid loss alone could not account for the reduced level of expression.To address possible plasmid loss more directly, we compared newly transformed strains and ones that had been subcultured repeatedly for their ability to form individual colonies on LB agar or LB agar containing ampicillin. As β-lactamase is a periplasmic enzyme, it is possible that sufficient leakage of the enzyme occurred to inactivate the ampicillin in liquid culture and allow growth of bacteria that no longer retained plasmid DNA despite the high dilution (1:1000) upon subculturing. If that were the case, we would expect the strain that had been subcultured many times to contain fewer plasmid-containing cells and therefore to form fewer colonies on LB+ampicillin. If no significant plasmid loss were occurring, we would expect the number of colonies to be equal whether or not ampicillin was present. We found that the pJW25-containing strain formed equivalent numbers of colonies on LB and on LB-ampicillin, even after subculturing eight times to result in a culture that did not overproduce SecB (data not shown). Thus, plasmid loss is not sufficient to account for the observed decrease in protein overproduction in these strains.The second hypothesis was that the plasmids had suffered mutations that rendered them unable to overexpress target genes, for example mutations to the T7 RNA polymerase binding site or to the Shine Dalgarno region. To observe such a great decrease in protein production, a mutated plasmid must confer a growth advantage that results in cells containing the mutated plasmid to overtake the culture. Therefore, strains that no longer overproduce significant amounts of target protein should contain mutated plasmid. To test this possibility, we isolated plasmid DNA from strains that no longer synthesized significant amounts of target protein and retransformed BL21(λDE3) with the plasmid DNAs. If plasmid mutations were occurring, we would expect a large number of the transformants to be poor expressors. On the other hand, if plasmid mutations were not responsible for decreased expression, then we would predict that the new transformants would produce high levels of protein. Indeed, this was the result we observed. Every colony examined, from every plasmid tested, synthesized target protein efficiently after retransformation. Results with pJW25 and pJGGV are shown in Figure  1 . The levels of protein production in newly transformed cells were similar to that of the original strain. This result indicated that plasmid mutation was not a significant cause of decreased target protein production from these plasmids.The third hypothesis was that mutation of the bacterial chromosome might lead to decreased production of active T7 RNA polymerase, in turn decreasing expression from the plasmid. We felt this explanation was probable as a single chromosomal mutation that diminished T7 RNA polymerase expression could be sufficient to abolish protein production from all resident plasmids and this mutant could quite likely overtake the culture in a fairly short period of time, even in the presence of antibiotic to maintain the plasmid. We examined this possibility by titering a mutant T7 phage, Δ4107, that requires host production of T7 RNA polymerase for growth [ 2 ]. T7 Δ4107 produced very large, clear plaques when grown on BL21(λDE3) (Figure  2A ). When the host strain was BL21(λDE3) that had been newly transformed with pJW25, the titer decreased to about 25% the original, and the plaques produced were slightly smaller (Figure  2B ). Furthermore, resistant colonies could be seen emerging within the plaques. After BL21(λDE3) containing pJW25 had been subcultured 10 times so that SecB was no longer detectably overproduced, the phage was again titered. The titer was about 25% of the original, but the plaques produced on this strain were very small and cloudy (Figure  2C ). These results are consistent with loss of functional T7 RNA polymerase. We would not predict complete loss of plaque forming ability as the strain culture probably consists of a mixed population at the time of plating; that is, while most cells in the population would be derived from a mutant in which there is little or no functional T7 RNA polymerase, there may be some cells that have not suffered a mutation and would therefore support growth of the phage. We conclude therefore, that a major cause of decreased production of proteins from pET plasmids may be a decreased level of functional T7 RNA polymerase due to mutations in the BL21(λDE3) chromosome.The loss of T7 RNA polymerase activity could be due either to mutation of the polymerase gene or excision of the λ prophage. To distinguish between these possibilities, we performed PCR using primers that amplify an 1100 base pair fragment of the T7 RNA polymerase gene. As template DNA, we used either BL21(λDE3), BL21(λDE3) freshly transformed with pJW25, or BL21(λDE3) containing pJW25 that had been subcultured eight times and no longer produced significant amounts of SecB. In all cases, a DNA product of the correct size was observed, indicating that prophage excision was not the major cause for decreased T7 RNA polymerase activity (data not shown). Therefore, our results suggest that mutation to the prophage, resulting in decreased levels of functional T7 RNA polymerase, was the predominant contributory factor for decreased production of target proteins.Discussion Optimal overproduction is an important first step towards high quantity purification of target proteins. It is critical that one be certain that the culture used will synthesize large quantities of the protein of interest, particularly if it will be used for large scale production procedures, such as growth in a fermentor. When used correctly, the T7 RNA polymerase/promoter system is an excellent choice for such overproduction. However, care must be taken to achieve optimal protein production levels. It had been suggested previously that plasmid loss is the primary cause for decreased expression from target genes in the pET system [ 2 , 3 ]. Our results differed. While plasmid loss may have occurred to a small extent, in the cases analyzed here the principal reason that lowered levels of protein production occurred was that mutations within the lysogen on the host bacterial chromosome resulted in decreased levels of functional T7 RNA polymerase. The fact that strains which no longer produced plasmid-encoded proteins also no longer supported vigorous growth of the mutant T7 phage, Δ4107, indicates that T7 RNA polymerase function was absent or greatly reduced in the majority of the population. Further, PCR amplification of the T7 RNA polymerase gene suggested that the reduction in polymerase function was due to mutation rather than to prophage excision. Surprisingly, there was no clear growth disadvantage to strains carrying pJW25 as assessed by monitoring growth over time (data not shown). We do not think that this finding negates our proposal that chromosomal mutation occurs, however. Rather, this may explain why we see loss of expression more rapidly with some plasmids than with others; for example, synthesis of proOmpA was drastically decreased after only three subculturings. Apparently, even in the uninduced state, sufficient expression of target protein occurred, so that a detrimental effect resulted. If that target protein is sufficiently detrimental, selection for polymerase mutations will occur. For this reason, the steps that are taken to ensure optimal expression from target genes must be directed at limiting opportunities for mutation to the T7 RNA polymerase and for those mutants to overtake the bacterial culture. Two methods have been suggested previously to ensure high levels of protein production; either screening individual colonies for protein synthesis [ 5 ] or testing the population of bacteria to assess the fraction of cells capable of overproduction [ 2 , 3 ]. The first is tedious and unreliable, while the second may be better used as an indicator of the degree of mutation that has occurred. However, as we found in this study, these precautions alone are not sufficient to ensure that the colony used for expression will be the highest protein producer possible. In particular, if any further growth of the strain occurs between the time of testing and the actual induction, mutations may occur that decrease expression levels. Our findings support previous recommendations for growth and storage of BL21(λDE3) containing pET plasmids [ 2 , 3 , 6 , 7 ] as the conditions described would limit mutation of the T7 RNA polymerase gene. Specifically, plasmid-containing BL21 (λDE3) grown to saturation in rich media will allow basal expression of the target gene [ 3 ]. Depending on the toxicity of the protein thus produced, these conditions will select for random mutations to the chromosomally encoded T7 RNA polymerase. Our studies also support the use of bacterial strains that contain the T7 lysozyme as the lysozyme is an inhibitor of the T7 RNA polymerase and will thus decrease uninduced expression even further [ 3 ]. We found that the quickest and most reliable method for obtaining optimal protein production was to freshly transform BL21(λDE3) with the desired plasmid, and use a colony directly from the transformation plate for expression studies with only a single subculture of an overnight culture. Should decreased levels of target protein production be observed, the problem can be simply remedied by isolating plasmid DNA from poor expressors and retransforming BL21(λDE3). However, we acknowledge that this approach may not be realistic for very large scale production facilities. Nevertheless, understanding the molecular basis for decreased protein production will facilitate development of techniques that will minimize conditions that may lead to selection and outgrowth of mutant strains.
Discussion
Optimal overproduction is an important first step towards high quantity purification of target proteins. It is critical that one be certain that the culture used will synthesize large quantities of the protein of interest, particularly if it will be used for large scale production procedures, such as growth in a fermentor. When used correctly, the T7 RNA polymerase/promoter system is an excellent choice for such overproduction. However, care must be taken to achieve optimal protein production levels.It had been suggested previously that plasmid loss is the primary cause for decreased expression from target genes in the pET system [ 2 , 3 ]. Our results differed. While plasmid loss may have occurred to a small extent, in the cases analyzed here the principal reason that lowered levels of protein production occurred was that mutations within the lysogen on the host bacterial chromosome resulted in decreased levels of functional T7 RNA polymerase. The fact that strains which no longer produced plasmid-encoded proteins also no longer supported vigorous growth of the mutant T7 phage, Δ4107, indicates that T7 RNA polymerase function was absent or greatly reduced in the majority of the population. Further, PCR amplification of the T7 RNA polymerase gene suggested that the reduction in polymerase function was due to mutation rather than to prophage excision.Surprisingly, there was no clear growth disadvantage to strains carrying pJW25 as assessed by monitoring growth over time (data not shown). We do not think that this finding negates our proposal that chromosomal mutation occurs, however. Rather, this may explain why we see loss of expression more rapidly with some plasmids than with others; for example, synthesis of proOmpA was drastically decreased after only three subculturings. Apparently, even in the uninduced state, sufficient expression of target protein occurred, so that a detrimental effect resulted. If that target protein is sufficiently detrimental, selection for polymerase mutations will occur. For this reason, the steps that are taken to ensure optimal expression from target genes must be directed at limiting opportunities for mutation to the T7 RNA polymerase and for those mutants to overtake the bacterial culture.Two methods have been suggested previously to ensure high levels of protein production; either screening individual colonies for protein synthesis [ 5 ] or testing the population of bacteria to assess the fraction of cells capable of overproduction [ 2 , 3 ]. The first is tedious and unreliable, while the second may be better used as an indicator of the degree of mutation that has occurred. However, as we found in this study, these precautions alone are not sufficient to ensure that the colony used for expression will be the highest protein producer possible. In particular, if any further growth of the strain occurs between the time of testing and the actual induction, mutations may occur that decrease expression levels. Our findings support previous recommendations for growth and storage of BL21(λDE3) containing pET plasmids [ 2 , 3 , 6 , 7 ] as the conditions described would limit mutation of the T7 RNA polymerase gene. Specifically, plasmid-containing BL21 (λDE3) grown to saturation in rich media will allow basal expression of the target gene [ 3 ]. Depending on the toxicity of the protein thus produced, these conditions will select for random mutations to the chromosomally encoded T7 RNA polymerase. Our studies also support the use of bacterial strains that contain the T7 lysozyme as the lysozyme is an inhibitor of the T7 RNA polymerase and will thus decrease uninduced expression even further [ 3 ].We found that the quickest and most reliable method for obtaining optimal protein production was to freshly transform BL21(λDE3) with the desired plasmid, and use a colony directly from the transformation plate for expression studies with only a single subculture of an overnight culture. Should decreased levels of target protein production be observed, the problem can be simply remedied by isolating plasmid DNA from poor expressors and retransforming BL21(λDE3). However, we acknowledge that this approach may not be realistic for very large scale production facilities. Nevertheless, understanding the molecular basis for decreased protein production will facilitate development of techniques that will minimize conditions that may lead to selection and outgrowth of mutant strains.Conclusion A common difficulty with overproduction of proteins using the T7 RNA polymerase based system is decreased target protein synthesis or even lack of detectable protein production. It is clear that synthesis of proteins, even at low basal levels, leads to the loss of induction capability. Effective strategies have been presented previously to avoid loss of expression, all of which are based on preventing basal expression levels. In this report, we demonstrate that a significant underlying mechanism leading to loss of expression is a decrease in functional T7 RNA polymerase, and selection of mutants unable to express the recombinant gene.
Conclusion
A common difficulty with overproduction of proteins using the T7 RNA polymerase based system is decreased target protein synthesis or even lack of detectable protein production. It is clear that synthesis of proteins, even at low basal levels, leads to the loss of induction capability. Effective strategies have been presented previously to avoid loss of expression, all of which are based on preventing basal expression levels. In this report, we demonstrate that a significant underlying mechanism leading to loss of expression is a decrease in functional T7 RNA polymerase, and selection of mutants unable to express the recombinant gene.Materials and methods Bacterial strains and plasmids Escherichia coli  B strain BL21(λDE3) was used for overproduction of proteins from plasmids containing T7 promoters and was obtained from Stratagene. All plasmids are derivatives of pET11 (Stratagene). Plasmids encoding SecB (pJW25) [ 5 ], SecA (pT7SecA) [ 9 ], and Tus (pET11Tus) [ 10 ] were generous gifts from Linda Randall, Bill Wickner, and Thomas Hill, respectively. The plasmid encoding proOmpA (pJGGV) was constructed in this laboratory by PCR amplification of the  ompA  gene from  E. coli  K12 strain MC4100 [ 11 ] using primers ompA-1 (gacctacccgggcat atgaaaaagacagctatcgc ) and ompA-2 (ggtcatcccgggtgatca ttaagcctgcggctgagttac , underlining indicates regions of homology to the  ompA  gene). The PCR product was digested with restriction enzymes NdeI and BclI and cloned into pET11c that had been digested with NdeI and BamHI. Standard protocols were used for PCR, restriction digests, ligations, and transformations [ 12 , 13 ]. Plasmid DNA was recovered from strains using a QiaPrep Spin Miniprep kit (Qiagen) following manufacturer's instructions. Induction and analysis of protein production All strains were grown in LB medium [ 14 ]. When plasmid was present, ampicillin was added to a concentration of 100 μg/ml. Cultures were induced for protein production at an A 600  of 0.4 by addition of IPTG to a final concentration of 1 mM. Growth was allowed to continue for 2 hours after addition of IPTG. Uninduced controls were grown the same except no IPTG was added. Cells were lysed by boiling in SDS [ 14 ], and proteins were analyzed by SDS polyacrylamide gel electrophoresis [ 13 ]. For experiments using newly transformed BL21(λDE3), colonies were picked directly from the transformation plate and inoculated into 5 ml LB containing ampicillin for overnight growth. The overnight culture was diluted 1:1000 into fresh LB with ampicillin and grown to an A 600  of 0.4 for induction. Thus, the bacteria were subcultured only once. For continuous subculturing experiments, samples were removed before addition of IPTG and used to inoculate fresh LB plus ampicillin media at a dilution of 1:1000. T7 phage titering Bacteriophage T7 mutant Δ4107 [ 2 ], which was a generous gift from Dr. William Studier, was grown for single plaques on BL21(λDE3) containing various plasmids using standard phage protocols [ 14 ]. PCR amplification of T7 RNA polymerase Genomic DNA was isolated using the Qiagen DNeasy kit. Oligonucleotides used as primers (t7pol1 – gattaacatcgctaagaacg and t7pol2 – gattcatgtcgatgtcttcc) were obtained from Midland Certified Reagents, Midland, TX. PCR was performed using the FailSafe PCR kit (Epicentre Technologies, Madison, WI) following manufacturer's recommendations. PCR products were visualized by agarose gel electrophoresis.
Materials and methods
Bacterial strains and plasmids Escherichia coli  B strain BL21(λDE3) was used for overproduction of proteins from plasmids containing T7 promoters and was obtained from Stratagene. All plasmids are derivatives of pET11 (Stratagene). Plasmids encoding SecB (pJW25) [ 5 ], SecA (pT7SecA) [ 9 ], and Tus (pET11Tus) [ 10 ] were generous gifts from Linda Randall, Bill Wickner, and Thomas Hill, respectively. The plasmid encoding proOmpA (pJGGV) was constructed in this laboratory by PCR amplification of the  ompA  gene from  E. coli  K12 strain MC4100 [ 11 ] using primers ompA-1 (gacctacccgggcat atgaaaaagacagctatcgc ) and ompA-2 (ggtcatcccgggtgatca ttaagcctgcggctgagttac , underlining indicates regions of homology to the  ompA  gene). The PCR product was digested with restriction enzymes NdeI and BclI and cloned into pET11c that had been digested with NdeI and BamHI. Standard protocols were used for PCR, restriction digests, ligations, and transformations [ 12 , 13 ]. Plasmid DNA was recovered from strains using a QiaPrep Spin Miniprep kit (Qiagen) following manufacturer's instructions.
Bacterial strains and plasmids
Escherichia coli  B strain BL21(λDE3) was used for overproduction of proteins from plasmids containing T7 promoters and was obtained from Stratagene. All plasmids are derivatives of pET11 (Stratagene). Plasmids encoding SecB (pJW25) [ 5 ], SecA (pT7SecA) [ 9 ], and Tus (pET11Tus) [ 10 ] were generous gifts from Linda Randall, Bill Wickner, and Thomas Hill, respectively. The plasmid encoding proOmpA (pJGGV) was constructed in this laboratory by PCR amplification of the  ompA  gene from  E. coli  K12 strain MC4100 [ 11 ] using primers ompA-1 (gacctacccgggcat atgaaaaagacagctatcgc ) and ompA-2 (ggtcatcccgggtgatca ttaagcctgcggctgagttac , underlining indicates regions of homology to the  ompA  gene). The PCR product was digested with restriction enzymes NdeI and BclI and cloned into pET11c that had been digested with NdeI and BamHI. Standard protocols were used for PCR, restriction digests, ligations, and transformations [ 12 , 13 ]. Plasmid DNA was recovered from strains using a QiaPrep Spin Miniprep kit (Qiagen) following manufacturer's instructions.Escherichia coliompAE. coliompAInduction and analysis of protein production All strains were grown in LB medium [ 14 ]. When plasmid was present, ampicillin was added to a concentration of 100 μg/ml. Cultures were induced for protein production at an A 600  of 0.4 by addition of IPTG to a final concentration of 1 mM. Growth was allowed to continue for 2 hours after addition of IPTG. Uninduced controls were grown the same except no IPTG was added. Cells were lysed by boiling in SDS [ 14 ], and proteins were analyzed by SDS polyacrylamide gel electrophoresis [ 13 ]. For experiments using newly transformed BL21(λDE3), colonies were picked directly from the transformation plate and inoculated into 5 ml LB containing ampicillin for overnight growth. The overnight culture was diluted 1:1000 into fresh LB with ampicillin and grown to an A 600  of 0.4 for induction. Thus, the bacteria were subcultured only once. For continuous subculturing experiments, samples were removed before addition of IPTG and used to inoculate fresh LB plus ampicillin media at a dilution of 1:1000.
Induction and analysis of protein production
All strains were grown in LB medium [ 14 ]. When plasmid was present, ampicillin was added to a concentration of 100 μg/ml. Cultures were induced for protein production at an A 600  of 0.4 by addition of IPTG to a final concentration of 1 mM. Growth was allowed to continue for 2 hours after addition of IPTG. Uninduced controls were grown the same except no IPTG was added. Cells were lysed by boiling in SDS [ 14 ], and proteins were analyzed by SDS polyacrylamide gel electrophoresis [ 13 ].For experiments using newly transformed BL21(λDE3), colonies were picked directly from the transformation plate and inoculated into 5 ml LB containing ampicillin for overnight growth. The overnight culture was diluted 1:1000 into fresh LB with ampicillin and grown to an A 600  of 0.4 for induction. Thus, the bacteria were subcultured only once. For continuous subculturing experiments, samples were removed before addition of IPTG and used to inoculate fresh LB plus ampicillin media at a dilution of 1:1000.T7 phage titering Bacteriophage T7 mutant Δ4107 [ 2 ], which was a generous gift from Dr. William Studier, was grown for single plaques on BL21(λDE3) containing various plasmids using standard phage protocols [ 14 ].
T7 phage titering
Bacteriophage T7 mutant Δ4107 [ 2 ], which was a generous gift from Dr. William Studier, was grown for single plaques on BL21(λDE3) containing various plasmids using standard phage protocols [ 14 ].PCR amplification of T7 RNA polymerase Genomic DNA was isolated using the Qiagen DNeasy kit. Oligonucleotides used as primers (t7pol1 – gattaacatcgctaagaacg and t7pol2 – gattcatgtcgatgtcttcc) were obtained from Midland Certified Reagents, Midland, TX. PCR was performed using the FailSafe PCR kit (Epicentre Technologies, Madison, WI) following manufacturer's recommendations. PCR products were visualized by agarose gel electrophoresis.
PCR amplification of T7 RNA polymerase
Genomic DNA was isolated using the Qiagen DNeasy kit. Oligonucleotides used as primers (t7pol1 – gattaacatcgctaagaacg and t7pol2 – gattcatgtcgatgtcttcc) were obtained from Midland Certified Reagents, Midland, TX. PCR was performed using the FailSafe PCR kit (Epicentre Technologies, Madison, WI) following manufacturer's recommendations. PCR products were visualized by agarose gel electrophoresis.Abbreviations used IPTG – isopropyl-β-D-galactoside PCR – polymerase chain reaction LB – Luria-Bertani SDS – sodium dodecyl sulfate
Abbreviations used
IPTG – isopropyl-β-D-galactosidePCR – polymerase chain reactionLB – Luria-BertaniSDS – sodium dodecyl sulfateAuthors' contributions JGGV was responsible for the original observations of inconsistent expression levels and the data for Figure  1 , as well as the agarose gel analysis not shown. AMF performed the experiments for Figure  2  and prepared the manuscript. Both authors read and approved the final manuscript.
Authors' contributions
JGGV was responsible for the original observations of inconsistent expression levels and the data for Figure  1 , as well as the agarose gel analysis not shown. AMF performed the experiments for Figure  2  and prepared the manuscript. Both authors read and approved the final manuscript.


Background In 2002, the World Health Organization published a health system performance ranking for 191 member countries. The ranking was based on five indicators, with fixed weights common to all countries.
Background
In 2002, the World Health Organization published a health system performance ranking for 191 member countries. The ranking was based on five indicators, with fixed weights common to all countries.Methods We investigate the feasibility and desirability of using mathematical programming techniques that allow weights to vary across countries to reflect their varying circumstances and objectives.
Methods
We investigate the feasibility and desirability of using mathematical programming techniques that allow weights to vary across countries to reflect their varying circumstances and objectives.Results By global distributional measures, scores and ranks are found to be not very sensitive to changes in weights, although differences can be large for individual countries.
Results
By global distributional measures, scores and ranks are found to be not very sensitive to changes in weights, although differences can be large for individual countries.Conclusions Building the flexibility of variable weights into calculation of the performance index is a useful way to respond to the debates and criticisms appearing since publication of the ranking.
Conclusions
Building the flexibility of variable weights into calculation of the performance index is a useful way to respond to the debates and criticisms appearing since publication of the ranking.

Background The World Health Organization recently published a performance ranking of the health systems of its 191 member countries, and intends to update it at regular intervals [ 1 - 4 ]. It was based on a framework outlining a set of social goals to which health systems should contribute [ 5 ]. It was argued that systems should contribute to improving population health, be responsive to the people they serve and be financed fairly. Five outcome indicators were defined – the level of population health, inequalities in health, the level of responsiveness, inequalities in responsiveness and fairness in financial contributions. Estimates of attainment on these five indicators were made for the 191 countries that were members of WHO at that time, and a composite (overall) attainment indicator was constructed for each country as a weighted average of attainment on the five individual outcome indicators. Publication of the analytical framework and the resulting ranking provoked considerable comment, and a variety of issues concerning the methodology and country positions in the ranking have been raised. A central component of the methodology for measuring overall attainment was the use of fixed weights, common to all countries, to aggregate the five indicators. This feature has been controversial, with some arguing that people in different cultural and social settings value individual health system goals in different ways [ 6 - 13 ]. The fixed weights had the virtue of being based on expert opinion, having been derived from the valuations of 1,007 respondents – largely health system professionals – to a WHO survey [ 14 ]. However the weights were common to all countries, regardless of their development status and cultural traditions. In this paper we examine the sensitivity of the attainment scores to alternative weighting schemes that allow weights to vary across countries. These country-specific weights may reveal varying objectives of policy makers or constraints under which they operate. Melyn and Moesen [ 15 ] have referred to such weights as 'benefit of the doubt' weights.
Background
The World Health Organization recently published a performance ranking of the health systems of its 191 member countries, and intends to update it at regular intervals [ 1 - 4 ]. It was based on a framework outlining a set of social goals to which health systems should contribute [ 5 ]. It was argued that systems should contribute to improving population health, be responsive to the people they serve and be financed fairly. Five outcome indicators were defined – the level of population health, inequalities in health, the level of responsiveness, inequalities in responsiveness and fairness in financial contributions. Estimates of attainment on these five indicators were made for the 191 countries that were members of WHO at that time, and a composite (overall) attainment indicator was constructed for each country as a weighted average of attainment on the five individual outcome indicators.Publication of the analytical framework and the resulting ranking provoked considerable comment, and a variety of issues concerning the methodology and country positions in the ranking have been raised. A central component of the methodology for measuring overall attainment was the use of fixed weights, common to all countries, to aggregate the five indicators. This feature has been controversial, with some arguing that people in different cultural and social settings value individual health system goals in different ways [ 6 - 13 ]. The fixed weights had the virtue of being based on expert opinion, having been derived from the valuations of 1,007 respondents – largely health system professionals – to a WHO survey [ 14 ]. However the weights were common to all countries, regardless of their development status and cultural traditions.In this paper we examine the sensitivity of the attainment scores to alternative weighting schemes that allow weights to vary across countries. These country-specific weights may reveal varying objectives of policy makers or constraints under which they operate. Melyn and Moesen [ 15 ] have referred to such weights as 'benefit of the doubt' weights.Methods WHO used fixed weights (0.25, 0.25, 0.125, 0.125, 0.25) to aggregate five health system outputs (respectively, the level of population health, inequality in the distribution of health, the level of health system responsiveness, inequality in the distribution of responsiveness, and fairness in financial contributions) into a scalar health system attainment index. Overall attainment ranged from 35.7 (Sierra Leone) to 93.4 (Japan) on a [0–100] scale. We propose here an analytical framework that reduces to the WHO fixed-weight methodology as a special case, but that allows varying degrees of freedom for weights to be defined that – in the sense of Melyn and Moesen – implicitly take into account individual country circumstances. For shorthand, we say countries "choose" such weights, which in reality are determined as solution values of a linear program. If the linear program is a fair representation of the objective function of and constraints faced by decision makers, the weights are indeed "chosen", but even if this condition is not necessarily met, the resulting weights may still be of interest. The extent of freedom to choose weights in a linear program can be set by the analyst. The analyst can enforce fixed weights common to all countries, allow countries complete freedom to choose their own weights, or adopt a middle ground in which countries are granted limited freedom to choose weights within bounds thought to be sensible by experts. A generic statement of the performance evaluation problem in primal-dual linear programming format is: In these programs y is a country's output vector, x is its input vector, Y is the sample output matrix and X is the sample input matrix. In the present context y is (5 × 1), Y is (5 × 191), x is (n × 1) and X is (n × 191), with n to be specified below. The primal program seeks the maximum radial expansion of a country's outputs, provided that it not exceed the standards established by a convex combination (λ ≥ 0, ∑λ = 1) of best-practice countries in the sample. The optimal value of φ provides a distance measure (i.e. an indication of how far a country has to go) to match best practice as observed in the sample. Since φ ≥ 1, the attainment of a country is evaluated as φ -1  ≤ 1. Best practice countries have φ -1  = 1, other countries have φ -1  < 1, so φ provides a basis for acomplete ranking of countries on their relative ability to deliver five health system outputs. The dual program seeks a set of nonnegative weights μ,υ attached to a country's outputs and inputs that maximize its attainment. Each country's output weights are normalized by μy = 1, but each is free to select its own set of nonnegative weights. In constructing the overall attainment index, WHO identified five output indicators and no inputs, preferring to treat each country's health system as a "health output management unit". Consequently each country's input vector is represented as a scalar with unit value. Under these circumstances the performance evaluation problem simplifies to: where ω = υ + υ 0 . The modified primal program seeks the maximum feasible radial expansion of a country's outputs consistent with best practice observed in the sample. The modified dual program seeks a set of nonnegative weights μ for a country's outputs that put it in the best light. A country can be expected to assign relatively large weights to those outputs at which it excels relative to best practice, and relatively low, possibly zero, weights to those outputs at which it lags behind best practice, subject to the normalization μy = 1. (See Annex for a graphical explanation.) By complementary slackness, μ m (Y m λ - φy m ) = 0, m = 1, ..., M, so slack in any element of a country's projected output vector φy implies that the country assigns a zero weight to that output. Since it is unreasonable to allow a country to assign a zero weight to any output deemed sufficiently important to have been included in the WHO performance evaluation exercise, it is desirable to restrict weights in some way. This can be accomplished most easily by appending constraints to the dual side of (2) of the form: γ m  ≥ μ m y m /μy ≥ β m , m = 1, ..., M.     (3) Restrictions (3) place lower and upper bounds on the relative importance of each output (as measured by μ m ) in total output. Implementation of the weight-restricted linear program requires specifying the 2M = 10 parameters γ m , β m . One procedure is to ignore the upper bounds γ m  and set the lower bounds β m  > 0. This eliminates the possibility of a country assigning zero weights to those outputs at which it lags behind best practice. A less arbitrary procedure is to follow Takamura and Tone [ 16 ] by adapting Saaty's Analytical Hierarchy Process (AHP) [ 17 ]. This procedure exploits expert judgment, that could be provided for example by the above-mentioned survey of health system professionals, to set lower bounds β m  > 0 and upper bounds 1 > γ m . Although these bounds are common to all countries, they allow countries limited freedom to select weights appropriate to their circumstances.
Methods
WHO used fixed weights (0.25, 0.25, 0.125, 0.125, 0.25) to aggregate five health system outputs (respectively, the level of population health, inequality in the distribution of health, the level of health system responsiveness, inequality in the distribution of responsiveness, and fairness in financial contributions) into a scalar health system attainment index. Overall attainment ranged from 35.7 (Sierra Leone) to 93.4 (Japan) on a [0–100] scale.We propose here an analytical framework that reduces to the WHO fixed-weight methodology as a special case, but that allows varying degrees of freedom for weights to be defined that – in the sense of Melyn and Moesen – implicitly take into account individual country circumstances. For shorthand, we say countries "choose" such weights, which in reality are determined as solution values of a linear program. If the linear program is a fair representation of the objective function of and constraints faced by decision makers, the weights are indeed "chosen", but even if this condition is not necessarily met, the resulting weights may still be of interest.The extent of freedom to choose weights in a linear program can be set by the analyst. The analyst can enforce fixed weights common to all countries, allow countries complete freedom to choose their own weights, or adopt a middle ground in which countries are granted limited freedom to choose weights within bounds thought to be sensible by experts.A generic statement of the performance evaluation problem in primal-dual linear programming format is:In these programs y is a country's output vector, x is its input vector, Y is the sample output matrix and X is the sample input matrix. In the present context y is (5 × 1), Y is (5 × 191), x is (n × 1) and X is (n × 191), with n to be specified below.The primal program seeks the maximum radial expansion of a country's outputs, provided that it not exceed the standards established by a convex combination (λ ≥ 0, ∑λ = 1) of best-practice countries in the sample. The optimal value of φ provides a distance measure (i.e. an indication of how far a country has to go) to match best practice as observed in the sample. Since φ ≥ 1, the attainment of a country is evaluated as φ -1  ≤ 1. Best practice countries have φ -1  = 1, other countries have φ -1  < 1, so φ provides a basis for acomplete ranking of countries on their relative ability to deliver five health system outputs. The dual program seeks a set of nonnegative weights μ,υ attached to a country's outputs and inputs that maximize its attainment. Each country's output weights are normalized by μy = 1, but each is free to select its own set of nonnegative weights.In constructing the overall attainment index, WHO identified five output indicators and no inputs, preferring to treat each country's health system as a "health output management unit". Consequently each country's input vector is represented as a scalar with unit value. Under these circumstances the performance evaluation problem simplifies to:where ω = υ + υ 0 . The modified primal program seeks the maximum feasible radial expansion of a country's outputs consistent with best practice observed in the sample. The modified dual program seeks a set of nonnegative weights μ for a country's outputs that put it in the best light. A country can be expected to assign relatively large weights to those outputs at which it excels relative to best practice, and relatively low, possibly zero, weights to those outputs at which it lags behind best practice, subject to the normalization μy = 1. (See Annex for a graphical explanation.)By complementary slackness, μ m (Y m λ - φy m ) = 0, m = 1, ..., M, so slack in any element of a country's projected output vector φy implies that the country assigns a zero weight to that output. Since it is unreasonable to allow a country to assign a zero weight to any output deemed sufficiently important to have been included in the WHO performance evaluation exercise, it is desirable to restrict weights in some way. This can be accomplished most easily by appending constraints to the dual side of (2) of the form:γ m  ≥ μ m y m /μy ≥ β m , m = 1, ..., M.     (3)Restrictions (3) place lower and upper bounds on the relative importance of each output (as measured by μ m ) in total output.Implementation of the weight-restricted linear program requires specifying the 2M = 10 parameters γ m , β m . One procedure is to ignore the upper bounds γ m  and set the lower bounds β m  > 0. This eliminates the possibility of a country assigning zero weights to those outputs at which it lags behind best practice. A less arbitrary procedure is to follow Takamura and Tone [ 16 ] by adapting Saaty's Analytical Hierarchy Process (AHP) [ 17 ]. This procedure exploits expert judgment, that could be provided for example by the above-mentioned survey of health system professionals, to set lower bounds β m  > 0 and upper bounds 1 > γ m . Although these bounds are common to all countries, they allow countries limited freedom to select weights appropriate to their circumstances.Results The fixed weights used by WHO)) [ 4 ] to aggregate the five health system indicators gave countries no freedom to choose weights appropriate to their circumstances. We compare the WHO attainment index with three alternative indexes allowing countries varying degrees of freedom to choose weights. The first index is based on the solution to program (2), without weight restrictions, thereby allowing complete freedom to choose. The second index is based on (2), with lower bounds in (3) of β m  = 0.10 on all weights, allowing substantial freedom to choose. The third index is based on (2), with lower and upper bounds in (3) set by a modified AHP procedure. In the modified procedure, "expert opinion" was taken to mean the average values of weights arising from population-representative country surveys, each of which included a module on health system goals [ 9 ]. Respondents were queried about their individual preferences on the five stated health system goals in a total of 51 countries, in some of which multiple surveys were performed, and country means were calculated on the basis of these individual responses [ 18 ]. The survey methods, reliability, validity, representativeness, sample size and respondent characteristics are extensively described in Ustün et al. [ 9 ], and are also reported in summary form in Sadana et al. [ 19 ], Mathers et al. [ 20 ], Mathers et al. [ 21 ] and Sadana et al. [ 22 ]. The survey instruments are available at  . For each output, the lower bound for calculation of the third index was taken as the minimum of the country average weights, and the upper bound the maximum [ 23 ]. Country mean weights and survey types, as well as survey wide maximum and minimum weights are shown in Table  1 . The lower bounds are accordingly β m  = (0.19, 0.17, 0.12, 0.11, 0.22) and the upper bounds γ m  = (0.29, 0.25, 0.18, 0.17, 0.30). This specification allows limited freedom to choose. We refer to the four indexes as WHO, LP1, LP2 and LP3, respectively. Summary statistics of the four attainment indexes appear in Table  2 . The three LP distributions have higher means than the WHO distribution, and two of them have lower dispersion. However increasing restrictions on freedom to choose reduce the mean, and increase the dispersion, of the LP attainment indexes toward the mean and dispersion of the WHO attainment index. Rank correlations between pairs of attainment rankings appear in Table  3 . Despite the distributional changes due to freedom to choose, rank correlations are positive, high and statistically significant. The lowest correlations involve LP1, the index allowing complete freedom to choose. With the exception of LP1, there is strong agreement about the identity of countries in the top and bottom quartiles of the distribution. Japan is ranked #1 and Sierra Leone is ranked #191 on all four indexes. Figure  1  shows plots of WHO attainment scores and rankings versus the three LP attainment scores and rankings. The convergence of the distributions of the LP attainment scores and rankings to the WHO scores and rankings is apparent. Results based on LP1 are unattractive. Over one-third of countries (70 of 191) assign a zero weight to four of the five indicators, and the vast majority of countries assign a weight in excess of 0.9 to either responsiveness distribution or fairness in financial contributions. This means they assign a low or zero weight to population health, the defining goal of the health system, which does not have face validity. In LP1, only Japan, Luxembourg and the United States assign positive weights to all five indicators. Consequently the attainment index is tightly distributed about a very high mean value. The ability to discriminate is sacrificed to freedom to choose, with 75% of countries receiving attainment indexes of 99 or above. Results from LP2 are somewhat more attractive. Nevertheless, when weights are bounded below by 0.1, over three-quarters (147 of 191) of countries assign the minimum weight to four of the five indicators and a 0.6 = 1 - (4 × 0.1) weight to either responsiveness distribution or fairness in financial contributions. The attainment scores are again compressed about a high mean value, and the ability to discriminate is not much improved, with 75% of countries receiving attainment scores of 93 or above. However despite this dramatic compression, the LP2 ranking is globally very similar to the WHO ranking. Eighteen of the countries ranked in the top 20 by WHO appear in the LP2 top 20, and 14 of the countries ranked in the bottom 20 by WHO appear in the LP2 bottom 20. Not surprisingly, the distribution of the attainment scores for LP3 looks even more similar to the distribution of the WHO scores, and has a similar mean and standard deviation. Rank correlation is very high, with only one country falling out of the WHO top 20 and only five countries rising out of the WHO bottom 20. Nonetheless, limited freedom to choose has an important impact on individual countries. The USA, given an ability to emphasize the importance of responsiveness level and responsiveness inequality, and to de-emphasize the importance of health level and inequality in the distribution of health, improves its ranking from #15 to #3. Australia improves from #12 to #7 for similar reasons. Italy, on the other hand, falls from #11 to #18, primarily as a result of the movement of other countries. In LP3, the largest positive changes in ranking are for Mauritius (+25) and Saint Vincent (+22), while the largest negative changes in ranking are for Kazakhstan (-39) and Albania (-36).
Results
The fixed weights used by WHO)) [ 4 ] to aggregate the five health system indicators gave countries no freedom to choose weights appropriate to their circumstances. We compare the WHO attainment index with three alternative indexes allowing countries varying degrees of freedom to choose weights. The first index is based on the solution to program (2), without weight restrictions, thereby allowing complete freedom to choose. The second index is based on (2), with lower bounds in (3) of β m  = 0.10 on all weights, allowing substantial freedom to choose. The third index is based on (2), with lower and upper bounds in (3) set by a modified AHP procedure.In the modified procedure, "expert opinion" was taken to mean the average values of weights arising from population-representative country surveys, each of which included a module on health system goals [ 9 ]. Respondents were queried about their individual preferences on the five stated health system goals in a total of 51 countries, in some of which multiple surveys were performed, and country means were calculated on the basis of these individual responses [ 18 ]. The survey methods, reliability, validity, representativeness, sample size and respondent characteristics are extensively described in Ustün et al. [ 9 ], and are also reported in summary form in Sadana et al. [ 19 ], Mathers et al. [ 20 ], Mathers et al. [ 21 ] and Sadana et al. [ 22 ]. The survey instruments are available at  .For each output, the lower bound for calculation of the third index was taken as the minimum of the country average weights, and the upper bound the maximum [ 23 ]. Country mean weights and survey types, as well as survey wide maximum and minimum weights are shown in Table  1 . The lower bounds are accordingly β m  = (0.19, 0.17, 0.12, 0.11, 0.22) and the upper bounds γ m  = (0.29, 0.25, 0.18, 0.17, 0.30). This specification allows limited freedom to choose. We refer to the four indexes as WHO, LP1, LP2 and LP3, respectively.Summary statistics of the four attainment indexes appear in Table  2 . The three LP distributions have higher means than the WHO distribution, and two of them have lower dispersion. However increasing restrictions on freedom to choose reduce the mean, and increase the dispersion, of the LP attainment indexes toward the mean and dispersion of the WHO attainment index.Rank correlations between pairs of attainment rankings appear in Table  3 . Despite the distributional changes due to freedom to choose, rank correlations are positive, high and statistically significant. The lowest correlations involve LP1, the index allowing complete freedom to choose. With the exception of LP1, there is strong agreement about the identity of countries in the top and bottom quartiles of the distribution. Japan is ranked #1 and Sierra Leone is ranked #191 on all four indexes.Figure  1  shows plots of WHO attainment scores and rankings versus the three LP attainment scores and rankings. The convergence of the distributions of the LP attainment scores and rankings to the WHO scores and rankings is apparent.Results based on LP1 are unattractive. Over one-third of countries (70 of 191) assign a zero weight to four of the five indicators, and the vast majority of countries assign a weight in excess of 0.9 to either responsiveness distribution or fairness in financial contributions. This means they assign a low or zero weight to population health, the defining goal of the health system, which does not have face validity. In LP1, only Japan, Luxembourg and the United States assign positive weights to all five indicators. Consequently the attainment index is tightly distributed about a very high mean value. The ability to discriminate is sacrificed to freedom to choose, with 75% of countries receiving attainment indexes of 99 or above.Results from LP2 are somewhat more attractive. Nevertheless, when weights are bounded below by 0.1, over three-quarters (147 of 191) of countries assign the minimum weight to four of the five indicators and a 0.6 = 1 - (4 × 0.1) weight to either responsiveness distribution or fairness in financial contributions. The attainment scores are again compressed about a high mean value, and the ability to discriminate is not much improved, with 75% of countries receiving attainment scores of 93 or above. However despite this dramatic compression, the LP2 ranking is globally very similar to the WHO ranking. Eighteen of the countries ranked in the top 20 by WHO appear in the LP2 top 20, and 14 of the countries ranked in the bottom 20 by WHO appear in the LP2 bottom 20.Not surprisingly, the distribution of the attainment scores for LP3 looks even more similar to the distribution of the WHO scores, and has a similar mean and standard deviation. Rank correlation is very high, with only one country falling out of the WHO top 20 and only five countries rising out of the WHO bottom 20. Nonetheless, limited freedom to choose has an important impact on individual countries. The USA, given an ability to emphasize the importance of responsiveness level and responsiveness inequality, and to de-emphasize the importance of health level and inequality in the distribution of health, improves its ranking from #15 to #3. Australia improves from #12 to #7 for similar reasons. Italy, on the other hand, falls from #11 to #18, primarily as a result of the movement of other countries. In LP3, the largest positive changes in ranking are for Mauritius (+25) and Saint Vincent (+22), while the largest negative changes in ranking are for Kazakhstan (-39) and Albania (-36).Discussion We began by questioning the appropriateness of the fixed weight approach to aggregating indicators adopted by WHO, on the grounds that fixed weights deny countries at varying stages of development the freedom to choose. We then proposed a sequence of linear programming models that allowed countries varying freedom to choose the weights assigned to their indicators. LP1 allows complete freedom to choose, and generates weights we consider unacceptable, particularly because so many countries give a zero weight to improving health. LP2 allows considerable freedom to choose, but generates many country-specific weights falling outside the range of the within-country means used as bounds in LP3. Clearly, the validity of a procedure that routinely assigns weights out of the range of representative cross-population preferences should be questioned, even without a sophisticated theory of empirical ethics or democratic choice. LP3 applies the AHP procedure to set plausible bounds on weights, and allows limited freedom to choose. It generates a very similar distribution of the attainment index, and a very high linear rank correlation with the WHO ranking. Despite these similarities, we find the LP3 approach intuitively appealing, and are encouraged by its global concordance with the WHO index. However it is fair to ask: if LP3 and WHO generate such similar rankings, why bother? What value does LP3 add? Indeed, individual countries may come to diametrically opposed conclusions about the benefits of LP3 or WHO on the basis of their implied rank changes (e.g. Mauritius vs. Kazakhstan). Howbeit, our first response to the question "why bother" focuses on the distribution of the LP3 weights in comparison to the WHO weights. The WHO weight on responsiveness inequality was 0.125. But the LP3 upper bound of 0.17 is binding on 182 countries, which implicitly want a higher weight on this indicator. The WHO weight on fairness in financial contributions was 0.25. But the LP3 upper bound of 0.30 is binding on 167 countries that want a higher weight. At the other end, the WHO weight on health level is 0.25. But the LP3 lower bound of 0.19 is binding on 170 countries that want a lower weight. The WHO weight on responsiveness level is 0.125. But the LP3 lower bound of 0.12 is binding on 103 countries that want a lower weight. It appears that a majority of countries at all stages of development may implicitly assign greater importance to indicators of health distribution, and less importance to indicators of health level, than the experts whose judgments formed the basis of the original WHO weights. Our second response is more general. By allowing countries limited freedom to choose their weights, LP3 takes a small but nevertheless significant step toward respecting their varying circumstances. While the narrower the bounds on weights, the smaller the step, even the limited freedom embodied in LP3 makes an important difference to some countries.
Discussion
We began by questioning the appropriateness of the fixed weight approach to aggregating indicators adopted by WHO, on the grounds that fixed weights deny countries at varying stages of development the freedom to choose. We then proposed a sequence of linear programming models that allowed countries varying freedom to choose the weights assigned to their indicators. LP1 allows complete freedom to choose, and generates weights we consider unacceptable, particularly because so many countries give a zero weight to improving health. LP2 allows considerable freedom to choose, but generates many country-specific weights falling outside the range of the within-country means used as bounds in LP3. Clearly, the validity of a procedure that routinely assigns weights out of the range of representative cross-population preferences should be questioned, even without a sophisticated theory of empirical ethics or democratic choice.LP3 applies the AHP procedure to set plausible bounds on weights, and allows limited freedom to choose. It generates a very similar distribution of the attainment index, and a very high linear rank correlation with the WHO ranking. Despite these similarities, we find the LP3 approach intuitively appealing, and are encouraged by its global concordance with the WHO index. However it is fair to ask: if LP3 and WHO generate such similar rankings, why bother? What value does LP3 add? Indeed, individual countries may come to diametrically opposed conclusions about the benefits of LP3 or WHO on the basis of their implied rank changes (e.g. Mauritius vs. Kazakhstan).Howbeit, our first response to the question "why bother" focuses on the distribution of the LP3 weights in comparison to the WHO weights. The WHO weight on responsiveness inequality was 0.125. But the LP3 upper bound of 0.17 is binding on 182 countries, which implicitly want a higher weight on this indicator. The WHO weight on fairness in financial contributions was 0.25. But the LP3 upper bound of 0.30 is binding on 167 countries that want a higher weight. At the other end, the WHO weight on health level is 0.25. But the LP3 lower bound of 0.19 is binding on 170 countries that want a lower weight. The WHO weight on responsiveness level is 0.125. But the LP3 lower bound of 0.12 is binding on 103 countries that want a lower weight. It appears that a majority of countries at all stages of development may implicitly assign greater importance to indicators of health distribution, and less importance to indicators of health level, than the experts whose judgments formed the basis of the original WHO weights.Our second response is more general. By allowing countries limited freedom to choose their weights, LP3 takes a small but nevertheless significant step toward respecting their varying circumstances. While the narrower the bounds on weights, the smaller the step, even the limited freedom embodied in LP3 makes an important difference to some countries.Conclusions Building in the flexibility of varying weights might be a useful way for WHO to respond to the debates and criticisms appearing since publication of the ranking. We conclude by speculating that a variant of LP3 incorporating information regarding which weights are binding, and in which direction, might yield even greater benefits in terms of respecting individual circumstances.
Conclusions
Building in the flexibility of varying weights might be a useful way for WHO to respond to the debates and criticisms appearing since publication of the ranking. We conclude by speculating that a variant of LP3 incorporating information regarding which weights are binding, and in which direction, might yield even greater benefits in terms of respecting individual circumstances.Competing interests JAL, CJLM and DBE were part of the team at WHO that developed the methods for the world health system performance rankings published by the Organization in 2000.
Competing interests
JAL, CJLM and DBE were part of the team at WHO that developed the methods for the world health system performance rankings published by the Organization in 2000.Abbreviations AHP (Analytical Hierarchy Process) DEA (Data envelopment analysis) LP (Linear program) LP1 (Linear Program 1) LP2 (Linear Program 2) LP3 (Linear Program 3) WHO (World Health Organization)
Abbreviations
AHP (Analytical Hierarchy Process)DEA (Data envelopment analysis)LP (Linear program)LP1 (Linear Program 1)LP2 (Linear Program 2)LP3 (Linear Program 3)WHO (World Health Organization)Authors' contributions JAL developed methods inspired by the benefit-of-the-doubt concept and drafted an earlier version of the manuscript. CAKL adapted and applied the DEA methods described here to health system performance and drafted the initial version of the current manuscript. CJLM conceived the methods for measuring health system performance. DBE coordinated the research at WHO. All authors revised and approved the manuscript.
Authors' contributions
JAL developed methods inspired by the benefit-of-the-doubt concept and drafted an earlier version of the manuscript. CAKL adapted and applied the DEA methods described here to health system performance and drafted the initial version of the current manuscript. CJLM conceived the methods for measuring health system performance. DBE coordinated the research at WHO. All authors revised and approved the manuscript.Pre-publication history The pre-publication history for this paper can be accessed here:
Pre-publication history
The pre-publication history for this paper can be accessed here:Supplementary Material Additional File 1 Annex.Graphical interpretation of DEA and conceptual links with value theory.doc Click here for file
Supplementary Material
Additional File 1
Annex.Graphical interpretation of DEA and conceptual links with value theory.docClick here for file