Investigating the size and mechanisms of the placebo response in clinical trials have relied on experimental procedures that simulate the double-blind randomized placebo-controlled design. However, as the conventional design is thought to elucidate drug rather than placebo actions, different methodological procedures are needed for the placebo response.We reviewed the respective literature for trials designs that may be used to elucidate the size of the placebo response and the mechanisms associated with it.In general, this can be done by either manipulation the information provided to the subjects, or by manipulation the timing of the drug applied. Two examples of each strategy are discussed: the "balanced placebo design" (BDP) and the "balanced cross-over design" (BCD) and their variants are based on false information, while the "hidden treatment" (HT) and the ""delayed response test" (DRT) are based on manipulating the time of drug action. Since most such approaches include deception or incomplete information of the subjects they are suitable for patient only with authorized deception.Both manipulating the information provided to subjects  or manipulating the timing of drug application  allows overcoming some of the restrictions of conventional drug trials in the assessment of the placebo response, but they are feasible mostly in healthy subjects for ethical reasons. Ever since the dawn of the first randomized placebo-controlled trials testing new drugs and treatments in the middle of the last century, and even before , placeboNot only do placebo responses in clinical trials impose significant limits to the testing of new compounds, but they are also linked to the drug adherence and compliance of patients in such trials in a paradoxical way. Patients that adhered to medication instructions by more than 80% showed better survival in a coronary disease study, and poor drug adherence in a myocardial infarction survivor study was associated with a higher risk of mortality  irrespecAttempts to unravel the mechanisms of the placebo response in clinical trials have used meta-analytic approaches of the placebo arm of trials - with mixed results. The drug-placebo difference in randomized controlled trials has been reported to be around 40% in functional disorders ,6. The rBecause of the difficulties to reliably identify placebo responders and predicting placebo response rates in clinical and experimental trials, different methodological attempts have been made to the way (novel) drugs are tested against placebo. At the same time, but to a varying degree, these and other designs have also be used to unravel some of the mechanisms behind the placebo response. This review will discuss experimental designs that may allow characterization and quantification the placebo response. We will introduce two distinctly different approaches to elucidate the mechanisms behind this response, i.e. by manipulating the information provided to participants and by manipulating the timing of drug action, and will present examples for both from the current literature.Throughout this review we will use the following terminology: placebo response is the response observed in patients receiving a placebo treatment in a trial, which is measured by within-group changes. Placebo effect, in contrast, is the effect attributable to the placebo treatment after controlling for other non-specific or extraneous factors such as the natural history and regression to the mean; this is measured by the difference between a placebo group and a no-treatment control group in a trial . Thus, tThe most traditional way to attempt to "control" for placebo response in clinical trials was the use of a crossover design, in which an individual patient serves as her/his own control, reducing the between-subject variability and the number of patients studied. Data from crossover trials can also be used to identify predictors of the placebo response in various clinical conditions.This model was almost completely abolished due to the fact that blinding may be rather difficult in such studies ,18. AlthAnother conventional model to rule out placebo responses is the use of a placebo run-in phase prior to drug and placebo dispensing to identify and exclude placebo responders: Placebo responders tend to exhibit less severe symptoms during run-in -25, and If being a placebo responder is a stable characteristic of an individual patient - which some of the published data question - study designs should take this into account by employing a design with multiple (>2) crossovers between placebo and drug, and to randomize and individualize in a "single subject trials" (SST) the timing for run-in and run-out for each phase . In theoWhile all of these designs may be able to improve drug testing and the distinction between drug and placebo response, most of these designs are not able to characterize the nature of the placebo response and the mechanisms behind it. Even meta-analyses of the placebo arms of drug trials as discussed above will only allow describing potential predictors of the placebo response that need prospective validation in experimental or clinical settings.In laboratory research, a number of experimental designs have been employed that may help to identify and characterize predictors of the placebo response in the future. They have been generated by theoretical issues such as the "additive model" as discussed by Kirsch 2000 . Some ofRandomized and placebo-controlled clinical trials usually operate under the assumption that drug and placebo effects are additive in the drug arm of the study Figure , thus suThis model has, however, been questioned . One argTherefore, Kirsch & Weixel  and KirsThe so-called "balanced placebo design" (BPD) was traditionally used in the testing for expectancy effects of frequently consumed everyday-drugs such as caffeine, nicotine and alcohol , recentlWhile one half of the study sample receives placebo and the other half the drug, half of each group is receiving correct information while the other half is receiving false information on the nature of their study condition (drug or placebo) immediately prior to drug testing, thus allowing to differentiate between the "true" drug response (those receiving the drug but are told they received placebo) and the true placebo response (those receiving placebo but are told they received the drug).The "non-additive model" according to Kirsch  can be tA variant of the BPD is the "half BPD" in which all subjects are given placebos, but half of them receive information that they receive the drug - this is the most common design in current placebo research, as it does not require approval for performing a drug study. However, effective double blinding of such as study is difficult unless - as in a recent test in our laboratory - the subjects and the experimenter(s) conducting the study are made to believe that they participate in a full BPD.As is evident, BPD studies imply deception of the subjects . This liOne of the pitfalls of the BPD is the fact that all subjects are informed  prior to testing whether and what they have received. In sceptical subjects , this may raise doubts about the truth of the information provided and may require additional measures, such as a reliable "cover story" about the rational why the information is given at all. This is usually done by informing them that once the drug is active, the information whether and what they received may no longer be relevant - however, the subjects' acceptance of such information is difficult to prove prior to the test, and its testing afterwards may be subject to another bias.In an attempt to overcome the serious limitations of the BPD, we designed another strategy that may account for some of the BDP limitations. In this case, subjects are divided into four groups, and all are told they participate in a conventional randomized double-blinded and placebo-controlled crossover trial, in which they will receive both the drug and the placebo at two different occasions in a randomized and double-blinded order. However, only group 2/3 will be exposed to drug and placebo in a balanced way, that is half the subjects will receive the drug first and the placebo at the second occasion, while the other half will receive first placebo and then the drug. Group 1 will receive drug - drug, and Group 4 will receive placebo - placebo instead  manipulate the information provided to the subjects, a few others, such as the "hidden treatment" paradigm  manipulaHidden treatment (HT) or covert treatment is an option that may be specifically useful for the test of drug effects in acute and highly symptomatic conditions such as with postoperative pain , anxietyA very unique approach based on such an assumption that has - to our knowledge - never been explored in any placebo testing is the following: Assume that a drug can elicit its action at a predefined time point hours after ingestion, either via a coating technology  or a radAll subjects receive the same information that they will receive either a drug or placebo in a double-blinded fashion, and no information is given about the timing of drug action; instead, a rational ("cover story") is provided for prolonged drug action monitoring, e.g. for 24 hours.Group 1 will receive the drug with immediate action, group 2 the respective placebo. Group 3 receives the delayed response medication, e.g. with drug release after 12 hours. To confirm the "non-additive model" (P1 ≠ P2), M1 + P1 ≠ M2 + P3, with P3 = P2 and M1 = M2.A variant of such a design that intended to elucidate the drug response in a clinical trial in Parkinson's Disease was recently described . While tInvestigating the size and mechanisms of the placebo response in different medical conditions has relied on experimental procedures that simulate the double-blind randomized placebo-controlled design of clinical trials. However, as the conventional design is thought to elucidate drug rather than placebo actions, different methodological procedures are needed for the placebo response. In general, this can be done by either manipulation the information provided to the subjects, or by manipulation the timing of the drug applied. Two examples of each strategy are discussed: the "balanced placebo design" (BDP) and the "balanced cross-over design" (BCD) and their variants are based on false information provided, while the "hidden treatment" (HT) strategy and the "delayed response test" (DRT) are based on altering the timing of drug action. While most such approaches include deception of the subjects this limitation can be overcome by implementing "authorized deception".The authors declare that they have no competing interests.PE and SK developed the idea for this manuscript, reviewed the respective literature, discussed the potentials of alternatives to traditional trial designs, PE wrote the manuscript, and SZ helped writing the manuscript. All authors read and approved the final manuscript.The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/11/90/prepub


A next generation approach to cancer envisions developing preventative vaccinations to stimulate a person's immune cells, particularly cytotoxic T lymphocytes (CTLs), to eliminate incipient tumors before clinical detection. The purpose of our study is to quantitatively assess whether such an approach would be feasible, and if so, how many anti-cancer CTLs would have to be primed against tumor antigen to provide significant protection. To understand the relevant dynamics, we develop a two-compartment model of tumor-immune interactions at the tumor site and the draining lymph node. We model interactions at the tumor site using an agent-based model (ABM) and dynamics in the lymph node using a system of delay differential equations (DDEs). We combine the models into a hybrid ABM-DDE system and investigate dynamics over a wide range of parameters, including cell proliferation rates, tumor antigenicity, CTL recruitment times, and initial memory CTL populations. Our results indicate that an anti-cancer memory CTL pool of 3% or less can successfully eradicate a tumor population over a wide range of model parameters, implying that a vaccination approach is feasible. In addition, sensitivity analysis of our model reveals conditions that will result in rapid tumor destruction, oscillation, and polynomial rather than exponential decline in the tumor population due to tumor geometry. An innovative approach to treating cancer envisions developing preventative anti-cancer vaccines to train a person's immune cells to eliminate early-stage tumors close to genesis. The design of such a treatment strategy requires an understanding of the tumor and immune interactions leading to a successful anti-cancer immune response. To engage this problem, we formulate a mathematical model of the immune response against incipient tumours consisting of as low as hundreds to thousands of cancer cells, which is far below the clinical detection threshold of over 100,000 cells. The model considers the initial stimulation of the immune response and the resulting immune attack on the tumor mass and is formulated as a hybrid agent-based and delay differential equation model. We apply the model to test dynamics over a wide range of dynamic parameters, including immune and tumor cell growth rates and the size of the initial anti-cancer immune population. Our results show that an anti-cancer memory immune cell population of 3% or less can successfully eradicate an incipient tumor population over a wide range of dynamic parameters, indicating that a vaccination approach is feasible. The most effective way to treat a disease is to prevent its development in the first place. Consequently, a next generation approach to cancer treatment envisions developing preventative cancer vaccines that would train a person's immune response to eliminate tumors near inception by stimulating a person's immune system, especially cytotoxic T lymphocytes (CTLs), to attack cancer cells expressing tumor-associated antigens A number of important questions need to be addressed. In particular, is it a realistic goal to immunize a person against cancer, and if so, how many anti-cancer CTLs would be required to provide significant protection against cancer development? There are several conceivable obstacles that could hinder a memory anti-tumor CTL response from being effective. Since cancers develop from colonies of several cells and grow much more gradually than most infectious diseases, developing tumors will only produce a weak antigenic signal, resulting in the activation of only a small fraction of antigen-specific CTLs. Furthermore, activated CTLs will have to encounter the incipient tumor mass in the midst of a large volume of surrounding tissue. It is conceivable that these effects could render an anti-tumor CTL response ineffective. Consequently, the aim of this paper is to assess the feasibility of preventative cancer vaccines from a quantitative perspective.A challenge to designing effective vaccines will be to understand the quantitative dynamics of the protective anti-tumor CTL response that initiates in the lymph node and proceeds to the tissue containing the tumor. CTL responses almost always begin in lymph nodes rather than the affected tissue. In particular, unactivated CTLs spend most of the time circulating through lymph nodes until they are stimulated by antigen-presenting cells, at which point they proliferate and migrate to the affected tissue To model this system, we synthesize current experimental research of CTL dynamics into a hybrid mathematical model consisting of a system of delay differential equations (DDEs) and an agent-based model (ABM). Using this hybrid framework, our model connects the fast-timescale dynamics of immune interactions within lymph nodes with the probabilistic, slow-timescale dynamics of immune surveillance in the tumor microenvironment. We then apply the model to investigate rates of tumor elimination under a wide range of parameters, including tumor and CTL proliferation rates, tumor antigenicity, CTL recruitment rates, and initial CTL populations. In addition, the model sheds light on the scale and nature of the dynamics relevant to an immune response against a clinically undetectable, localized microtumor.Mathematical modeling of tumor growth and tumor immunology has grown rapidly in recent years and several modeling approaches have been applied to understanding these phenomena. For example, a large body of tumor-immune models have been developed using ordinary differential equations (ODEs) The ABM-DDE system we develop in the following sections is most similar in formulation to the hybrid cellular automata-PDE model of Although our model is formulated in a way that could apply to multiple types of tumors by modifying parameters, a large body of experimental research has been directed toward developing treatment strategies for breast cancer, particularly by identifying potential antigens that could be targeted by preventative breast cancer vaccines The paper is organized as follows. In the The hybrid ABM-DDE model was simulated using Matlab R2011b. As shown in Figure 1(right), the tumor begins growing in the periphery. As the tumor size reaches approximately 1,000 cells, more and more immature APCs in the periphery become mature, begin presenting tumor antigen, and migrate to the lymph node  pertains to the number of tumor cells. The first term is the growth rate of the tumor mass. We assume that the growth rate is proportional to the surface area of the tumor, since nearly all growth will happen at or near the surface of the tumor. The second term is the rate that tumor cells are killed by CTLs. The rate of tumor death is proportional to the number of CTLs in contact with the tumor. We assume that all CTLs in the close vicinity of the tumor are in contact with the tumor up to a maximum number 3, the factor The second equation in (1) pertains to CTLs in close vicinity of the tumor. The first term is the rate at which CTLs in the periphery come into the close vicinity of the tumor. Since the ODE does not account for CTL diffusion, we assume all CTLs are evenly distributed throughout the periphery at concentration As with the ABM, the ODE system (1) for the tumor site is coupled with the DDE system (3) for the lymph node. We simulated the combined system using ‘dde23’ in Matlab R2011b. From These observations suggest that the rate of CTL killing of the tumor resembles a differential equation of the form We formulate a model of an anti-tumor memory CTL response elicited by vaccination that will act against an incipient tumor. The primary goal of the model is to assess whether it is realistic for a person's immune system to have a sufficient pool of anti-cancer memory CTLs to significantly reduce the chances of developing cancer. We focus our investigation on breast cancer, since extensive experimental research has been done on growth parameters and tumor sizes, e.g., in Our model suggests that protective immunity against the development of breast cancer could be feasible, because an anti-tumor memory CTL pool of 3% of CTLs could eliminate a developing tumor before it reaches an average size of 1,000 cells, and an anti-tumor memory CTL pool of only 1% of CTLs could eliminate a growing tumor in fewer than 30,000 cells (a diameter of approximately 0.35 mm). These predictions are corroborated by experimental results. In one mouse study, vaccination with telomerase led to telomerase-specific T cell responses of no more than 3% in different mouse strains and had a protective effect against tumor growth In our simulations, the effectiveness of the anti-tumor CTL response depends largely on how quickly CTLs can locate and then eliminate an incipient tumor. The key challenges to locating the tumor are that the incipient tumor expresses a very low antigenic signal to the draining lymph node and it takes up a tiny volume in the tissue. The rapidity of this phase depends primarily on the number of CTLs that become activated and migrate to the periphery. These dynamics are governed mostly by An additional observation from our simulations is that faster growing tumors are often destroyed faster than more slowly growing ones. This result agrees with experimental observations that CTL responses react more effectively to rapidly increasing sources of antigen than to constant or slowly increasing stimuli In our current study, we are interested conditions that allow the CTL response to eliminate a tumor before it reaches a sufficient size or diversity to effectively suppress the immune response, metastasize, or induce angiogenesis. The question remains: At what size or in what time frame is a tumor likely to develop these capabilities, and how would this development impact the immune response. Thus, a direction for future work would be to incorporate the mutation of tumors cells to model the competition between the CTL response and the evolution of the tumor cell population.Another extension of the model is to explicitly incorporate the chemotaxis of CTLs up a signal gradient to the tumor site. We currently model CTL recruitment using an approach analogous to that of As discussed at the beginning of this paper, various models have recently been developed for immune interactions with solid tumors, using both probabilistic agent-based (or cellular automata), deterministic differential equation, and hybrid approaches, e.g., Characterizing tumor-immune dynamics using different modeling perspectives will provide a means of assessing whether it would be feasible to prevent breast cancer using preventative vaccines. Since nearly all relevant cell interactions for protective anti-tumor immunity occur at a level below clinical detection, insights provided by models of immune responses against developing tumors will inform further modeling and experimental directions and aid the advancement of next-generation therapeutic strategies.Our model considers two compartments of immune activity: the site of the incipient tumor in the tissue and a tumor-draining lymph node. We model dynamics of the tumor compartment using a probabilistic ABM. The advantage of the ABM is that it allows us to capture the probabilistic nature and spatial structure of CTL-tumor interactions. In our simulations, cell populations fall under 100,000, making an ABM computationally practical.On the other hand, we model dynamics in the lymph node using a system of DDEs. The advantage of the DDE system is that it allows us to capture the dynamics of an arbitrary number of cells efficiently. In the lymph node, immune cells interact at a faster time scale and exist at orders of magnitude higher concentrations than in the periphery, making an ABM formulation computationally impractical. As a result, we devise a hybrid model connecting an ABM and a DDE system for the tumor site and lymph node.The ABM simulates tumor cells and CTLs at the tumor site. All cells are modeled as spheres of radius At every time step, each tumor cell divides with probability In this model, cancer cells do not move, so a tumor only grows by division, rather than migration. Since our primary focus is to model CTL responses against clinically undetectable microtumors, we assume cancer cells have not acquired the ability to migrate at this stage. In addition, for simplicity, we do not model the non-cancerous tissue surrounding the tumor. Instead, we assume that each new tumor cell can push away the surrounding tissue as it proliferates. Unlike cancer cells, anti-cancer CTLs continually move, and we model their movement using a 3-D Wiener process. At each time step A CTL that is in contact with a cancer cell stops moving and engages the cancer cell. Then, at each time step, the CTL may perform two possible actions: (1) recruit an additional CTL or (2) kill the cancer cell.et al.We model CTL recruitment in a fashion similar to that used in the celluar automata model of Mallet This modeling approach is an approximation of CTL chemotaxis along a chemokine gradient toward a site of CTL stimulation In addition to recruitment, at each time step, a CTL engaging a cancer cell can kill the cancer cell with probability When a CTL starts moving, it accelerates up to the maximum unit standard deviation At each time step, CTLs die with probability For practicality, we simulate CTL-cancer dynamics in a spherical region of radius CTL cloud. Then, at the beginning of each time step, we generate a random number of CTLs distributed uniformly at random throughout the CTL cloud so that no two cells overlap. The number of CTLs generated is given by a Poisson random variable with Poisson parameter Once we obtain The CTL cloud represents the region of CTLs that could emigrate into the region of interest during the next time step. To reliably capture nearly all such CTLs, we need to set the width To investigate the possible strength of a secondary anti-cancer CTL response, we simulate the anti-cancer immune dynamics in a vaccinated host. The anti-tumor CTL response begins when antigen presenting cells (APCs) bearing tumor antigen mature and migrate to the draining lymph node, where they activate memory CTLs that begin to proliferate and emigrate to the site of infection. We model this process in five steps illustrated in Tumor cells produce antigen at the tumor site,APCs pick up tumor antigen and migrate to the draining lymph node,In the lymph node, mature APCs activate memory CTLs that enter a minimal division program of Memory CTLs that have completed the minimal division program become effector CTLs that continue dividing upon further stimulation by APCs,Effector CTLs continually migrate out of the lymph node to the periphery.The model is formulated as the following system of DDEs:The first equation in (3) pertains to APCs waiting in the periphery. These cells are supplied at a constant rate, The second equation in (3) pertains to APCs that have matured, started to present tumor antigen, and migrated to the lymph node. The model accounts for APC maturation, antigen presentation, and migration as one collective event, because APCs that only undergo one or two of the three processes are not pertinent to the dynamics of the model, since they cannot stimulate tumor-specific CTLs. The first term of the equation corresponds to the rate at which these APCs enter the lymph node. The factor The third equation in (3) pertains to memory CTLs in the lymph node. The population is replenished up to an equilibrium capacity, The fourth equation in (3) pertains to effector CTLs that have finished the division program of The last equation in (3) pertains to effector CTLs in the tissue. The first term is the rate at which effector CTLs in the lymph node flow out to the tissue. As with the inflow rate of APCs into the lymph node, this term is scaled by the volume ratio To incorporate the model (3) with the ABM, we translate the DDE system (3) into a system of difference equations evaluated at time steps of length However, since the lymph node contains orders of magnitude higher concentrations of immune cells than the tissue Furthermore, we account for the time-delay terms by incorporating population values from earlier time steps into the difference equation system. In other words, the difference equations for populations at time Let The first equation in (4) pertains, as before, to APCs waiting in the periphery. The first term of the equation is the rate at which new APCs are supplied into the system during one time step As before, the fourth equation in (4) pertains to effector CTLs in the lymph node. The first term is the rate at which activated memory CTLs enter the effector state after completing The advantages of rewriting the DDEs (3) as the difference At time Simulate one time step of the ABM and difference equation system.Repeat from step 1 for the next time step.Parameter estimates for the ABM are shown in For our simulations, we set the time step to et al. estimate that the initial tumor cell doubling time is between 30 and 4800 days  undergo between 7 and 10 initial divisions et al., one T cell and one DC will have To calculate the mass-action coefficient, For the time delays, the duration of one division is between 6 to 12 hours  . In addition, the T cell population doubles approximately every 8 hours during expansion We do not have a good estimate of the antigenicity, For the flow rate of effector CTLs out of the lymph node to the tissue, we assume that effector CTLs that are not being stimulated to divide emigrate from the lymph node at a half life of 1 day, so that the flow rate To derive (2), we take advantage of the connection between random walks on a lattice and the Wiener process. Suppose that at each time step Suppose a stationary CTL accelerates at a constant rate and reaches the maximum velocity at time


Recently, biological applications start to be reimplemented into the applications which exploit many cores of GPUs for better computation performance. Therefore, by providing virtualized GPUs to VMs in cloud computing environment, many biological applications will willingly move into cloud environment to enhance their computation performance and utilize infinite cloud computing resource while reducing expenses for computations. In this paper, we propose a BioCloud system architecture that enables VMs to use GPUs in cloud environment. Because much of the previous research has focused on the sharing mechanism of GPUs among VMs, they cannot achieve enough performance for biological applications of which computation throughput is more crucial rather than sharing. The proposed system exploits the pass-through mode of PCI express (PCI-E) channel. By making each VM be able to access underlying GPUs directly, applications can show almost the same performance as when those are in native environment. In addition, our scheme multiplexes GPUs by using hot plug-in/out device features of PCI-E channel. By adding or removing GPUs in each VM in on-demand manner, VMs in the same physical host can time-share their GPUs. We implemented the proposed system using the Xen VMM and NVIDIA GPUs and showed that our prototype is highly effective for biological GPU applications in cloud environment. Virtualization technology has been widely adopted into computing systems to increase hardware resource utilization and reduce total cost of ownership (TCO). Virtualization technology enables multiple computing environments to be consolidated in a single physical machine. This consolidation brings efficient use of hardware resources and flexible resource provisioning to each computing environment . In clouAlthough virtualization adds an additional software layer over bare metal hardware, the overhead incurred by the additional layer has been reduced by various efforts. Hardware vendors have extended their architectures to support virtualization. In system software area, paravirtualization techniques, such as the Xen , reduce Meanwhile, graphic processing units (GPUs) are recently exploited in high performance computing because a GPU provides one or two orders of magnitude faster computation than a CPU does. The computing capability of GPU has been rapidly improved for a decade. To cover vastly increased demands of 2D and 3D data processing, GPU embeds hundreds of computing cores in a single chipset. In addition, memory bandwidth in GPU is also widened to follow the increased computing power. For example, NVIDIA Tesla C2090, a state-of-the-art GPU device, is equipped with 512 computing cores, 6 GB of dedicated memory and 177 GB/s internal memory bus [With the advance of GPU hardware, general purpose GPU computation on graphic processing units (GPGPU) has been emerging to exploit high performance computing capability of GPU not only for 3D graphic manipulation but also for general-purpose computation. The GPU programming framework such as the CUDA  and the Data copies between host memory and internal memory of GPU could be overhead of GPU computing. In GPU programming models, it is essential to store data on GPU-accessible memory for GPU to manipulate them. Accordingly, the data copies become a critical performance bottleneck of GPU computing. The data copy overhead, however, has been alleviated by making host memory GPU-accessible. In the application processing unit (APU) of AMD, CPU and GPU are integrated in a single chipset, and the two computing units share the same memory controller so that GPU can access the host memory. In addition, various studies have been conducted to lessen the data copy overhead. Despite of the data copy overhead, the performance improvement by GPU computing is significant in many research areas , 13–15.With the trend of GPU computing in high performance computing and biological applications, virtualization needs to be incorporated in the GPU based high performance computing platforms. By providing virtualized GPUs to VMs in cloud computing environment, many biological applications will willingly move into cloud environment to reduce their expenses for computations. However, few studies have been done to use GPU computing in virtual machine environment, and most of them have limitations in terms of performance penalties by GPU virtualization overhead. Therefore, it is also important to minimize the overhead caused by GPU virtualization. In this paper, we propose a BioCloud system architecture that enables VMs to use GPUs in cloud environment. From the high performance computing power of GPUs, biological applications hosted in cloud can also show high performance while minimizing TCO of their computing infrastructure. The proposed system exploits the pass-through mode of PCI express (PCI-E) channel. By making each VM to be able to access underlying GPUs directly, applications can show almost the same performance as when those are in native environment. In addition, our scheme multiplexes GPUs by using hot plug-in/out device features of PCI-E channel. By adding or removing GPUs in each VM in on-demand manner, VMs in the same physical host can time-share its GPUs. The rest of the paper is organized as follows. Virtualization provides an illusion, a VM, to its hosted operating system. By multiplexing underlying hardware resources, a physical host can consolidate multiple VMs simultaneously. The core of virtualization technology is virtual machine monitor (VMM) that is in charge of multiplexing hardware resources such as CPU, memory, and I/O devices to multiple VMs. The common role of VMMs is to provide virtualized resources albeit their implementations diverse from emulation of virtual devices to hardware-assisted virtualization. Hardware-level assists to virtualization take overhead out of a VMM. Previously, a VMM either emulates the behavior of virtualized devices or incorporates with operation systems in VMs. The overhead of these approaches affects the system performance as compared to native environment. To reduce this overhead, hardware vendors extended their CPUs to support virtualization. The Intel VT  and the Reimplementation and low flexibility of GPU APIs: for sharing of GPU among VMs, the management of GPU is concentrated on host operating system or VMM, and the communication between virtual device drivers is highly dependent on the implementation of them. Actually, they have to reimplement all GPU APIs, and this limits flexibility and portability. For example, if a version of GPU APIs is updated or modified, the virtual device drivers should be reengineered according to the changes. Even more, VMs in a physical host should use the same GPU APIs and version as the implemented virtual device drivers. High performance overhead: the architectures on related work adopt the fine-grained time-sharing technique among VMs to share GPU, but these largely depredate the overall performance of GPU by increasing the communication traffic between virtual device drivers and system bus. Moreover, the response times of GPU are not uniform due to the scheduling of VMs. According to the papers, they show 10–40% performance degradation.In GPU virtualization, there have been some related previous research such as the gVirtuS , the GViIn order to overcome these limitations, our scheme uses the direct pass-through approach to use GPU in virtualized environment . By usinThe term GPU was mentioned by NVIDIA Corporation first, when it announced a new graphic controller named GeForce in 1999. In the early 1990s, graphic controllers in general PCs were in charge of simply translating computation results of CPU to visual characters on monitors. After the mid of 1990s, the role of graphic controller started to change into manipulating rich multimedia contents. Multimedia contents usually require lightning effects and texture mapping in order to make the contents more realistic. These computations burden CPU, thus an additional coprocessor, like GPU, is needed to lessen the computation load for multimedia processing in CPU. Although the architecture of GPU is similar to that of CPU, GPU has enhanced processing parallelism as shown in Since computations supported by GPU are specialized to graphical data manipulation, general-purpose computations cannot be easily ported into GPU. To address this obstacle, various general-purpose GPU frameworks have been built. The CUDA of NVIDIA and the OpenCL of Khronos Group are most representative frameworks in the area of general-purpose high performance computing. These frameworks provide an extended C language so that non-graphic-friendly programmers can intuitively translate their computation logics into GPU-friendly ones. From the consensus between GPU performance improvement and easy programming APIs, biological applications prevalently start to use GPUs for their computations. Especially, computations requiring a large amount of data, such as next generation sequencing and protein simulation, are proper targets to exploit GPUs. Manavski and Valle suggested a GPU implementation of Smith-Waterman sequence alignment , and VouOur approach for virtualizing and sharing of GPUs is based on the GPU direct access scheme in our previous work . The preThe direct pass-through approach in virtualized system should be supported by input/output memory management unit (IOMMU) hardware feature. Similar to a traditional memory management unit (MMU) which translates CPU-visible virtual addresses to physical addresses, IOMMU takes care of mapping device-visible I/O addresses to physical addresses and also provides memory protection from misbehaving devices.However, the direct pass-through mechanism has a limitation that the sharing of GPU among VMs is not possible. For example, once a GPU is allocated to a VM at booting time of the VM, it cannot be deallocated until the VM halts. To address this problem, we extend our previous work to have the coarse-grained sharing mechanism based on the hot plug functionality of PCI-E channel. The sharing mechanism enables VM0 to allocate GPUs to VMs and deallocate GPUs from VMs while the VMs are running. To enhance the utilization of GPUs, we add two features: (1) allocating GPUs when a VM actually requires GPUs to compute its data and (2) deallocating GPUs immediately after the computation.The hot plug-in/out mechanism that is adopted in this research is a mechanism to install or remove PCI devices on online. To utilize the hot plug mechanism in virtualized systems, VMM needs the functionality of IOMMU. The system we suppose has GPU installed on a PCI-E channel, and a GPU can be allocated or deallocated by using the PCI-E channel hot plug-in/out. Due to this mechanism, the users of VMs can utilize virtualized GPUs in the same way with native GPUs. GPU-Manager module in the guest VMs, respectively, as shown in In this mechanism, largely, we have two main considerations. One is when to allocate and deallocate GPUs. Although the users of guest VMs can directly request the allocation and deallocation of GPUs, it might decrease usability. It is inconvenient to allocate GPUs manually before invoking a GPU application. We need to provide more convenient interfaces. The other is how to prevent excessive occupation of GPUs by VMs. It is crucial to increase the overall utilization of GPUs in a system. Therefore, we need a compulsory revoking mechanism when a GPU of a guest VM is not utilized for computation. Then, the reclaimed GPU can be used for other guest VMs. For this mechanism, we designed and implemented the GPU-Admin module in the VM0 and the The GPU-Admin is a daemon process in the VM0 and performs allocation and deallocation of GPUs according to the request of the GPU-Managers. It also takes in charge of compulsory revocation of unused GPUs. For this responsibility, it periodically checks the status of GPUs allocated for guest VMs and deallocates them if GPUs are not actually used for computation. In the initial stage of the GPU-Admin, it identifies the number of GPUs installed on a virtualized system and records the PCI-E channel information and the slot ID of each GPU. Using this information, the GPU-Admin registers all GPUs into the GPU pool for later management. All of the registered GPUs are initialized and stay in available state, and one of them is allocated via the GPU-Admin when a guest VM requests a GPU. At this moment, the GPU-Admin stores the virtual machine identification (VMID) and IP address of the guest VM to be referred for usage checking and compulsory revoking operations. After the initialization of GPU pool, the GPU-Admin creates two worker threads: the ManagerListener which handles the requests of the GPU-Manager and the PoolChecker to prevent unnecessary GPU occupation of VMs. GPU allocation request by a user: it is the case that the user of a guest VM explicitly requests the allocation of GPU for computation. Responding to this request, the ManagerListener picks an available GPU from the GPU pool and allocates it into the requested guest VM via hot plug-in mechanism. GPU deallocation request by a user: on contrary to (1), this message is to explicitly deallocate an allocated GPU of guest VM by a user. The ManagerListener deallocates the GPU from the guest VM and registers it again into the GPU pool after reinitialization of the GPU. GPU allocation request by the WrapCUDA library: when a user of a VM invokes a GPU application without an allocated GPU, the WrapCUDA library detects the situation and automatically requests GPU allocation to the GPU-Admin. This request message is delivered implicitly and transparently without recognition of users. The handling of this message is the same with the case (1). GPU deallocation request by the WrapCUDA library: before finishing GPU application, if it calls several specific CUDA library call, the WrapCUDA library requests the deallocation of GPUs implicitly. This mechanism is also performed without the intervention of users. Responding to this message, the ManagerListener deallocates the GPU similarly to the case (2).Disconnection from WrapCUDA library: from the case (3) to (4), the ManagerListener and the WrapCUDA library keeps their connection. If the connection is disconnected for any reasons such as halting or finishing the GPU application, the ManagerListener recognizes it and then deallocates the GPU of the guest VM.The ManagerListener worker is a part of GPU-Admin to accept and handle the requests from guest VMs. The message types of GPU-Managers in guest VMs are five, and the corresponding reactions are as follows.In our prototype, we use TCP/IP network communication method between the GPU-Admin and the GPU-Manager to utilize the virtue of its well-defined and concrete interfaces. Since the communication messages are not incurred frequently, its overhead is negligible. The hot plug-in/out operations of GPUs by the ManagerListener are performed via the management interface of the Xen. In the case that there is no available GPU in the GPU pool, the allocation request is inserted into the waiting queue of the ManagerListener, and the response is blocked until a GPU become available by being deallocated from other guest VMs. If more than one allocation request messages are waiting in the waiting queue, the ManagerListener prioritizes them according to their waiting time.The major role of the PoolChecker is to find the allocated GPUs to guest VMs and decide that they are currently used for computation or not. We add this mechanism to prevent the situation that a guest VM excessively occupies GPUs while it does not actually utilize GPUs for computation. Due to this mechanism, the system can achieve higher overall utilization of GPUs by inhabiting unnecessary occupation of GPUs. The PoolChecker works periodically. When triggered, it checks the GPU pool to search GPUs already allocated to guest VMs and sends a message requesting usage report of GPU to each GPU-Manager of guest VMs. On receipt of this message, each GPU-Manager investigates the usage of its own GPU and replies to the PoolChecker. The PoolChecker tracks the utilization of each GPU and deallocates it forcibly, if it has not been used for computation for a while. This GPU utilization check mechanism of the PoolChecker is a last resort to prevent useless occupation of GPUs by guest VMs. In most cases, where the users of guest VMs are not evil, the unused GPUs are immediately reclaimed by the deallocation message or the disconnection message of the WrapCUDA library. Our prototype sets the trigger period of the PoolChecker as five seconds and the reclaim time limit as ten seconds. These configured values are decided intuitively because we assume that the users of VMs might not need GPUs currently, if the GPUs are not used for computation more than ten seconds. The time-out value of the PoolChecker is easily reconfigurable by editing the configuration file of the PoolChecker. The GPU-Manager is a module working in a guest VM to provide interfaces allocating or deallocating GPUs and report the utilization of its GPU responding to the request of the PoolChecker in the GPU-Admin. The interfaces to request GPU allocation and deallocation can be divided in two. One is a transparent and implicit interface, which is requested by the WrapCUDA library when a user starts and finishes a GPU application. The other is an explicit interface, which is performed by a user action of guest VM.In the initialization stage of the GPU-Manager, it identifies its hostname, local IP address, and GPU-Admin IP address to communicate with the GPU-Admin for allocation and deallocation of GPUs. The GPU-Manager consists of three parts: the AdminListener thread which handles the request of the PoolChecker, the WrapCUDA library hooking the API calls of the native CUDA library to support the implicit allocation or deallocation, and the RequestSender to provide the explicit user interface.AdminListener thread works similarly to the ManagerListener of GPU-Admin. It opens a specified port and waits the GPU usage report request. After allocation of GPUs, the PoolChecker sends periodical requests to check the utilization of GPUs, and the AdminListener responds to this request. To get the utilization of its GPUs, the AdminListener searches the usage of GPU driver module in the proc filesystem, a virtual filesystem presenting process and system information. At least, if more than one GPU application utilizes GPU hardware, the driver module works and the usage of the driver module increases. Therefore, sending this information, the AdminListener reports the usage of GPUs to the PoolChecker. Then, the GPU-Admin can deallocate the GPUs of guest VMs based on this criteria, if it decides that the GPUs are not used for computation.The WrapCUDA library of GPU-Manager is to allocate and deallocate GPUs automatically. To increase the usability of GPUs, the WrapCUDA library enables GPU allocation and deallocation without the intervention of users, when a GPU application starts and finishes. The WrapCUDA library is implemented in a shared library and is preloaded using LD_PRELOAD environment variable after the startup of guest VMs. It is in charge of dynamic allocation of GPU when a user starts a GPU application even if one's VM does not have a GPU. A GPU application should use a specified programming framework such as the CUDA to interface GPU, and several APIs of them are generally called to probe and initialize its GPU in the initial stage of the application. For example, the cudaGetDeviceCount function to get the number of GPUs installed on system and the cudaMalloc to allocate heap memory on GPU are representatives. The WrapCUDA library hooks these kinds of several native CUDA library calls. If a GPU application calls the wrapped CUDA library functions to access GPU, they are redirected to the WrapCUDA library which implements the same function due to LD_PRELOAD environment value. Then, the WrapCUDA library checks whether its guest VM has a GPU and requests GPU allocation if the guest VM has no GPU. After a GPU is allocated, the function of WrapCUDA library executes the native function of the CUDA library. Similarly, a GPU application calls several CUDA APIs to release the resource of GPU at finishing time, and the WrapCUDA library catches these calls before transferring to the native CUDA library for deallocation of GPU from its VM. Although a GPU application might not call these resource release APIs by implementation, the WrapCUDA library can deallocate the GPU of VM immediately, since the connection between the WrapCUDA library and the GPU-Admin is disconnected after the halt of a GPU application. Additionally, as mentioned in In order to interpose the API calls of GPU applications, the WrapCUDA library has to define API functions to catch and embed the implementations of them. These might not be a burden to implement the WrapCUDA library, since the number of APIs to start GPU application is limited to several ones and the patterns to develop a GPU application are regularized. Actually, our prototype implements the API functions listed in To run GPU applications without the WrapCUDA library mechanism, the users of VMs can explicitly request the allocation and deallocation of GPUs. The RequestSender module provides the explicit interfaces. In this case, the GPU explicitly allocated via the RequestSender is specially handled as an exception for the PoolChecker. Therefore, it is not deallocated automatically until the user requests deallocation of the GPU via the RequestSender interface. Since this mechanism enables a user of VM to monopolize GPUs in a system, it should be allowed to trusted users when they need continuous and reliable GPU computing environment. We used the Xen VMM for implementing and evaluating the prototype of the proposed BioCloud architecture. For GPU computation, we used the NVIDIA GPUs and the CUDA programming frameworks. The host machine specification is summarized in The main goals of the prototype evaluation are (1) the GPU virtualization overhead of our scheme, when GPU applications run in virtualized environment as compared to those in native environment, (2) the latencies of GPU hot plug-in/out in VMs, and (3) the benefits by our scheme. In this section, we compare the proposed virtualization scheme with others which are mentioned in the related work and measure the biological application execution time when each application runs in native environment and in virtualized environment using our scheme. x-axis denotes the workloads and biological applications, and the y-axis is normalized execution time which includes disk I/O, CPU computation, and GPU computation. The overhead of our scheme is 3% on average, and the maximum overhead case is the MUMmerGPU with the SSUIS workload by 10%. From our analysis, the most part of overhead is caused by the disk I/O of virtualized system to read workload data files, not by the GPU virtualization. Except the disk I/O time, the GPU computation time is the same as that in the native environment. Comparing the result of Our scheme uses GPU hot plug-in/out to provide sharing of GPUs among VMs. In this evaluation, we measure additional time overhead to hot plug-in/out of GPUs to a VM. E. coli genome data. One execution of the application takes 18 seconds in average. One workload in a VM runs the barraCUDA application five times with sleep time between consecutive runs. The sleep time models the behavior of a user that interprets the result of a previous run and adjusts parameters for the next run. We varied the sleep time to 9, 18, and 36 seconds, and each are 50%, 100%, and 200% of the application execution time, respectively. During this sleep time, an allocated GPU to a VM is returned to the GPU-Admin and could be reallocated to another VM for sharing. Accordingly, we can expect performance improvement by reducing the idle time of GPUs. The main role of our scheme is to make a GPU directly accessible and to coordinate the ownership of a GPU among VMs for time sharing. In order to reveal the effect of GPU sharing in our scheme, we measured the application execution time when multiple VMs share a finite number of GPUs. In this evaluation, we used the barraCUDA application that performs next generation sequencing. Each of the barraCUDA applications performs five-million read mappings of Execution(S9) denotes that the sleep time is 9 seconds. Execution denotes the real execution of four VMs while Theoretical denotes the theoretical execution time in four VMs when a GPU(s) is exclusively used by the VM. Hence, the time is the same as the execution time when four workloads run in a single VM sequentially with the given number of GPUs. Albeit the theoretical execution time is not realizable without our mechanism, the time is used for comparison purpose. The execution time in our scheme is normalized to the theoretical time with the same configuration of sleep time and the number of GPUs, for each. Execution(S9), the sleep time after each completion of application is 9 seconds. When four VMs share one GPU, our scheme shows reduced total execution time by 20% as compared to the theoretical time. When the sleep time increases, our scheme reduces the total execution time because the increased sleep time naturally increases the probability to share the GPU during the sleep time. In case of Execution(S36) with one GPU, our scheme shows reduced total execution time by 53%. As the number of GPUs increases, the performance benefit of our scheme reduces. When the number of VMs is the same as the number of GPUs, our scheme shows no performance benefit. But, we believe that the number of VMs requiring GPUs will be more than the number of GPUs installed in a physical host, since multiple applications and VMs will be consolidated in a single physical machine in the cloud computing environment. In case of Finally, we measure the waiting time, the time to wait for allocation of a GPU, in the same evaluation. When there is no available GPU, a VM should be idle until a GPU becomes available. Accordingly, this additional waiting time can worse the overall execution time of each workload. The modeled scenario in the two previous subsections is synthetic and might be different case by case. Note that the benefit gap and the waiting time in these evaluations will diverse according to the running time and the sleep time of GPU applications. But, at least, we can confirm that our mechanism to virtualize and share GPUs works fine and is reasonably effective for biological GPU applications in BioCloud. In this section, we discuss how our proposed architecture is suitable for biological applications in terms of memory and time of workloads in bioinformatics. One of major characteristics of workloads in bioinformatics is massively data-intensive computing. For example, in a protein sequence alignment workload, a large volume of gnome references are indexed by large-size hashing index (tens of GBs). The workload can perform better when the reference indices are (mostly) in GPU memory and most of them are filled in GPU's internal cache memory . When muOur scheme, however, does not cause those penalties. In our GPU virtualization architecture, a workload can fully exploit the memory in a GPU device while the workload is running. A GPU context switch only occurs after a currently scheduled workload finishes. Accordingly, our scheme can show better performance by eliminating those memory penalties.The other characteristic to note is that bioinformatics workload usually takes a long time from several minutes to hundreds of minutes depending on the workloads , 28. TheOur GPU-virtualization architecture can avoid the frequent context switches in a GPU device thereby showing better performance. Since a GPU is multiplexed at coarse-grained level (at workload), each scheduled workload can fully exploit the cache and memory in a GPU without unnecessary flushing of data until the workload finishes. When the context switch occurs, the previous workload never reuses the warmed-up data in those memory spaces because the workload is finished. As evidence, our architecture shows minimal performance degradation as compared to the other schemes.For higher utilization of systems, the machine virtualization and cloud computing trend will prevail more and more. Besides, the high performance computing based on GPU also has proved its outstanding capabilities for general-purpose computation in many research areas. Especially, the biological applications are outstanding, since their computation can derive a large amount of benefit from many cores of GPUs. For biological GPU applications, we propose a cloud system to exploit GPUs in VM while multiplexing them among VMs and achieving almost the same performance as that with native use of GPUs. Considering the characteristics of bioinformatics workloads which have long execution time, our GPU virtualization mechanism is focused on high GPU computation throughput, rather than sharing GPUs among VMs. Although our prototype is based on the Xen VMM and NVIDIA GPUs, it can be easily ported into other implementations. In the evaluation section, we showed the effectiveness of our mechanism using a modeled scenario. Although the performance benefit and the waiting time can diverse in case by case, we believe that our scheme is highly effective for biological computation using GPUs in cloud environment.


Knowing the mappability of a genome is crucial for the interpretation of massively parallel sequencing experiments. We investigate the properties of the mappability of eukaryotic DNA/RNA both as a whole and at the level of the gene family, providing for various organisms tracks which allow the mappability information to be visually explored. In addition, we show that mappability varies greatly between species and gene classes. Finally, we suggest several practical applications where mappability can be used to refine the analysis of high-throughput sequencing data . This work highlights mappability as an important concept which deserves to be taken into full account, in particular when massively parallel sequencing technologies are employed. The GEM mappability program belongs to the GEM (GEnome Multitool) suite of programs, which can be freely downloaded for any use from its website ( Whole or targeted genome sequencing and re-sequencing, genome-wide profiling of chromatin status and transcription factor binding through ChIPSeq, quantitative estimation of transcriptome abundance through RNASeq, metagenomic and metatranscriptomic sequencing are a just a few examples. HTS technologies typically produce huge amounts of short sequences , with different lengths and characteristics  depending on the technology employed. A fundamental step of the post-sequencing protocol often consists in efficiently aligning the sequence reads onto a genome used as reference. Numerous short read aligners (or mappers) have been developed to achieve this goal with suitable efficiency uniquely mapping reads) from reads matching multiple possible locations in the reference (multiply mapping reads). The latter are often interpreted differently by existing mappers, and sometimes even altogether discarded during the analysis steps subsequent to mapping. The specific interpretation of multiply mapping reads may have implications on the outcome of downstream analysis, in particular when reads are used to obtain quantitative estimates. Typical examples are the determination of transcription factor binding affinity (in ChIPSeq experiments) or transcript abundance (in RNASeq experiments). Thus, for a given HTS run one would aim at maximizing the number of unique mappings obtained, in order to exploit most of the signal from the biological input.Dramatic advances in massively parallel sequencing technologies – which can influence their mappability profiles, we use our mappability method to explore the variations in mappability of different classes of transcripts, taking the GENCODE annotation Third, we investigate how the use of paired-end sequencing or mate-pair libraries relates to mappability. To this end, we predict and quantify how many of the pairs obtained from a typical DNASeq experiment can be rescued by taking advantage of the uniqueness of one of its reads and of the distance information for the pair.In conclusion, we are able to precisely link our findings to the design of a better experiment when the focus is on some particular element in the genome. Our results suggest that the mappability is an important concept to be taken into account when one is trying, for instance, to re-sequence a particular genomic region, or to produce quantitative estimates of transcript abundance from RNASeq experiments.-frequency-mappability or -uniquenessFormally, our definition of the mappability is the following. Given some read length Various methods can be employed to compute the exact frequency (and hence the mappability) of a sequence. The simplest one is a brute-force approach, consisting in the explicit enumeration and counting of all the In the case of mismatches, essentially, the problem of computing the frequencies becomes equivalent to that of exhaustively mapping all the positions in the sequence after a suitable choice of the alignment parameters. At a first glance, this goal would seem well within the range of existing high-performance mappers, which may easily attain speeds of several tens of millions of mapped reads per hour; assuming a mapping speed of H.sapiens genomic locations having ACGGTGGCTCATGCCTGTAATCCCAGCACTTTGGGAGGCCGAGGCGGGCG, which appears 15,323 times with less than There are two algorithmic issues, however. The first one is that most of the available mappers are not based on exhaustive alignment algorithms; this fact implies that they are unable to report the exact count of all the existing matches for a given sequence . The second problem concerns performance: most mapping algorithms are usually optimized to quickly report a few matches and most of them become (much) slower when requested to perform full counting queries. In practice, the speed of all implementations of mapping algorithms always shows some dependency on the number of matches found in the reference; this means that aligning one million reads which all map to thousands of locations in the genome will be (much) slower than aligning one million reads mapping uniquely. Performance degradation may become very relevant in the case of a brute-force enumeration of a sequence in a genome To compute the -frequencies of a sequence of length  up to  mismatches, given an approximation parameter :initialize and zero an array  of  numbersfor all positions  in the sequence doifthen(a) take the -mer  starting at position (b) compute all the positions  in the sequence to which  maps up to  mismatches(c) set(d) ifthenfor all positions  in doifthensetelsesetoutput the array When it is exact for the whole sequence when when the difference between approximated and real frequency can be large in absolute terms, but in the majority of cases it represents only a relatively small fraction of the correct value, since the optimization affects the locations in proportion to their redundancy.else branch of case (2d): this choice regulates the case when a Another important observation is about the presence of the maximum function in the Finally, we emphasize that when the exact mappability is needed the approximation can always be turned off by setting We made use of the latter property to test our approximation, and assess how well it correlates with the exact results. To this end we performed two runs for each example, one with the value of C.elegans .The algorithm presented in the previous Section has been implemented on top of the GEM (GEnome Multitool) library for the indexing of HTS data One relevant difference between the algorithm of last Section and our implementation is that we chose to encode the frequency array as reduced-precision Finally, we note that along with the mappability program we also provide a fast retriever; such a program allows the user to pre-index the obtained mappability files (which are huge), and to subsequently query them in an efficient way.We evaluated the program on an Intel machine with 8 Xeon X5570 CPU cores clocked at 3 GHz, using 8 cores for all the tests. Such a number of processors has been chosen to reduce the wall-clock running time as much as possible; however, it is likely to be suboptimal, since due to the competition in memory access the efficiency of shared-memory distributed computation decreases when many threads are used.H.sapiens genome, such a method is several times slower than the approximate one at Typical running times and memory occupancy for various genomes are shown in The mappability scores produced by our program can be easily converted to other formats, like those suitable for display in the UCSC genome browser D.melanogaster (dm3) and C.elegans (ce6)– we also generated additional tracks which are not part of the UCSC genome browser; they can be obtained from the authors upon request, and subsequently visualized as custom tracks. In case no precomputed track is available for some organism, the users can easily compute it on their own even on modest hardware.We have created mappability tracks for multiple genome species and annotations, including human (both assembly versions Hg18 and Hg19/GRCH37) and mouse (mm9) genomes. At the time of this writing such tracks are integrated in the official UCSC genome browser see , and henproper lengthD.melanogaster is Given a random sequence of length However, this is often not the case. There are two main reasons for this deviation from the expected behavior:genomes are far from being random; they are the result of a long evolutionary history that includes frequent duplications, involving the whole genome or specific regions mismatches are often allowed when mapping HTS reads, and hence in mappability computations. As a result, they lower the number of unique Even relatively long sequences may map to multiple locations, in particular when mismatches are allowed. Using our method, we have sought to characterize the mappability profiles at a whole genome level for four model organisms : in D.melanogaster genome is lower than that of mammalian genomes, the fraction of its mappable genome is not larger. Interestingly, in the fly genome, in contrast with the other genomes analyzed, uniqueness does not seem to increase substantially with the read length when moving from 36 to 75 bp. Indeed, with 75-bp reads and two substitutions, about 20% of the mammalian genome remains unmappable, but the proportion raises to 30% in the case of the fly genome. Even after removing the heterochromatic fraction of the D.melanogaster genome –which mainly corresponds to repetitive sequences and could lead to an underestimation of the uniqueness of the genome– the proportion of uniquely mappable positions in fly remains lower if compared to other model genomes.While there is a negative correlation between the uniqueness of the genome and its repetitive content, the relationship between genome structure and mappability is more complex. For instance, while the proportion of repeats in the D.melanogaster genome might rely on the different nature and proportion of repetitive elements. Indeed, among repeat elements of class I the fly genome does not contain non-LTR retrotransposons such as SINEs D.melanogaster could lead to an over-representation of duplicated One hypothesis which could explain, in part, the lower fraction of uniquely mappable regions observed in In this section we show that computing the mappability of a given genomic region can be critical when inferring single nucleotide variants from HTS data. In fact, these are called by looking at the pileup over the same genomic position, i.e. at the set of symbols which are assigned  to that position.pileup mappability.Before proceeding, we observe that our definition of mappability should be refined if we are to deal with pileups. This is due to the fact that the number of possible bilities . From noIn an ideal case, the pileup would either be composed of one kind of nucleotide only, or it would split into two  subsets, if that position corresponds to an heterozygous locus in a diploid organism. In practice things often go differently, for at least two reasons. First of all there are sequencing errors, which introduce spurious nucleotides in the reads; customarily, one dampens the effect of mis-sequencing by scrutinizing the quality score associated to each letter. Futhermore, due to the fact that genomes contains repeats, each alignment suffers from a certain, unescapable, degree of ambiguity.Namely, when two or more stretches of the sequence under study are very similar to each other, it is not possible to exclude that a read contributing to the pileup in a certain position actually belongs to another portion of the molecule: this leads to occasional mismatches in the alignment, which in turn imply variability in the pileup. It is to quantify the above phenomenon that the concept of (pileup) mappability turns out to be very useful. In fact, if we count the number of symbols different from the reference in the pileup over a certain region of the genome , we expect this quantity to be, on average, inversely related to its uniqueness.H.sapiens chromosomes 15 and 17. We sampled uniformly 100000 positions from each pileup. We then computed the mean heterozygosity (number of symbols in the pileup different from the reference) as a function of the pileup mappability of the position where the read is mapped, grouping together positions with similar levels of mappability.This is indeed what we observe in bona fide diploid SNPs it could be certainly worth excluding those coming from regions of low pileup mappability.The figure clearly suggests that to obtain a set of As we have already pointed out, genome mappability is essential when normalizing counts of reads mapping to the genome in order to obtain quantitative estimates from ChIPSeq experiments. Similarly, transcriptome mappability is also essential when computing normalized counts of transcript abundances after an RNASeq experiment. Here, we sought to apply our method in order to investigate transcriptome mappability.We use the term transcriptome in the sense being used in RNASeq experiments: a transcript annotation of a reference genome, that is a set of genomic coordinates specifying the exonic structure of transcripts , or directly the sequence of such transcripts. Most RNASeq protocols map reads to both the genome and the transcriptome, since transcript sequences across splice junctions are not represented in the sequence of the genome.In this regard, mappability can be understood in two different ways. First, we may compute frequencies by counting Alternatively, we can compute frequencies, and from them the mappability, by counting As a reference transcriptome dataset, we use the GENCODE annotation of the human genome As expected, the mappability score of a particular It is worth noting that, even if in general protein-coding genes all share a high mappability, large paralogous families are likely to originate smaller fractions of uniquely mappable reads. For instance, the fraction of mappable reads for the roughly Pseudo-genes, which still share a relevant sequence similarity with their parent genes, show an even lower mappable fraction. Interestingly, the highest variation observed between the mappability computed with 0 and 2 substitutions concerns pseudogenes . This observation might be due to the fact that duplicated pseudogenes present in many copies escape purifying selection, and thus tend to accumulate more mutations if compared to their parent genes.The long non-coding RNAs (lncRNAs) imes see  showed tTranscriptome mappability can be used to produce more accurate estimates of transcript abundances from RNASeq experiments. As we have seen, some gene classes and families are characterized by low sequence uniqueness, meaning that reads mapping to their sequence are likely to map to (many) other locations in the genome/transcriptome. Different RNASeq mapping and quantification strategies deal with this issue in different ways. They may simply ignore reads that either map to multiple locations A widespread measure of exon, transcript and gene abundance in RNASeq experiments is the so-called RPKM (Reads Per Kilobase of exon per Million mapped reads) For instance, let us assume that two genes exist, both having exactly the same length and originating exactly the same number of reads, but the first being uniquely mappable on its entire length , while the second is uniquely mappabile only in half of its length. Insofar as non-unique reads are ignored, if the two genes are equally expressed the RPKM of the second one will artificially turn out to be the half of that of the first one.In fact, if reads mapping to multiple locations are discarded, only locations with mappability of one can actually contribute to the normalization of expression. In such a case, therefore, we suggest to compute RPKM considering only the fraction of uniquely mappable positions of the feature being quantified . More specifically, we suggest to compute instead the Reads Per Kilobase of Unique exon per Million mapped reads (RPKUM). Separately, we produced the mappability profile of the human genome with identical mapping parameters  as identified by the Ensembl Compara database A striking example is the HLA-A gene  involved in the major histocompatibility complex, which has many paralogs . More geIt should be noted that the importance of computing the uniquely mappable area of a transcript in order to refine its RNA abundance quantification is gaining more and more attention: for instance, a very sophisticated strategy to accurately perform this task has been recently presented in If the chosen RNASeq mapping strategy consists of selecting one mapping location among the many possible ones, mappability can still be used as an additional criterion to help in the selection. Finally, if the contribution of mapped reads to the quantification of transcriptional features  is weighted by the number of mapping locations, the frequencies as computed by our method may also be relevant. Indeed, being not exhaustive, most mapping algorithms are unable to report the exact number of existing matches, and hence the exact frequency value. Thus the frequencies produced by our method  to a repetitive location having a low mappability. This fact is illustrated with the toy example of not enough to establish, when mismatches are considered, that such a location is unique. In this case, a better indicator for the “uniqueness of the read” is likely to be the theoretical mappability of the region, which has to have been computed separately. The existence of this problem is often overlooked.In conclusion, knowing that a read maps uniquely to a location is in general Strictly speaking, this not-so-straightforward connection also complicates the (re)definition of expression measures able to take correctly into account the reduced number of unique reads in repetitive loci; however, neither this observation diminishes the need for such measures, nor it makes less natural and appealing the definition of the RPKUM measure previously presented earlier.In this last section, we examine how the mappability information can be used together with paired-end/mate-pair sequencing to improve the design of an HTS experiment. In particular, we show that when the mappability is known it is possible to tune the insert size in order to maximize the number of sequencing pairs which one will be able to rescue by resorting to the uniqueness of either end.paired-end mappability of a position When using paired-end reads (or mate-pairs), the mapping information of one end can be used to discard spurious mapping positions of the other end if one takes into account the expected distance between ends imposed by the library size used for sequencing. In consequence, when sequencing with a paired-end type strategy, the size see .To facilitate the analysis, we have assumed that the standard deviation of the fragment size in each library is zero .Given the assumptions just presented, it is straightforward to conclude that exactly three cases are possible, as follows  along the whole human genome. To this end, we have estimated which proportion of positions having a non-1 single-end mappability can be rescued completely owing to the fact that both possible pairs in a paired-end experiment are unique. Remarkably, at small library sizes . In particular, the analysis of the mappability profiles of gene families (like the olfactory receptors) and pseudo-genes shows that even long HTS reads are not enough to make some features easily accessible: just using a longer read length may not be enough by itself to completely eliminate the ambiguity which arises from the repetitive nature of some interesting features of the genome.The connection with the design and the analysis of HTS experiments at the level of the single locus is therefore straightforward. We further emphasized it by examining how mappability impacts the study of single-nucleotide polimorphisms, and how it relates to paired-end sequencing schemes.Finally, one could note that the systematic fast computation of mappability may be used in various situations of common interest in biology other than those related to the analysis of HTS data – typical examples being the identification of interesting repeated motifs, or the refinement of primer design. Overall, we believe the present work still far from being exhaustive: more and more practical applications of the study of sequence mappability will certainly follow in the future.


In Kenya, the maternal mortality rate had ranged from 328 to 501 deaths per 100,000 live births over the last three decades. To reduce these rates, the government launched in 2006 a means-tested reproductive health output-based approach (OBA) voucher program that covers costs of antenatal care, a facility-based delivery (FBD) and a postnatal visit in prequalified healthcare facilities. This paper investigated whether women who bought the voucher for their index child and had a FBD were more likely to deliver a subsequent child in a facility compared to those who did not buy vouchers.We used population-based cohort data from two Nairobi slums where the voucher program was piloted. We selected mothers of at least two children born between 2006 and 2012 and divided the mothers into two groups: Index-OBA mothers bought the voucher for the index child (N=352), and non-OBA mothers did not buy the voucher during the study period (N=514). The most complete model indicated that the adjusted odds-ratio of FBD of subsequent child when the index child was born in a facility was 3.89 (p<0.05) and 4.73 (p<0.01) in Group 2.The study indicated that the voucher program improved poor women access to FBD. Furthermore, the FBD of an index child appeared to have a persistent effect, as a subsequent child of the same mother was more likely to be born in a facility as well. While women who purchased the voucher have higher odds of delivering their subsequent child in a facility, those odds were smaller than those of the women who did not buy the voucher. However, women who did not buy the voucher were less likely to deliver in a good healthcare facility, negating their possible benefit of facility-based deliveries. Pathways to improve access to FBD to all near poor women are needed. In Kenya, the national maternal and neonatal mortality rates had remained high over the last three decades. The national mortality rate ranged from 328 to 501 deaths per 100,000 live births in 2003 for the preceding decade. More recent estimates of 488 obtained in 2008 fell within that same range .To reduce the country’s high maternal and child mortality rates in accordance with the Millennium Development Goals, the government of Kenya with funding from the German Development Bank (KfW) launched in 2006 a reproductive health voucher program using an output-based approach (RH-OBA). The RH-OBA program currently operates in Korogocho and Viwandani, the two informal settlements of Nairobi mentioned above and four rural counties. The RH-OBA voucher uses a demand side approach by selling vouchers at nominal fee to low-income women who qualify on their poverty status. Women who purchased the vouchers, in turn, used them to access subsidized healthcare services in accredited public, private or faith-based health facilities of their choice . The ideA facility was accredited if it met criteria set by the public authorities in terms of staffing and quality of care. Limiting the choice of facility to accredited ones helped ensure acceptable quality of care as many facilities operating in the slums provide sub-standard care . ProvideThe Kenya RH-OBA program sold safe motherhood and long-term family planning vouchers in targeted communities but gave a gender-based violence voucher free of charge at the facility to incentivize integrated service delivery, psychosocial support and legal services . This stA previous study based on cross-sectional datasets from those two informal settlements reported that in 2006–08, women who used the voucher almost universally attended at least one antenatal visit (99.5%), and 96% of the women who used the voucher actually delivered in a facility. Among women who did not use the voucher, 94% received antenatal care and 61% delivered in a facility . The natThis study seeked to answer the question: “Were women who bought the RH-OBA voucher for their index child, and delivered in a facility, more likely to deliver a subsequent child in a facility compared to women who did not buy the voucher during the period of the study?”This study data came from Korogocho and Viwandani, two informal settlements where the African Population and Health Research Center (APHRC) has been running the Nairobi Urban Health and Demographic Surveillance System (NUHDSS) since 2003. Each settlement covers about one kilometer square and is located within five to 10 kilometers from the city center. The NUHDSS records demographic events  every four months and detailed household expenditures data once a year. The Viwandani informal settlement neighbors the Industrial Area of Nairobi and was a magnet for young males and relatively educated migrants in search of work while Korogocho was home for more settled families, some ofThe MCH/IVP projects collected information on the child’s place of delivery during the recruitment interview but the information on RH-OBA knowledge and use was collected at the second interview. Hence, information on the RH-OBA voucher was unavailable for mothers who dropped out of the MCH/IVP projects after the first recruitment interview, 22% in the MCH project and 27% in the INDEPTH Vaccination Project. This study included mother-children pairs from the MCH and the IVP datasets from 2006 to 2012. We selected mothers of at least two children, and for whom no important information—like the date of birth of the child or the mother—was missing or implausible. For twin births, we kept only one record knowing the mother’s information was the same and assuming both children were born in the same place. Since this paper focused on longitudinal effect of the voucher, we excluded women who did not buy the voucher for the index child but for the subsequent child.http://www.aphrc.org/insidepage/page.php?app=dataThe APHRC owns the datasets used in this analysis. APHRC has a data sharing policy that enables other researchers to access this dataset and others. APHRC data sharing policy is available at We created three main binary variables for this study.Facility-Based Delivery (FBD) =1 if the child was born in a health facilityIndex child=1 if the child was the first one of that mother recruited in the MCH/IVP study, regardless of the mother’s paritySubsequent child =1 if the child was a younger sibling of the index child from the same motherWe created two subgroups depending on whether the mother purchased a RH-OBA voucher. Group 1 included mothers who bought the RH-OBA when pregnant with the index child. Some mothers in Group 1 bought the vouchers for the subsequent child as well. Group 2 included mothers who did not buy the RH-OBA voucher during the study period. We created other independent variables as well. Maternal education was defined as the highest level attained by the mothers and distributed in three categories: 1) never attended school or did not finish primary school, 2) finished primary school, and 3) secondary or higher education levels. Parity at the time of the birth of the index or subsequent child was the number of children, dead or alive, born to the mother. The site where the household lived was a binary with Korogocho=1. We created also a quality of the facility variable. A facility was deemed good if it was among the 12 accredited by the voucher management agency, belonged to the government, or was operated by a reputable non-governmental or faith-based organization. In contrast, the quality in the other facilities was either unknown or bad. In any case, few healthcare facilities in the slums were able to provide objective quality of care measures aside from their recognized accreditation or affiliation ,14.We created a per capita expenditure variable at household level using household expenditures on food and non-food items  collected in the NUHDSS and adjusted by the size of the household. Children were assumed to consume half an adult’s expenditures. We merged both datasets and selected the information on household expenditures and assets in the year of birth of the child. We imputed the missing variables for the household expenditures per capita using site, village, previous and subsequent per capita expenditures of the household, household size, education level, ethnicity, age, duration of stay in the site of the household head. We used the Stata add-in for Imputation by Chained Equations (ICE) command in Stata 12, set the number of imputation to 5 and average the obtained values. This method of imputing missing values was applied to a cross-section data on the voucher program by some of the same authors Previous analyses indicated that the higher the parity, the less likely the mother was to use voucher for FBD , residenThe analytical dataset included 352 Index-OBA mothers, and 514 Non-OBA mothers. We first conducted descriptive analyses of all dependent and independent variables to gain an understanding of the main features of the dataset. We calculated unadjusted odds of FBD of the subsequent child if the index child was delivered in a facility. We then calculated adjusted odds-ratio with two models. Model 1 controlled for the specific settlements, maternal age, parity, year of birth of the subsequent child, whether the mother was in union, her education, and Model 2 included additionally the monthly household per-capita expenditures. We used Stata12 SE (State College) to merge the datasets and run the analyses. The variable In-Union indicated that mothers who bought the voucher for their index children were less likely to be in union than mothers who did not. The main difference across the groups related to poverty and place of residence. The monthly household per capita expenditures were captured at two different points in time: during the years of the birth of the index and the subsequent children. Initially, the OBA mothers had a per-capita household expenditures of KES 2464 (USD 30.80) which were KES 276 (USD 3.45) lower than the monthly expenditures of KES 2740 (USD 34.25) of the non-OBA mothers (p<0.05). Conversely, the OBA mothers had slightly higher household expenditures (> KES 32) during the year of the birth of the subsequent child but the difference was not statistically significant. Results using imputed values for the missing household expenditures indicate a similar pattern: OBA mothers had significantly lower household expenditures during the index child year of birth but not when the subsequent child was born. The majority of women in the dataset come from Korogocho. Interestingly, six children out of 10 were born in a facility in both groups. However, the trend was mixed: OBA mothers had FBD proportions of 92% for their index child, but only 84% of the subsequent ones. On the contrary, non-OBA mother had FBD proportions of 64% and 71% respectively, which indicated an increasing secular trend toward FBD.The quality of facilities where women delivered followed a trend similar to the FBD: 90% of OBA mothers attended a good quality facility for the index child but only 82% did for the subsequent child. Conversely, those proportions were 53% and 63% among non-OBA mothers. Only 78% of women who bought the RH-OBA voucher for the index child used it for his/her delivery, leaving an open question on the place of delivery of the 22% who did not. Furthermore, only 54% of women who bought the voucher for the index child did so for the subsequent child as well. Among them, 94% had a FBD. In contrast, among women who did not buy the voucher for the subsequent child, only 72% delivered in a facility (p<0.01). No statistical difference was found in the per-capita expenditures or education level of the mothers who bought again the voucher or those who did not. In both groups, the odds that a subsequent child was born in a facility if the index child was also born in a facility were higher . The numFor both models and groups, the year of delivery was statistically significant suggesting a secular trend toward FBD other factors held constant. The odds of FBD were also higher if the mother lived in Korogocho. Other variables were not significant in terms of either magnitude or statistics. Interestingly, while the expenditures per capita variable itself was not significant, its inclusion in the regression improved the precision of the odds-ratio of the main variables. We ran the again the analyses using the imputed values for household expenditures when those were missing. The results were similar to those obtained without imputation in terms of magnitude of the coefficients and statistical significance (not shown).This study found that women who had an index child in a facility were more likely to have a subsequent child also in a facility. However, the odds of FBD were smaller among women who bought the voucher for their index child, compared to those who did not. A higher proportion index-OBA mothers had a subsequent child in a facility compared to non-OBA mothers.Surprisingly, only 54% of the women who bought the voucher for the index child also bought it for the subsequent one. Many reasons might explain this fact. First, the OBA mothers might not have been poor anymore. Second, they might have decided against further participation in the program because of cumbersome checks of eligibility criteria. Finally, the OBA mothers might have given birth again during the year where no new voucher was issued as the stakeholders were in court over management dispute. In any case, the fact that those who bought again the voucher were significantly more likely to deliver in a facility corroborated the positive correlation between access to the OBA voucher and FBD. Note that not all FBD were equal. Women who used the voucher for FBD had to attend accredited facilities, so they probably received appropriate quality care. The same cannot be ascertained about women in the non-OBA group who could deliver in non-accredited facilities. The importance of accredited place of delivery should be emphasized as a previous study showed that although two third of women in Korogocho and Viwandani had delivered in a facility, fewer than half of them were assisted by a skilled birth attendant  who coulAnother difference between the index-OBA and non-OBA groups related to poverty. Mothers who bought the voucher had statistically lower household per capita expenditures than mothers who did (p<0.05) during the year of birth of their index child. However, there was no statistical difference between the two groups of mothers at the birth of the subsequent child. We speculate that the absence of difference may be an artifact of the small sample size. This study observed a secular trend of increasing facility-based deliveries among residents of Korogocho and Viwandani, similar to the previous findings. Nevertheless, among the non-OBA mothers, a substantial proportion of mothers did not have a FBD. A couple of qualitative studies conducted in the same slums in 2005 and 2006 just before the start of the voucher project found that poverty was one of the key factors behind homebirths. Study respondents including women who had a previous life-threatening birthing experience, their partners and other opinion leaders indicated that costs were a major impediment to seeking emergency obstetric care,17. ThosRespondents in both qualitative surveys also pointed out other non-monetary reasons to avoid FBD such as the unfriendly/disrespectful attitude of providers toward them, poor physical access to hospital etc. To address the non-monetary issues, prequalified providers received an output-based reimbursement, thus tying the quality of their service to their financial rewards. Providers who offer quality care in a friendly manner were more likely to attract patients and reap the benefit of the program.This paper presented some limitations, which suggests caution in the interpretation of the results and the policy recommendations. The first one was the small sample sizes of the study. Larger sample size would probably reduce standard errors and improve the statistical significance of the other variables in the models lending strength to the results. A second limitation was the measurement of the socio-economic status of the mother. We approximated it with per-capita household expenditures. This variable was not collected in the IVP survey but in the NUHDSS and only once a year. The expenditures during the year of birth of the child used in this study might fail to capture the household’s standing during of pregnancy. Compounding the issue was the missing values of the expenditures for at least 15% of the sample. We used imputed values and obtained similar results. Other explorations using an asset index or income per capita collected at the same time as the household assets gave similar results suggesting their robustness. In addition, recruitment in the study was suspended twice for about three months each time, but those suspensions were unlikely to bias substantially the result of this study since they were limited in time and no significant event occurred then. Despite issues of sample size, the study has a number of strengths. First, it was population-based cohort data and as such presents reduced instances of sampling bias. In addition, the data include a cohort of mothers with multiple births lending consistency to the information collected. Finally, information related to each child was collected before another was born, reducing the risk of recall bias or confusion between specific place of birth or OBA use for each child. The study indicated that the voucher program improved poor women access to FBD. Furthermore, the FBD of an index child appeared to have a persistent effect, as a subsequent child of the same mother was more likely to be born in a facility as well. While women who purchased the voucher have higher odds of delivering their subsequent child in a facility, those odds were smaller than those of the women who did not buy the voucher. Note, however, that women who did not buy the voucher were also less likely to deliver in a good healthcare facility, which negated their possible benefit of facility-based deliveries. Additional pathways to improve access to FBD to all near poor women are needed.