A correlation dimension analysis of people’s visual experiential streams captured by a smartphone shows that visual experience is two-scaled with a smaller dimension at shorter length scales than at longer length scales. The bend between the two scales is a phase transition point where the lower scale primarily captures relationships within the same context and the higher dimensional scale captures relationships between different contexts. The dimensionality estimates are confirmed using Takens’ delay embedding procedure on the image stream, while the randomly permuted stream is shown to be space-filling thereby establishing that the two-scaled structure is a consequence of the dynamics. We note that the structure of visual experience closely resembles the structure of another domain of experience: natural language discourse. The emergence of an identical structure across different domains of human experience suggests that the two-scaled geometry reflects a general organizational principle. The widespread availability of sensor technology and the development of networks to accumulate information have given rise to a gold rush of applications that take advantage of our ability to track human behavior. Monitoring patient symptoms and actions as they occur provides a more relevant, sensitive and actionable approach to client care than is provided by traditional assessment methods Dynamical systems analysis techniques are well suited for uncovering patterns in data that are otherwise not revealed by standard statistical methods. In the current study, five participants wore a device around their necks that automatically captured images from their lives for a period of 1–2 weeks. In the following sections, we present what we believe is the first instance of application of dynamical systems techniques on lifelogging data. We use recurrence plots to visualize the pattern of recurrent visits to the same locations in visual context space over time. We then characterize the structure of experience by computing the correlation dimension of the space occupied by the images and show that visual experience has a two-scaled structure for all participants. The two scales are shown to capture different aspects of experience. In the section that follows, we demonstrate the link between the temporal sequence of visual experience and the structure of visual experience as described by the correlation dimension. To do this, we use Takens’ embedding theorem The research protocol was reviewed and approved by the institutional review board (IRB) at the Ohio State University. Written informed consent was obtained from participants.Five participants provided data for this study. Participants AS and NV were recruited to collect image data using an android phone with our custom lifelogging app installed on it. Participants wore the phone around their necks in a pouch attached to a neck strap as they went about their daily lives and the app automatically captured image data. Participants were compensated at the rate of $10 per day. Additionally, three of the authors  also collected data. SD collected data for a period of about two weeks. Whereas SD’s android app used a movement based trigger to capture images, AS and NV used a regular interval setting (∼1 min) to trigger image captures. VS and YZ used Microsoft Research™ SenseCams to capture images at regular intervals of ∼8 seconds.The participants had control over what data they wanted to share with the experimenters. They were instructed on how to delete data off the phone. They were also allowed to turn the app off at any time during the data collection period when they felt the need for privacy. More details about the devices and the app can be found in The image data sets are labeled as follows: AS , NV , SD , YZSC  and VSSC  where “SC” in YZSC and VSSC stands for “SenseCam”. We present results for NV in the main manuscript and the analogous plots for the 4 remaining participants in the supporting materials since we get consistent results across individuals.Entropy calculations based on recurrence patterns in human trajectories have revealed that human mobility is surprisingly predictable Results and discussion. The RPs of all 5 participants' visual experiences using three different image representations are presented in In the next section, we quantify the structure of experience by computing the correlation dimension of visual context space.2 is the most widely used due to its ease of calculation. D2 is a type of fractal dimension Grassberger and Procaccia 2 describes how C(r) scales with r . As we increase r, more points populate the RP and the correlation dimension Dth r see . For N p2 is calculated from the slope of the straight line scaling region of a log[C(r)] versus log(r) plot. The correlation dimension is only defined for r→0 and N→∞. However, in practice, r→0 means r<<L where L is some “natural” scale of the system. So we avoid computing the slope at length scales that are comparable to the length scales of the system. Systems can exhibit different well-defined dimensions at different length scales, as long as these length scales are well separated.DResults and discussion. For all the analyses described in this paper, images were first converted from the RGB space into the HSV space. The images in each data set are then represented by the color correlogram Contexts exhibit two separate scales that are clearly distinguishable, very much like the structure of natural language discourse Again, following To further understand what the two separate scales mean, we calculated the ratio of the number of pairs of images above the bend to the number of pairs of images below the bend in NV's correlation dimension plot as a function of time difference. Specifically, we computed the ratio of the number of pairs of images that are separated by more than a distance of exp (µ+γ) to the number of pairs of images that are separated by less than a distance of exp (µ−γ). This ratio was plotted as a function of binned time differences  on a log-log plot for clarity. In Further insight into the nature of the two scales can be gleaned by examining the correlation dimension plots of the remaining participants. For example, the correlation dimension plot for participant AS who reported having an unusually repetitive and monotonous lifestyle during the weeks of data collection, is relatively linearized, and the dimensionalities at the two scales are closer together  than the dimensionalities that characterize the other participants’ lives at the two distance scales . This means that the correlation dimension need not be dependent on temporal properties - but rather could be strictly a property of the geometry of the points. Indeed, Doxas et al. e dimensional vector. The starting point of the window is then advanced to the next point in the time series and the procedure is repeated. The delay embedding theorem To establish a link between the structure and dynamics of experience, we attempt to recover the correlation dimension estimates from the temporal information in the experiential stream. We first construct a time series consisting of the values of the first dimension of the image vectors . Takens’ theorem e is usually chosen by computing the correlation dimension for increasing values of De until it asymptotes, at which point we assume that the system has completely “unfolded”. A more precise topological treatment of the notion of an embedding and a detailed introduction to Takens' embedding theorem are presented in section 4 in An appropriate value of embedding dimension DResults and discussion. The length of the time series x(t) is N = 2181 for NV’s data. Delay embedded versions of x(t) are constructed using embedding dimensions of De = {5,10,15,20,25,30,35,40,45,50} and a time delay τ that is optimized for each subject (see ject see .i) is  where τ = 10 min for NV’s data. This process is repeated for i = 1 to Nmax. Beyond Nmax, we no longer have sufficient data to construct a De dimensional time delayed vector, i.e., Nmax+(De−1) τ>N. For each value of embedding dimension De and time delay τ, this procedure gives us a set of Nmax delay embedded vectors.The delay embedded vector corresponding to x. Semantic spaces were constructed for each corpus using Latent Semantic Analysis . Doxas et al. Doxas et al., Recurrence plots of experience were presented to visualize the remarkable regularities in our movement patterns. We showed that visual context space has a two-scaled geometry with a smaller correlation dimension at shorter length scales and a larger dimension at longer length scales. The lower scale primarily captures transitions within the same context while the top scale captures transitions between different spatiotemporal contexts. Additionally, by recovering the correlation dimension estimates directly from time series that represents sequential experience, we showed that the two-scaled structure of context is related to the dynamics of how people move through their environments. The reconstruction of the structure from the dynamics, but not from a randomized sequence of images, is an important extension of earlier work that showed that the semantic space created during discourse has the same two-scaled geometry Finally, while the general properties of the dynamics of context appear common across individuals, the recurrence plots also reveal robust differences that may provide novel and useful ways of characterizing individuals. The approach taken in this paper could be put to good use in many applied settings. For example, the correlation dimension technique could potentially be used in lifelogging retrieval systems since it provides us with a personalized distance threshold for each individual that could be used for extracting images that belong to the same context as a cue image. As another example, employing a lifelogging system to track the behavior patterns of people with disorders such as schizophrenia and Alzheimer's disease may prove to be valuable in developing time critical interventions. The current study demonstrates the power of using dynamical systems methods on lifelogging data to answer complex questions about our interaction with the world.Figure S1Comparison of the color histogram and color correlogram representations. In the first image, there are 5 black pixels and 3 white pixels surrounding the pixel at the center of region marked by the grey square. In the second image, there are 3 white and 5 black pixels surrounding the same pixel. Both images contain the same total number of black and white pixels. The histogram representation being a global description of the number of pixels of each color, is identical for the two images but the correlogram representation takes into account local spatial color correlations and makes a distinction between the two images as shown by the difference in the number of pixels of j  =  {white, black} from pixel i (denoted by the arrow).(TIF)Click here for additional data file.Figure S2Common neighbor ratio averaged over five subjects. The representation with the highest common neighbor ratio is more likely than the other representations to identify images that come from the same context as being similar to each other. The correlogram representation outperforms both the color histogram and SIFT representations.(TIF)Click here for additional data file.Figure S3Common neighbor ratios for individual subjects. The representation with the highest common neighbor ratio is more likely than the other representations to identify images that come from the same context as being similar to each other. The correlogram representation outperforms both the color histogram and SIFT representations.(TIF)Click here for additional data file.Figure S4Global (unthresholded) recurrence plots for 5 subjects using three different image representations. The left panel shows the recurrence plots constructed using the color histogram representation, the middle panel using the color correlogram and the right panel using SIFT. The plots for the color histogram and correlogram representations look similar. SIFT identifies many more points as being recurrence points. Signatures of each participant’s individual lifestyles are present in their corresponding recurrence plots. AS reported having led an unusually monotonous lifestyle during the data collection period. The greater off diagonal structures in AS’ recurrence plots capture the fact that AS visited the same locations over time.(TIF)Click here for additional data file.Figure S5Demonstration of the correlation dimension calculation.A The thresholded recurrence plot (RP) for a threshold of r = 1.101. The number of points in this RP is C (r) and is the lower point marked in the log-log plot of panel C. B The RP for a threshold of r+dr = 1.290. The corresponding C (r+dr) is the upper point plotted in panel C. C The slope of the log[C (r)] vs log(r) plot is the estimate of the correlation dimension D2.(TIF)Click here for additional data file.Figure S6Correlation dimension plots for individual participants. Every single participant’s space of visual context exhibits a two-scaled geometry. The bent-cable estimates for the lower and upper scales respectively are 6.30 and 9.55 for AS, 3.91 and 13.39 for SD, 4.70 and 13.09 for YZSC, and 4.80 and 10.81 for VSSC. The solid lines indicate the best fitting bent cable regression and the dotted lines indicate the bend point and the associated width of the estimated bend.(TIF)Click here for additional data file.Figure S7Takens’ delay embedding procedure: Reconstruction of the lower scale. Different time delays were used for different participants to construct the delay embedded vectors at each value of embedding dimension. As the embedding dimension is increased, the correlation dimension of the reconstructed delay embedded vectors asymptotes to a value close to the original lower scale dimension for each participant.(TIF)Click here for additional data file.Figure S8A The average mutual information plot of NV’s time series guides our search for an optimal time delay τ for Takens’ delay embedding procedure. B Takens’ embedding works approximately equally well for τ = 10, 20, and 30 for NV’s data. In contrast, the randomized time series fills space, to within the limits of the number of points in the dataset, for all values of τ (only τ = 10 is presented here for clarity).(TIF)Click here for additional data file.Table S1Cross validation residual sum of squares (CV RSS) for each subject and model. Presented in the table are the mean values of CV RSS with the standard deviation presented in parentheses. The models considered are the polynomial (Poly.) regression models with degree 1 to 3, and the bent-cable regression model. The bent cable regression is chosen as the best predictive and generalizable model for every participant’s data.(DOCX)Click here for additional data file.Information S1Supporting Information for “The geometry and dynamics of lifelogs: Discovering the organizational principles of human experience”.(DOCX)Click here for additional data file.


Xenograft studies are commonly used to assess the efficacy of new compounds and characterise their dose–response relationship. Analysis often involves comparing the final tumour sizes across dose groups. This can cause bias, as often in xenograft studies a tumour burden limit (TBL) is imposed for ethical reasons, leading to the animals with the largest tumours being excluded from the final analysis. This means the average tumour size, particularly in the control group, is underestimated, leading to an underestimate of the treatment effect.Four methods to account for dropout due to the TBL are proposed, which use all the available data instead of only final observations: modelling, pattern mixture models, treating dropouts as censored using the M3 method and joint modelling of tumour growth and dropout. The methods were applied to both a simulated data set and a real example.All four proposed methods led to an improvement in the estimate of treatment effect in the simulated data. The joint modelling method performed most strongly, with the censoring method also providing a good estimate of the treatment effect, but with higher uncertainty. In the real data example, the dose–response estimated using the censoring and joint modelling methods was higher than the very flat curve estimated from average final measurements.Accounting for dropout using the proposed censoring or joint modelling methods allows the treatment effect to be recovered in studies where it may have been obscured due to dropout caused by the TBL.The online version of this article (doi:10.1007/s00280-016-3059-x) contains supplementary material, which is available to authorized users. The main purpose of these studies is to assess the efficacy of the compound and characterise the dose–response relationship . Analysith index , or usinth index .Comparing the tumour sizes from the final day of the study alone can cause potential issues, as any animals that dropout early are excluded from the analysis, which is equivalent to carrying out a complete case analysis, known to cause bias when data are not missing at random . Often iDropout due to the tumour burden limit means that animals with larger tumours are removed first from the study, reducing the group average tumour size, which disproportionately affects the control group where tumours are expected to be largest. This causes a bias in the estimate of treatment effect, as the tumour size in the control group is being underestimated, and so any tumour shrinkage caused by the compound will look less significant , 7. ThisThis dropout effect has previously been discussed in the literature, notably by Pan et al. , where dInformative dropout in a wider context has also been extensively investigated. Wu and Carroll  focus paIn the present study, four methods for accounting for dropout due to the tumour burden limit were investigated on both a simulated data set and a real data set to see whether they could improve the estimate of the treatment effect. Throughout this paper, tumour growth has been simulated and modelled using the Simeoni model as the base model ; howeverFour methods proposed for dealing with informative dropout due to the tumour burden limit in xenograft experiments are explained below. Each method results in a model for tumour growth, from which tumour size can be estimated at any time. This allows a comparison between dose groups which is based on all available data, rather than focussing on a single time point. The models can also be used for simulating future preclinical trials and translation to clinical trials.f(t) be the structural part of the model, which will give the expected tumour size at time t. If e(t) is the residual error which is assumed to arise from a distribution with mean zero and variance g(t), the observed tumour size will be given by y(t) = f(t) + e(t).The modelling method involves the fitting of a tumour growth model to the available data. Let Vβ) can be computed using Eqs. M is the number of imputations, Bβ is the between-imputation variance of estimates, m, Uβm is the within-imputation variance of estimates.It is not possible to simulate from a pattern mixture model; therefore, it is not possible to calculate the confidence intervals for the dose–response, as the method cannot be bootstrapped and standard errors cannot be calculated; this means that there will be no estimate of uncertainty around the dose–response and whether the dose groups are significantly different to the control group cannot be determined.The pattern mixture method treats the population as a combination of animals with different patterns of dropout and those who complete the study. The population parameters are then averaged across the patterns . The datIn the censoring method, the same base model can be used as in the methods above, but the observations missing due to dropout are treated as censored using the M3 method from Beal . This meThe likelihood for non-missing observations below the tumour burden limit is calculated using Eq. In the joint modelling method, the same base model can be used as in the methods above to describe the tumour growth profiles and then the missing data are modelled using logistic regression. The two models are jointly fitted through the sharing of random effects.A dropout model similar to that in Hansson et al.  is propoA data set was simulated using a model of the anticancer drug paclitaxel’s effect on tumour growth in xenograft studies reported in Simeoni et al. . The modK10 0.868, K12 0.006 and K21 0.0838 h−1, which was assumed to be the same for all animals, as no variation in parameter values was provided. The tumour growth profiles were simulated using the parameter estimates in the original paper for paclitaxel experiment 1, which can be found in the results table, and with the tumour growth model shown in Fig. x1 is the main cycling cells of the tumour and x2, x3 and x4 represent states of cell death following anticancer treatment. Cell growth in x1 is first exponential then followed by a linear growth phase, described by λ0 and λ1, respectively.Once the full data set was simulated, the tumour burden limit was applied, when an animal had an observed tumour size of 2.5 g or above that measurement and all subsequent measurements for that animal were excluded. The study was then truncated, as is often done in practice, with the final day chosen based on the number of animals in the control group, as when very few animals remain in the control group comparisons to the treated groups cannot be made.In this example, the comparison of dose groups is based on the estimation of the dose–response curve, where response is defined as tumour size on the final day of the study, as this allows a comparison with more traditional types of analysis. However, it is possible to use the resulting model to estimate drug efficacy in other ways. The true dose–response curve was simulated from the model and compared to the dose–response curve estimated using the average size from the simulated data following dropout. A t test was also carried out on the simulated data on the final day to see whether differences between the dose groups and the control could be detected.The four model-based methods described above were applied to the simulated data set. For each method, the same base model for tumour growth was used as the one used to simulate. All fitting was carried out in NONMEM 7.3 [The drug concentrations were simulated from the reported two compartment intravenous model, with volume of distribution 0.81 L/kg, NMEM 7.3  using ADA real example was also analysed using the proposed methods. The data were from an AstraZeneca study into an anticancer compound (referred to as drug A), in combination with another drug (referred to as drug C), where a tumour burden limit of 2.5 g was used. The study included 48 animals in 6 equally sized groups, a control group, a 20 mg/kg dose of drug A group and a 10 mg/kg dose of drug C group, as well as three combination groups, where 5, 10 or 20 mg/kg of drug A was given, in addition to 10 mg/kg of drug C. Both drugs were given as single dose on day 13. The dose–response for drug A was considered in the four groups where 10 mg/kg of drug C was given.t tests were carried out on the data at the latest time point where animals remained in each group to be compared. All the available data were then used to develop the model. The same base tumour growth model was used as in the simulated data example; however, a K-PD model was used in place of a PK model as no drug concentrations were available [vailable . The PK The simulated data set is shown in Fig. λ1) which is highly underestimated by the modelling and pattern mixture methods. In general, the inter-individual variation and residual error are less well estimated, the inter-individual variation (IIV) for K1 and K2 could not be well estimated together, so IIV on K2 was removed from the model for all analyses. The modelling method tended to overestimate the variation and underestimate the residual error, whilst the pattern mixture method overestimated both. The censoring and joint methods estimated the IIV well, but the residual error is inflated by both methods, particularly in the censoring method, which may be due to some model misspecification.The parameter estimates obtained using each of the methods are given in Table S1 in the supplementary material. In general, the parameter estimation is good, and all estimates are within 25 % of the true values, with the exception of the estimate of the linear growth phase  are reasonable for most of the parameter estimates, with the censoring method having the highest, and the joint model the lowest. However, some of the RSEs for the IIV estimates are high for the modelling and censoring methods, which were not observed when using the joint model, where all RSEs remained low. The modelling, censoring and joint modelling methods take a comparable amount of time. The pattern mixture method takes longer than the other methods as a model need to be fitted to each dropout pattern, and then to each of the imputed data sets, additional time is also required for the imputation of missing data and pooling of the results from each imputation.t test found that the tumour sizes in the high dose group were significantly smaller than those in the control group, but no difference was found between the lower dose group and the control group.The true dose–response can be assessed using the population tumour size at each dose, as simulated from the model, which can be seen in Table K2) also provides an estimate of the drug’s effectiveness. It was underestimated in the modelling method and overestimated in the other two methods, which is consistent with the greater efficacy estimated by the second of the two methods.The expected tumour sizes on day 17 from each method are also given in Table The pattern mixture model may be affected by small sample size, as there are relatively few animals in each dropout pattern. For this reason, the dropout patterns were combined into four groups (A–D); dropout before day 11 with 7 animals, dropout on day 11 with 7 animals, dropout after day 11 with 16 animals and those who completed the study with 42 animals. This grouping meant there was a minimum of seven animals in each dropout pattern. The ACMV method was chosen, meaning the model fitted to animals in pattern A was used to impute missing values for animals in pattern B, and the model fitted to animals in patterns A and B was used to imputed missing values for animals in pattern C and so on.A plot of the M3 method for the control group is given in Fig. The parameter estimates from the logistic dropout model of the joint modelling method were −20.4 for the intercept, 8.75 for the effect of tumour type, and the random effect CV % estimate was 11.3 %. This resulted in the probability of dropout curve in Fig. The data used to estimate the dose–response of drug A are shown in Fig. There were 128 observations missing because the tumours were too small to measure. The number generally increased as the dose increased. Using the M3 method to account for these unmeasurable tumours was investigated but did not provide an improvement to the model. The parameter estimates from each of the methods are given in Table S2 in the supplementary material; the inter-individual variation could not be estimated for all parameters, due to the high number of parameters relative to the amount of data. The pattern mixture method could not be implemented due to the small number of animals, with only 16 animals dropping out altogether, there was a maximum of two animals in any dropout pattern, which was not enough to build a robust model for the imputation of missing data. In this example, the models were fitted starting at the time of first dose (day 13), not the time of inoculation (day 0).λ0 and λ1) are estimated to be higher in the censoring and joint modelling methods than in the modelling method, explaining the higher estimated tumour sizes on the final day. The standard errors follow a similar pattern to the simulated example, with the largest being observed for the censoring method, followed by the modelling method.The observed variation in the control and drug A only groups was very low, suggesting most of the variability is introduced by drug C; however, this could be due to the shorter follow-up time used in these groups (9 days compared to 69 days). As in the simulated example, the growth parameters  in the above cases, or area under the tumour growth inhibition curve, which may be more effective at summarising the tumour growth throughout the whole experiment.The success of the methods was assessed through the dose–response curve, where response was defined as the estimated tumour size on the final day; this was done so that a comparison could be made to other methods that compare the tumour sizes on the final day. However, this may not be the best way to assess the differences between groups. Other metrics could be used, such as the parameter describing the drug effect Below is the link to the electronic supplementary material.


A-asymmetry will be determined by the uncertainty of the neutron beam polarization measurement which is at the level of (1–2) × 10−3, as shown in previous studies.We are going to use a polarized cold neutron beam and an axial magnetic field in the shape of a bottle formed by a superconducting magnetic system. Such a configuration of magnetic fields allows us to extract the decay electrons inside a well-defined solid angle with high accuracy. An electrostatic cylinder with a potential of 25 kV defines the detected region of neutron decays. The protons, which come from this region will be accelerated and registered by a proton detector. The use of coincidences between electron and proton signals will allow us to considerably suppress the background. The final accuracy of the  In the framework of the Standard Model of V-A-weak interactions the probability of neutron β-decay is written in the following form:W and λ = −1.2739(19). The required level of accuracy has to be 2 × 10−3 or better to reach the comparable accuracy of udV determination from neutron β-decay and from high quark generation decay.The coefficient .A0=−2λλ+/1+3λ2);λ2;A is tA-asymmetry measurement.We are going to reach the necessary level of accuracy using a new experimental scheme for the Cθ can go through the magnetic plug field and reach the electron detector. Electrons outside this solid angle will be reflected by the magnetic field of the plug. The angle Cθ is defined by the ratio of the magnetic field in the uniform part and in the magnetic plug, and it is not dependent on the energy of electronHO is the magnetic field value in the region of neutron decay (uniform part) and Hm is the magnetic field value in the magnetic plug.The scheme of the correlation spectrometer is shown in L = 3320 mm produces a magnetic field HO = 0.35 T. The short solenoid produces a magnetic field Hm = 0.87 T. For this configuration of magnetic fields θC is equal to 39 °. The electron and proton detectors are installed on the axis of the magnetic system. The region of detected β-decay events is defined by the intersection of the neutron beam with the magnetic field lines that go through the electron detector. This region is crosshatched in a and B correlation coefficients will be equal to zero in this case. Therefore we can measure the A correlation coefficient without any correction for admixtures of other correlation coefficients. The signals from the electron and proton detectors have to be detected in coincidence to fulfill this condition. Additionally the use of a coincidence regime of registration of β-decay events will allow us to considerably suppress the background, and to separate effect and background by means of the delayed coincidence method.The long solenoid a and B correlations. The protons can be collected during a time of less than 10 µs. This helps to reduce the background of random coincidences.To detect the protons which have very low initial energy, the decay region is placed at high voltage (25 kV). When protons leave the high voltage region they are accelerated up to the energy of 25 keV and they can be registered by the proton detector. The electrostatic system of the correlation spectrometer consists of a multi-section cylinder with very thin nets at the ends. The sections of the electrostatic system are under different potentials from 21 kV up to 26 kV. Due to the gradient of high voltage potential an electric field appears inside the cylinder that is aligned along the axis of the spectrometer. All protons will be accelerated in the direction of the proton detector independently of their initial momentum. This scheme of the electrostatic system allows us to collect all the protons and to obtain zero average cosines of N↑ and N↓ are the counts of coincidences for different directions of neutron beam polarization.The main task of the experimental procedure is the measurement of the experimental asymmetry which is defined in the following way:A-asymmetry measurements of 10−3 can be reached when all values: the average cosine of the A correlation coefficient P, the electron velocity X are measured or determined with a relative uncertainty better than 10−3.A correlation coefficient can be determined with very high accuracy. It can be attained by the magnetic collimation method, which is independent of the position of neutron β-decay.The uncertainty of determination of the average cosine 2. We plan to use a Si(Li)-detector for registration of electrons. This detector will be divided into 16 individual sections. Previously we performed a test of a Si(Li)-detector with a diameter of 70 mm and a thickness of 3.5 mm to measure the backscattering effect of electrons from Si. A magnetic β-spectrometer was used for this purpose. Thanks to these measurements we can distinguish the tail of the 976 keV line with total area 12 %. In our measurement with the correlation spectrometer the effect of electron backscattering will be even more suppressed because the backscattered electrons will be reflected again from the magnetic plug with probability about 80 %. Those backscattered electrons that do go through the magnetic plug, will be registered by the proton detector. This fraction will be about 2.5 % only. We can reject these events due to a signal of prompt coincidence from both detectors. Therefore the energy resolution of the electron detector will be high enough. The energy calibration of this detector can be done by means of a set of conversion sources.The electron detector will have a size of 55 × 160 mm2, i.e., 20 mm broader and 40 mm higher than the size of the electron detector. This is done to insure 100 % collection of protons. The proton detector will have 16 sections as does the electron detector. This will help us to suppress the background of random coincidences and is necessary in any case to reduce the capacitance of the detector. We are going to use surface barrier detectors made from pure silicon. As a possible alternative to this we are considering the use of microchannel plates.The proton detector will have a size of 75 × 200 mmThe task of precisely collimating the neutron beam is very important for our experimental scheme because the neutron beam passes close to the detectors.6Li, which practically does not produce any γ rays after neutron capture. Because of its chemical activity, 6Li is used in form of 6LiF with some ingredients added to reduce the brittleness of LiF-ceramics. The small angle scattering effect from this material was studied to obtain initial data for the design of precise collimators.The most appropriate collimator material is The experiment was carried out at the PNPI reflectometer of the WWR-M reactor. Using the data of the small angle scattering process that was obtained, a multi-diaphragm collimator has been designed. The distribution of neutron intensity at a distance of 4 m from the collimator was calculated. It was shown that the suppression factor of intensity reaches eight orders of magnitude at a distance of 2 cm to 3 cm from the edge of the beam. This result must be checked experimentally to be sure that satisfactory background conditions can be reached.The possibility of precise measurements of neutron beam polarization has been studied in our detailed experiments , 5. The 3He filters was done in our work [−3 and better.The comparison of the two analyzers and two flippers method with the method of our work . It was X is defined by 8 events have to be registered to obtain the accuracy ΔX ≈ 10−4.The statistical accuracy of the experimental asymme-1 try ΔΦ) at the exit of the polarized cold neutron beam of PSI fundamental facilities “Funspin” is about 2 × 108 cm−2s−1. The neutron flux density after the collimator will be one order of magnitude less (ΦC = 0.15 × 108 cm−2s−1). We expect that the counting rate of neutron β-decay event (in coincidence) will be about 110 s−1. Therefore 108 events will be collected during two weeks. In this estimation it was assumed that the background of random coincidences is considerably less than the signal.The neutron flux density  × 10−3. As a first step of the realization of the experiment, test measurements of background should be carried out.The above-mentioned considerations show that the proposed experiment could reach an accuracy of B and a correlation coefficients as well. To do that, it is necessary to switch off the accelerating electric field in the spectrometer and to measure the longitudinal proton momentum by means of the time-of-flight method. Another way is to use the electrostatic spectrometer in front of the proton detector.Additionally, it should be mentioned that this spectrometer allows one to measure the


Sequence alignment data is often ordered by coordinate (id of the reference sequence plus position on the sequence where the fragment was mapped) when stored in BAM files, as this simplifies the extraction of variants between the mapped data and the reference or of variants within the mapped data. In this order paired reads are usually separated in the file, which complicates some other applications like duplicate marking or conversion to the FastQ format which require to access the full information of the pairs.In this paper we introduce biobambam, a set of tools based on the efficient collation of alignments in BAM files by read name. The employed collation algorithm avoids time and space consuming sorting of alignments by read name where this is possible without using more than a specified amount of main memory. Using this algorithm tasks like duplicate marking in BAM files and conversion of BAM files to the FastQ format can be performed very efficiently with limited resources. We also make the collation algorithm available in the form of an API for other projects. This API is part of the libmaus package.In comparison with previous approaches to problems involving the collation of alignments by read name like the BAM to FastQ or duplication marking utilities our approach can often perform an equivalent task more efficiently in terms of the required main memory and run-time. Our BAM to FastQ conversion is faster than all widely known alternatives including Picard and bamUtil. Our duplicate marking is about as fast as the closest competitor bamUtil for small data sets and faster than all known alternatives on large and complex data sets. Both formats were introduced as part of the SAMtools package (cf.). SAM isbamtofastq and bammarkduplicates2 are more efficient in terms of run-time and memory requirements then equivalent tools in the widely used Picard suite . bammartil (see) for smabamtofastq and bammarkduplicates2 show-casing some applications of fast collation of alignments by name can be found in the biobambam source package (cf., cf) to the 231], cf) for insH of fixed size h without collision resolution. If there is a collision because two alignments with different names are assigned the same hash value, then the alignment previously in the hash table is removed from the table and inserted into a list L of fixed size l of alignments to be handled later. Each time the list L runs out of space we sort the alignments in L by read name. This sorting may yield some new pairs, which we extract before storing the unpaired alignments still in L in a temporary file and emptying the list L. Pseudo code for the insertion of alignments into the hash table H and the list L is given in FigureH over to the list L and in the end flush the list L by sorting the remaining elements by name, extracting the resulting pairs and writing the remaining unpaired alignments to another temporary file. As all the temporary files are sorted by name, we can easily merge the files together to obtain a stream of alignments that is sorted by read name. In this stream it is again simple to detect and output pairs. A diagram of this data flow can be seen in Figuren alignments let nH denote the number of alignments processed without resorting to the list L and let nL = n-nH. Then the time complexity of the algorithm is O(nH + nL log nL), where log without loss of generality denotes the base 2 logarithm. In the unlikely worst case we may have nH = 0 and thus the worst case run-time of the algorithm would become O(n logn), i.e. the run-time would be equivalent to a full sorting of the input file by name. Let sL denote the size of the list L in memory. Then we need O(sL) space for sorting each single block of alignments written to a temporary file in memory and O(n/sL) space for merging the blocks together. Choosing sL inH requires constant space, thus the total space complexity of the algorithm isk by using k merging runs). The algorithm may use O(n) space in external memory in the unlikely worst case.One such data structure would be a hash table with collision resolution by separate chaining. In practice however we see cases where some read ends stay in this hash table for an extended time when we process a BAM file sorted by coordinates from start to end. This may happen for reads where the two ends map to different chromosomes (split reads). There are also often regions in a genome where the sequencing depth is much higher than on average, which can lead to a drastic increase in the amount of memory required to store the hash table at certain points. Instead of using a hash table with collision resolution we use a hash table H and the list L in the following way. The hash table H is implemented as a fixed size character array R which we use as a ring buffer, an array P of integers and a B-tree B. P is the actual hash table storing pointers into R, R is used to store alignments as uncompressed BAM entries and B contains the starting positions of all alignments currently stored in R. A pointer r which is initially set to 0 marks the current position in R. When a name q is to be searched in H, then we first compute the hash value h of the name and check whether position P[h] in R designates the start of an alignment and the stored alignment has the name q. An alignment with hash value h can be erased from H by first removing P[h] from B and then setting P[h] to a special value marking a free position in P. To insert a new alignment with hash value h into H we first need to make sure there is sufficient space. If P[h] is used, then the currently stored alignment for h needs to be moved to L and erased from H. Then we possibly need to remove more alignments from H until the difference between the current insert pointer r and the next higher value in B (considered in a circular way as R is a ring buffer) contains sufficient space to store the new alignment. As soon as sufficient space is available, we can copy the alignment data to position r in R, insert r into B, store P[h]=r and advance r by the length of the alignment data we have just stored. FigureH. We store the list L as a byte array. The alignment data is filled in at the front end of the array. The pointers to the alignment starting positions in the byte array are filled in from the back of the array. The list runs full if we are no longer able to add the next alignment in the same way as the ones already stored. FigureL containing 5 alignments A0,A1,…,A4. A full list can be sorted by keeping the alignment data in place and reordering the pointers at the end of the byte array. Storing H and L in this way requires a very small amount of memory allocation and freeing operations for handling large sets of alignments.To avoid the overhead resulting from the allocation of a small block of memory for each single alignment, we implement the hash table bamtofastq and bammarkduplicates2 to available programs offering the same functionality. For bamtofastq we compared to In the following we will compare the performance of our new tools • Picard’s SamToFastQ component,• bamUtil’s bam2fastq option (see),• bam2fastq (see) and• bampe2fqworphans (see).bammarkduplicates2 we compared to For • Picard’s MarkDuplicates component and• bamUtil’s dedup optionComparisons were performed with the current versions of the programs when our benchmarking for this paper started. These versions are • 1.99 for Picard,• 1.09 for bamUtil and• 1.1.0 for bam2fastq.We used the corrected version of the bampe2fqworphans program as posted by Richard Finney on the SEQanswers forum on the 24th of September 2013. Picard was run using Oracle’s Java (Java SE 7u21).We have run two types of tests. In the first type we test the performance of the programs for a wide variety of input files using a small PC type and a large compute farm type system. These tests were generally run with multiple identical instances in parallel, which reduces the total amount of wall clock time required to obtain the at least ten run time values we measured for each setting to compute a mean run time and standard deviation. In the second type of test we measured the dependence of the run time on the number of concurrently running identical processes. This included the case of running a single instance of a program on an otherwise idle machine. As consequently the time required for each single program and data set was considerably longer we have run this test on a reduced number of input files.a of memory and a fast solid state type drive (SSD) for storing temporary files. The machine was running version 13.10  of Ubuntu Linux. We have run up to four jobs of the same kind in parallel whenever concurrency was not hindered by excessive memory requirements. The machine was completely dedicated to benchmarking during the experiments. We use this machine type as an example for a small system. Due to the absence interfering factors the measured run times should be relatively reliable. The second machine type were HP ProLiant BL465c Gen8 server blades equipped with two 16 core AMD Opteron 6272 processors running at 2.4 Ghz and 256 GB of memory. Files were stored on a high performance parallel network based file system (Lustre). These machines were running version 12.04 (Precise Pangolin) of Ubuntu Linux. We have run ten jobs of the same kind parallel. The machines were not processing other jobs during the benchmarking. The file system however was not fully dedicated to our benchmarking and may have shown punctual performance fluctuations during our testing. We use this machine type as an example for a large scale compute farm type system. For both machine types the times given below consist of an average and standard deviation over 10 runs of the programs with identical parameters. The memory usage given denotes the maximum resident set size (RSS) observed during the programs run. For running Picard we have adjusted the amount of memory available by setting the -Xmx switch of the virtual machine and chosen the serial garbage collector . Available memory was generally set to a multiple of 64 MB. The amount of memory used (the RSS) by the virtual machine can exceed the desired value by a small amount. This explains why the values given below for memory used by Picard is generally not a multiple of 64 MB. Before starting the timing tests on the PC type system we have checked the capability of the programs to handle our test data sets on a machine equipped with 512 GB of memory. This machine uses Intel Xeon E7440 processors running at 2.4 GHz. We have considered a program on a data set as admissible for the PC system if it was capable of handling the data set using 10 GB of memory. On the server blades we have provided 240 GB of memory to the set of 10 concurrently running process (due to features of the employed job scheduling system 240 GB denotes 240·109 bytes in this context). Thus each single process had an implicit limit of 24 GB (i.e. 24·109 bytes). We have set a run-time limit of one day for each data set. Thus a program was considered not capable of handling a data set when it ran for more than one day or required more than 24 GB of memory. On both system types we used the programs were in no case slowed down by insufficient input speed. On the output side we generally did not write the resulting files to disk for any of the programs as we wanted to measure the speed of the algorithms without interference of output speed. For the case of duplicate marking in BAM files the output is generally not produced at a rate which would pose a challenge for commonly used current storage systems. On the PC system the output rate for the BAM to FastQ conversion of biobambam is for at least one of the input files we used exceeding 120 MB/s. This would be too much for a single conventional hard drive but not for the SSD storage or a high performance network file system. If the output speed in BAM to FastQ conversion is a limiting factor, then the option of bamtofastq for generating gzip compressed output may be helpful. Due to the long run-time for some data sets the set of input files we have used on the PC system is a subset of the files we have evaluated on the server blades as only a single PC system was available. On the PC system we have evaluated the dependence of program run-time on the amount of available RAM for programs which allowed setting any parameters controlling memory usage. On the server blades we have given all available memory of the machine to the set of 10 concurrently running processes of the same type. For Java we set the target memory per virtual machine to 16 GB as we needed to give a fixed amount at the start of the program. This was sufficient to not let run Picard out of memory on any data set. For cases where Picard was using close to all of the provided memory we also state the run-time for higher amounts of available memory.For the first type of test we have used two kinds of hardware configurations. The first was a standard PC equipped with an Intel Core i7-2700K four core processor running at a frequency of 3.5 GHz, 16 GBFor the second type of test measuring the dependence of run time on the number of concurrently running identical processes we have also used two types of systems. The first type is the compute farm setting described above. Again the machines used were not processing other jobs during our benchmarking but the file system was not fully dedicated. We have tested the run time for each program when running 1,2,…,10,12,16,24 and 32 identical instances in parallel. We have replaced the PC type system for this type of test by a small server type machine with a higher number of processor cores to observe the behaviour of the programs for more than four parallel instances in a controlled input/output setting. This system was fitted with two Intel Xeon E5-2620 processors and 32 GB of RAM. Each of the processors has 6 cores and is capable of hyper threading, thus the total number of processes the machine can run in parallel is 24. The machine was running version 14.04 (Trusty Tahr) of Ubuntu Linux. Like the PC type system this machine was also fitted with a fast solid state drive for storing files. We have tested the run time for each program when running 1,2,…,10,12,16,20 and 24 identical instances in parallel. For both machine types the times given below consist of an average and standard deviation over at least 10 runs of the programs with identical parameters.For evaluating the performance of our approach for converting BAM to FastQ on the PC system in comparison with the other alternatives mentioned above we have used the following data sets, each stored in a single BAM file . Most of this memory usage consists of program code. Even with default settings the memory usage of bamtofastq is so far below the memory per core on a current machine that we do not find a further reduction useful. Both bamtofastq and SamToFastQ did not benefit from using more memory.1. The low depth single chromosome human data set HG03520 (see) from thbamtofastq requires 5.3±0.04 minutes using 132 MB of memory and 5.46±0.04 minutes using 64 MB. All the other programs have a higher run-time . None of bamtofastq and SamToFastQ gained significantly by using more main memory. SamToFastQ fails to process the data set given 128 MB of RAM.2. The low depth human data set HG00096 (see) from th7. The size of the resulting sorted BAM file is 95 GB. bamtofastq was able to handle the file in 44.53±0.62 minutes and 133 MB of RAM. Using more main memory resulted in only a slight decrease in run-time (42.99±0.36 minutes using 2.44 GB). Picard ran out of memory on the file when provided with 5 GB and less of main memory. It took 497.71±5.41 minutes using 6.05 GB of RAM. The run-time drops to 180.03±2.21 minutes for 8.07 GB and further to 169.69±0.98 minutes for 12.09 GB. The high run-time of Picard using 6 GB of main memory suggests that the employed memory management gets very inefficient when the program uses most of the provided memory. For 5 GB it fails after a run time of more than 8 hours. The moderate drop of the run-time between 10 and 12 GB suggests that a further increase in memory would not lead to a significant decrease in run-time. Indeed, a comparison of the run-time on the compute farm nodes when allowing 12 and 230 GB of RAM showed a gain of less than 1 percent . bamUtil failed the admissibility test for this data set. It aborted with an out of memory type error. On the Xeon E7440 type machine it ran for 58 hours before producing this error. Note that even if the program would be capable of handling the data set using more memory the run-time would still exceed the 58 hour mark on the Xeon E7440 machine. This suggests conceptual problems in the approach of the program for larger data sets. bam2fastq processed the file in time 144.20±1.11 minutes using 4.35 GB of RAM, bampe2fqworphans required 97.97±1.10 minutes using 3.48 GB. Thus bamtofastq outperforms the other tools while using less memory on this data set.3. The high depth human data set ERP001231  with a 7. bamtofastq is able to handle this file in 37.00 ± 0.29 minutes using 206 MB of main memory. In this memory setting a large amount of reads need to be handled by resorting to temporary files on secondary storage because of the high depth of the input data. Due to this effect the run-time decreases to 24.27 ± 0.16 minutes when we let the program use 2.45 GB of memory. Picard fails with an out of memory type error when given 8 GB of main memory. Using 9.18 GB of memory it processes the file in 236.54 ± 2.75 minutes. Increasing the main memory given to 12.29G and 13.28G decreases the run time to 111.93 ± 0.82 and 97.94 ± 0.65 minutes respectively. As above for the high depth human data set ERP001231 we have checked the effects of drastically increasing the amount of available memory using the compute farm node setting. Here an increase from 13 to 230 GB resulted in a speed gain of 6.5% . Again as for the high depth human data set bamUtil fails the admissibility test for this data set. Given a limit of 10 GB the program fails with an out of memory type error after more than 36 hours on the Xeon E7440 machine. bam2fastq processed the file in 59.53 ± 1.03 minutes using 6.56 GB of memory. bampe2fqworphans took 37.19 ± 0.36 minutes using 5.20 GB. bamtofastq outperforms the other tools in every aspect on this data set.4. The high depth E. coli data set from the study SRP017681  with a HashMap class and keeps each end in this hash table until the other end of the read is found in the file. This explains its high memory requirements. The performance is also low due to frequent object allocation and implicit deallocation (garbage collection) processes, in particular when the memory used is close to the memory given. bamUtil, bam2fastq and bampe2fqworphans are similar in the sense that they also keep data structures containing all the reads whose mates have not been seen so far. In consequence they require more memory as the size and depth of the input file grows. bamtofastq can handle all the given files easily with its default small memory foot print. In particular it does not require the user to adjust the input parameters to process any of the files.The left column of FigureLP of aligned pairs and a list of aligned reads (first or second mates of pairs and single) LS in external memory. Both lists are sorted by coordinates, where the sorting of the list LP is lexicographic in the coordinates of the two ends (i.e. the pairs are first sorted by the leftmost mapping end and then those which have the same leftmost mapping position are sorted by the mapping coordinate of the other end). The coordinates used denote the position of the 5′ end of the read on the reference (i.e. the left end position on the reference for those mapped to the forward strand and the right end position on the reference for those mapped to the reverse strand). Pseudo code for this list generation stage is shown in FigureLP and LS into subsets of read pairs and single reads respectively which map to the same coordinates. In each such subset a single element with the highest score computed from the base qualities of the reads is selected as representant and the other elements are considered and marked as duplicates. In addition the current code also considers single ended reads and orphans as duplicates if they map to the same coordinate as one end of a mapped pair. Pseudo code for the deduction of reads to be marked as duplicates based on the lists LS and LP is shown in FiguresLarge sets of sequenced reads often contain reads or read pairs which are not unique, i.e. such reads and read pairs which map to the same coordinates on a given reference genome. This may happen for several reasons including artefacts of library preparation (e.g. duplication by PCR), sequencing artefacts  or just by chance as the selection of sequenced molecules is usually a random process. For some data sets the number of duplicate reads can be very high. In the E. coli data set SRP017681 mentioned above (see) for insLS and LP in external memory. If the input file is given in coordinate sorted order it is relatively easy to detect when all reads mapping to a coordinate or all read pairs mapping to a pair of coordinates have been observed during a linear scan of the file. One approach is thus to keep lists of reads in memory for single coordinates and pairs of coordinates and process these lists as soon as it is clear that no more reads will be added down stream. As this does not require a full sorting the approach is faster in practice. For some files the size of the lists which need to be kept in memory can become very large and the approach suffers from similar problems than the collation of reads by name without resorting to external memory. In biobambam we have implemented a program bammarkduplicates2 which combines the two approaches. We keep lists in memory like bamUtil, but whenever the the number of elements stored in these lists exceeds a given threshold we flush the list with the lowest coordinate out to disk and mark the coordinate for handling in external memory. While filling the lists in RAM we use our collation by read name approach to fill the lists for read pairs. This hybrid approach gives bammarkduplicates2 a very stable and predictable memory footprint. This is different from Picard’s MarkDuplicates tool and bamUtil’s dedup, which use a large amount of main memory for some data sets featuring high coverage in some regions or as a whole and thus are harder to handle in automated sequencing pipelines, as they sometimes require manual intervention due to out of memory type errors.The dedup option in bamUtil produces the same results as Picard’s MarkDuplicates component using different data structures in RAM and without resorting to the construction of the lists bammarkduplicates2 in comparison to Picard’s MarkDuplicates and bamUtil’s dedup option for the same BAM files as we have used above for the BAM to FastQ evaluation on the PC system.We have evaluated the performance of bammarkduplicates2 in 2.43±0.013 minutes using 68 MB of RAM, MarkDuplicates requires 3.57±0.02 minutes with 113 MB and bamUtil dedup 2.31±0.01 using 29 MB. bammarkduplicates2 and MarkDuplicates do not profit from using more memory. bamUtil dedup is the fastest program for this data set and about 5% faster than bammarkduplicates2.1. The small low coverage file HG03520 is processed by bammarkduplicates2 handles the file in 33.90±0.37 minutes using 295 MB of RAM, MarkDuplicates in 48.06±0.39s using 302 MB and bamUtil dedup in 32.83±0.20 using 356 MB. bammarkduplicates2 and MarkDuplicates do not benefit from using more memory. For this data set bamUtil dedup outperforms bammarkduplicates2 in speed by about 3.3%.2. Like the smaller file HG03520 the low depth data set HG00096 can efficiently be handled by the three programs with small amounts of heap space. bammarkduplicates2 processes the file obtained by mapping the reads from ERP001231 as described above in 306±2.9 minutes using 453 MB of main memory. Picard is not able to handle the file using 512 MB of RAM. With 832 MB it runs for 396±3.9 minutes. For 4.06 GB the run-time slightly decreases to 381±1.9 minutes. The experience with other data sets suggests that a further increase of the memory provided for the Java virtual machine will not lead to a drastic decrease in run-time once the slope of the run-time against space curve has reached a low value. For this reason and in the light of the fact that the comparison between biobambam and Picard is already biased by the reduced amount of parallelism available for Picard due to its high memory requirements (i.e. we had to reduce the number of concurrent Picard processes to get 4 GB for a single one. As we will see later on in the paper a reduction of the number of concurrently running processes leads to a speed up of the single processes) we have not tested Picard for higher amounts of memory on this machine type. Picard’s MarkDuplicates fails to process this file on the compute farm nodes due to I/O patterns which are not compatible with the employed file system (see below), so we were not able to test it with amounts of memory exceeding 16 GB. bamUtil dedup fails the admissibility test for this data set. On the Xeon E7440 type machine given 10 GB of RAM it fails with an out of memory type error after running for more than 53 hours.3. bammarkduplicates2 in time 152±1.1 minutes using 311 MB of main memory. The run time decreases to 138±0.89 when the main memory provided is increased to 2.54 GB. Picard’s MarkDuplicates tool is not capable of handling the file given 3 GB of memory, it aborts with an out of memory type error. Given 4.05 GB it processes the file in time 176±0.97 minutes, where as for ERP001231 this already required a reduction of the concurrently running Picard processes in comparison with biobambam, thus leading to reduced comparability. Further increasing the main memory threshold to 8.06 GB and 12.07 GB decreases its run time to 163±0.92 and 161±1.60 minutes respectively. A test on the compute farm nodes shows that there is no further run-time improvement when increasing the amount of memory available from 12 to 64 GB . Picard’s MarkDuplicates fails to run given 96 GB of memory as it attempts to allocate an array which is too large for the JVM system (Java limits the size of a single array to 231-O(1) for architectural reasons). bamUtil dedup does not pass the admissibility test for this data set. It fails with an out of memory type error after more than 37 hours given 10 GB on the Xeon E7440 machine.4. The BAM file obtained by mapping the reads from the study SRP017681 as stated above is handled by bammarkduplicates2. In settings with variable types of data which may be large or feature high depth bamUtil dedup can fail on data sets which are easily handled by bammarkduplicates2. Picard’s MarkDuplicates cannot keep up with bammarkduplicates2 in both respects, run-time and RAM requirements.The right column of Figurebamtofastq with default settings is the fastest program for all sets but SRP017681, where bampe2fqworphans is faster. When the memory usage of bamtofastq is increased to 2.5 GB, which is still smaller than the 5.2 GB used by bampe2fqworphans, then bamtofastq is also the fastest for this data set. The lower performance of bamtofastq with default settings can be attributed to the higher amount of data written to temporary files for this high depth input file. The influence of this effect on the server blades is more prominent than on the PC system due to the fast temporary space on the PC. As Picard runs in a Java virtual machine the memory stated in TablesFor the comparison of program performance on server blades we have used the data sets as specified in Tablebammarkduplicates2 on server blades as we have used for the comparisons concerning bamtofastq. As on the PC system we have compared with bamUtil’s dedup option and Picard’s MarkDuplicates program. The run-time comparisons are shown in the Tablesbammarkduplicates2 was the only of the programs which was capable of handling all the input files. bamUtil dedup failed on the files SRP017681 and ERP001231. For SRP017681 its run-time exceeded 24 hours, for ERP001231 it required more than 24 GB of memory. It is capable of processing the file ERP001231 with more memory  the confidence intervals for the run-time of the two programs overlap. For the files ERR196957, ERR054938, ERR328190 and ERP001231 bammarkduplicates2 has the lowest average run-time using its default settings where for ERR196957 the confidence interval overlaps with bamUtil dedup. Picard MarkDuplicates is slightly (7%) faster than bammarkduplicates2 running with default settings for the file SRP017681, but this changes when bammarkduplicates2 is allowed to use more memory than by default. In summary bammarkduplicates2 is the only program which is capable of handling all the files within the given limits of 24 hours and 24 GB of memory. bamUtil dedup has a slight run-time advantage of up to 5% for small and low depth larger files but fails for more complex higher depth files. Picard is consistently slower than bammarkduplicates2 with default settings on all files but SRP017681 and slower on all files when the amount of used memory is at the same level. Some files trigger input/output patterns in Picard which are not suitable for network file systems. As Picard used close to all of the memory provided for some of the data sets, we have verified that no speed ups were available through the usage of more memory . In this scenario Picard’s Java implementation was suffering the most from concurrently running instances with a slow down of 2.8 while biobambam’s bamtofastq is consistently the fastest solution and suffers the least from running in a busy environment.We have used the files HG03520 and HG00096 as described above (see Tablebamtofastq is the fastest solution for any amount of concurrency.Figurebammarkduplicates2, Picard’s MarkDuplicates and bamUtil dedup on the data sets HG03520 and HG00096. The results are presented in the forms of plots in FiguresAs above for BAM to FastQ conversion we have tested the effects of concurrently running instances on duplicate record marking in BAM files using biobambam’s bamtofastq and bammarkduplicates2 based on the API. bamtofastq is faster than all widely known competitors while using only small amounts of RAM. bammarkduplicates2 is close the performance of bamUtil dedup on small data sets while easily handling larger and more complex data sets for which bamUtil dedup fails and it outperforms the duplication marking tool of Picard in both speed and memory aspects. biobambam is well suited for environments running many instances of the problems solved concurrently.In this paper we have presented efficient algorithms and data structures for name collated BAM file input. We have provided an implementation of these in libmaus, an open source programming library for C++. As part of the biobambam package we have developed two tools Project name: biobambam/libmausOperating systems: Linux and MacOS XProgramming language: C++Other requirements: noneLicense: GPL3Any restrictions to use by non-academics: noneaunless otherwise stated we use MB and GB to denote 220 and 230 bytes respectively in this paper.The authors declare that they have no competing interests.bammarkduplicates2 tool more compatible with Picard’s MarkDuplicates module. Both authors read and approved the final manuscript.GT wrote the code, ran the tests and benchmarks and wrote the paper. SL contributed to the testing and provided patches for making the Appendix A.Click here for file


The maximum height of a siphon is generally assumed to be dependent on barometric pressure—about 10 m at sea level. This limit arises because the pressure in a siphon above the upper reservoir level is below the ambient pressure, and when the height of a siphon approaches 10 m, the pressure at the crown of the siphon falls below the vapour pressure of water causing water to boil breaking the column. After breaking, the columns on either side are supported by differential pressure between ambient and the low-pressure region at the top of the siphon. Here we report an experiment of a siphon operating at sea level at a height of 15 m, well above 10 m. Prior degassing of the water prevented cavitation. This experiment provides conclusive evidence that siphons operate through gravity and molecular cohesion. Although the siphon has been used since ancient times, the means of operation has been a matter of controversy1234578Both siphon models–atmospheric and cohesion–predict that the maximum height of a siphon is dependent on the ambient barometric pressure. In the case of the atmospheric model, the pressure of the atmosphere is required to hold the column of water together. In the cohesion model, the limit is explained by the pressure at the top of the siphon falling below the vapour pressure of water, at the given temperature, so that cavitation occurs, i.e. the water starts to boil thereby breaking the column.2. It costs energy to make bubbles in water because of the energy of the bubble surface. For a bubble to be stable it must be supported either by internal pressure of a gas or by the equivalent tension (negative pressure) in the water. For gas in a bubble the pressure (P) is given by (1). This equationHowever, the cohesion model predicts that if cavitation can be prevented, the barometric height limit can be broken. The reason for the cohesion is that surfaces cost energy and the water/air surface is no different. For water, the surface energy is often referred to as surface tension. The surface energy of the water/air interface is 0.072 J/mγ is the surface energy (J/m2 or N/m), and r (m) is the bubble radius. A good benchmark pressure is the atmospheric pressure which is = 1.013 × 105 Pa (N/m2). An internal pressure of one atmosphere  could support a bubble of radius r where:where That is, an internal pressure of one atmosphere is generated by a bubble of 1.42 μm radius (a diameter of 2.8 μm). Equivalently, tension equal to the support of one atmosphere would occur for an empty bubble of diameter 2.8 μm. A smaller bubble would support greater water tension and a larger bubble a lesser water tension. A bubble of 2.8 nm diameter could tolerate water tension equal to 1000 atmospheres (100 MPa).Many experiments have been performed to measure the tensile strength of water1213141516171819−3 Pa for a period of more than three weeks. During the initial degassing process, significant volumes of gas were evolved from both the water and capping layers. This process is commonly attributed to boiling, but as qualified in subsequent sections, this effect is entirely due to dissolved gasses coming out of the water. A small amount of water (~2 ml) was evaporated from the initial volume, mainly due to the exposure of the surface of the water when large bubbles passed though the capping layer.In an initial experiment, 60 ml of ordinary tap water with a 4 ml silicon oil-capping layer was held under a vacuum of <10Once the water and capping layer were fully degassed, there was no further loss of either fluid. After allowing the vessel to return to atmospheric pressure for a short time, subsequent evacuations did not cause more gas to evolve from the water . However−1 Pa the U-tube was raised by lifting the apex of the tube to a height of 300 mm above the surface of the oil. With a density marginally lower than that of water, the oil surface was assumed to be close to that of a hypothetical water vacuum interface. It was observed that the water formed a continuous column with no bubbles/cavities forming at the top of the tube  of a siphon isFor a siphon with dissolved gases the maximum height . Therefore, it is evident that atmospheric pressure plays no part in carrying the water over the apex of the siphon tube. Therefore it is clear that a new equation for the maximum height of a siphon is required for situations where cavitation does not occur.The new equation is much simpler and isw is the tensile strength of water. So for example if the tensile strength of a sample of water was 1 MPa, the maximum height of a siphon would be about 100 m. In the case of the siphon is this experiment we can say that the tensile strength of the water was greater than −0.15 MPa.where TSExtrapolating these results from even the most conservative experimental measurements of the tension under which cavitation occurs it is possible that the cohesive strength of fully degassed water is able to support a continuous vertical column greater than several hundred meters. While the experiment performed here did not reach anywhere near the absolute limit predicted it does shed light on the stability of flowing water under tensile stress and the possibility of constructing apparatus of suitable dimensions to test such a limit. These experiments also lend support to the cohesion-tension theory of sap ascent in trees. It would be interesting to perform further experiments to see if it is possible to operate a flowing siphon at above 100 m. If tensions as high as the transient tension of several 100 bar can be maintained at the apex of a siphon, then in principle a siphon should work up to a height of several km. However, it would be challenging to verify this experimentally, requiring a helicopter or UAV with a ceiling of several km capable of supporting several kg of water-filled tubing and cable supporting the siphon. It would also be interesting to repeat the experiment with a larger diameter tube. In view of the many anomalies of bulk waterIn previous attempts to degas water in vessels constructed from materials such as metal and rubber it was observed that the surfaces acted as effective nucleating sites leading to continual bubble formation and water loss. Consequently, in the experiments described here, glass or nylon was used for surfaces that came into contact with the water. To aid observation, a red water-soluble dye was used , after first performing each experiment using untreated tap water. There was no noticeable difference in nucleation/degassing rates observed or absolute pressure attained between the ordinary tap water and the dyed water throughout all subsequent experiments. As the dye was not soluble in oil the capping layer remained transparent and colourless throughout.−2 Pa, 1 to 1 × 10−6 Pa and 10 to 0.01 mm of Hg for each respectively. To provide a complete pressure range once the pressure in the chamber was reduced below the operational limit of the Pirani gauge the AIM gauge was then used to record the pressure. To provide independent verification of the two electronic gauges a mercury-filled Edwards Vacustat McLeod gauge was also attached to the chamber. While less accurate than the two electronic gauges, the McLeod gauge was able to show that the vacuum was of a similar order of magnitude and that the electronic gauges were not unduly affected by water in the partial atmosphere.For the initial experiment, vacuum pressure measurements were made using three independent gauges mounted on the side of the Perspex fronted vacuum vessel . The vac−6 Pa at 25 °C, specific gravity 0.863 g/cm3 at 25 °C, viscosity 60 mPa s at 40 °C) was then floated above water. As the two liquids are immiscible the less dense silicon oil formed a visible capping layer over the surface of the water. Once the water was completely covered with a layer of oil the air was removed from the chamber by evacuating through a 6 cm diameter hole in the base of the chamber to which a BOC Edwards EXT70 turbomolecular turbo vacuum pump (B722–01–000) pump is directly attached. Starting at atmospheric pressures this was achieved by opening the SP16K Edwards speedy valve at the base of turbo pump which was connected, via a 60 cm long KF25 vacuum bellows, to an Edwards 5 rotary vane pump (A653–01–903). Once vacuum pressures below 1 Pa were achieved the turbo molecular pump was then switched on reducing the pressure to below 1 × 10−3 Pa.In the first degassing experiment, 60 ml of ordinary tap water was held in a 100 ml measuring cylinder placed inside a Perspex fronted vacuum chamber . To measAs the volume above the fluids was evacuated to pressures below 3 kPa, dissolved gasses were immediately released at a very high rate, appearing in a manner not unlike that of nucleated boiling. After the initial burst of gas the rate of release slowed significantly with occasional bubbles still appearing after a period of many hours. At intermediate stages in the degassing process the low partial pressure caused occasional bubbles to expand explosively through the surface of the oil, exposing the water to the vacuum. When this occurred a small amount of water at the surface would evaporate leading to further bubbles in a continuous cycle, leading to a marked drop in temperature observed on the thermometer. To prevent complete evaporation of the sample, and possible freezing of the water, the vacuum vessel was immediately isolated until the partial pressure rose and bubble formation abated. Once the water surface had settled and a continuous layer of oil reformed over the water pumping was resumed.−3 Pa by switching on the turbo pump.During later stages of the degassing process it was necessary to slow or temporarily halt the vacuum pump to allow the layer of oil to reform and gas emerge, as the flow of small bubbles through the oil had turned it into translucent foam. The pump-stop-pump process was then repeated until the oil became fully transparent. When no more gas was observed to evolve from either liquid the partial pressure was then reduced to <10−4 Pa was recorded using an Edwards Penning gauge upstream of the diffusion pump. During initial degassing of the samples the vacuum line could be diverted directly to the rotary pump. The rotary pump was also used when the sample was pumped continuously over extended time frames.For the inverted U-tube and vacuum siphon experiment, pressures were measured using an Edwards Priani gauge mounted at the top of a short 400 mm long, 6 mm inner diameter, flexible reinforced plastic hose, in front of two rubber sealed vacuum taps connecting to the vacuum pumps and to air. Pumping of the vacuum was achieved using an Edwards diffusion pump, roughed using an oil-filled rotary vacuum pump (Edwards 8). When not connected to a sample chamber, a partial pressure of less than 10The U-tube glass vessel was made by connecting a 500 mm long, 6 mm inner diameter glass U-tube to the glass reservoir. The base of the U-tube was shaped such that as the apex was raised vertically the connections to the reservoirs would remain below the surface of the liquid. During the degassing process the U-tube was angled downwards away from the surface of the evacuated volume, allowing bubbles formed in the U-tube to float freely to the surface of the water before passing through the oil. It was observed that at later stages of the degassing process, bubbles formed within the U-tube would expand, almost filling the entire length of the tube. Once the bubble reached the surface, the water columns would re-join at great speed, causing a loud ringing noise within the glass vessel, often nucleating new bubbles within the column. Once the degassing process was complete after a period of several hours, the apex of the U-tube was then raised while the volume above the liquids was continually evacuated using the Edwards rotary pump and then at reduced partial pressures using a vacuum diffusion pump. A similar construction and degassing method was used for the vacuum siphon. These methods were then repeated for each of the U-tube and vacuum siphon experiments with dye added to the water.To create the siphon above the barometric limit, a 30 m long 6 mm inner diameter flexible nylon tube (RS components) was used. Attached at either end of the tube were two stainless steel vacuum taps. Prior to filling with the previously degassed water, the tube was continually flushed with tap water for 4 hours to remove any deposits from within the pipe. The tube was then connected at one end to the vacuum pumps and evacuated continually for a period of 48 hours to allow all volatile compounds to be removed. Priming of the tube was achieved by placing the closed end of the evacuated tube into the degassed water, which was then opened allowing the water to flow up the tube, while the other end remained open to the vacuum system. Care was taken to prevent the capping oil from entering the tube during this process. Once the tube was entirely filled with degassed water, both ends of the tube were closed ready for the siphon to be set into position.Before setting the tube in position the siphon was first inverted so that the ends of the tube were at the highest point with the bend at the lowest. This was done to allow additional degassed water to be added as the increased weight caused a slight expansion in the length of the tube. Once the extra water was added the tube was re-inverted with the bend at the apex and the legs hanging straight down into the reservoirs. To prevent kinks occurring in the hose at the apex the siphon the tube was set into a pulley of 12 cm diameter. Once primed, one end of the siphon was set into a reservoir containing more degassed water, with the other venting 30 cm lower inside an empty 1 litre glass beaker with both reservoirs open to the air. Taps at both ends of the tube were then opened so that the liquid could flow freely over the 14.5 m rise into the lower reservoir . Once thThroughout the siphoning process no bubbles were observed in the tube, however small bubbles were observed to emanate from the lower end of the siphon during in the final stages of draining the tube. These bubbles were thought to come from air trapped in the tap and the glass vessel at the base of the tube, which became dislodged by the fast flowing liquid.How to cite this article: Boatwright, A. et al. The height limit of a siphon. Sci. Rep.5, 16790; doi: 10.1038/srep16790 (2015).


Loss of trunk height caused by scoliosis has been previously assessed using different mathematical formulae. However, these are of differing algebraic construction and will give a range of values for the same size of scoliosis curve. As such, the following study attempted to determine the most valid published formulae for calculating height loss caused by idiopathic scoliosis based on reported growth charts.The height and sitting height for a group with idiopathic scoliosis were measured. These were plotted on published growth standards. The size of the coronal curves and the thoracic kyphosis was measured. Height was corrected for the size of the scoliosis using the formulae and replotted on the growth standards. The data spread on the standard was analysed for significant differences between the median and the 5th or 95th centile, and between data outside the 5th and 95th centile.The sitting to standing height ratio growth standard was used in the analysis as it minimised errors across the different growth standards, given that these standards come from different original populations. In the female group significant differences in the data spread were seen using the formulae of Bjure, Ylikoski and Hwang. Non-significant results were seen for the Kono and Stokes formulae. All formulae caused no significant differences in data spread across the growth standard in the males group.When assessing against growth standards, the formulae of Kono and Stokes are the most valid at determining height loss caused by idiopathic scoliosis. Idiopathic scoliosis (IS) is of unknown origin and includes scoliosis seen in the adolescent, between the ages of 10 and 18, and in early adult life once older than 18. It is a growth related deformity of the spine. The deformity leads to a loss of standing height and it is common for surgeons to be asked how much height will be regained when a patient undergoes a corrective scoliosis fusion procedure. Whilst the ‘height gained’ during surgery is dependent on many factors that cannot reliably be predicted pre-operatively, it is possible to calculate the height that has been lost through formulae that have been published . All of The standing and sitting heights of a group of patients with IS were measured. All had one curve of at least 10° in the coronal plane to satisfy the definition of structural scoliosis  even if The measured data were plotted on the published growth standard. The World Health Organisation (WHO) standard for standing height was used with the data sub divided by sex . This waThe size of the coronal scoliosis curves and the sagittal kyphosis and lordosis was measured from standing whole spine radiographs accessed digitally. The radiographs were all taken in the same standardised fashion with the sagittal radiograph taken with the arms in the ‘fists on clavicle’ position to eliminate the effect of arm position on overall sagittal alignment .All coronal curves were measured between the most angled end plates as per the Cobb method . The kypThe formulae for calculating height loss are shown mathematically in Table p value ≤ 0.05.Two different assessments were made to assess which formulae were most valid for calculating the height loss due to the scoliosis. The first method counted the number of data points on either side of the median but between ± 2 z-scores or between the 5th and 95th centile (from here on defined as the inner data spread) and compared them. Second the number of data points above the + 2 z-scores or the 95th centile were compared with the number of data points below – 2 z scores or the 5th centile (defined as the outer data spread). A statistical comparison was made using a test of equal or given proportions . With day is the calculated height lost and x is the Cobb angle or sum of Cobb angles. In the Hwang [z is the kyphosis angle. As not all radiographic series included a sagittal radiograph concordant with the time of height measurement that could be measured for thoracic kyphosis, the total number of data points in the Hwang formula calculation was reduced accordingly.In all of these formulae, he Hwang  formula,Bland Altman analysis  was alsoAll statistical and graphical analysis was performed using R . The patIn the scoliosis group, there were 161 females and 44 males. In the analysis for the Hwang formula there were 137 females and 37 males. Tables p < 0.01), representing an unequal inner data spread. Similarly, there was a significant difference in the number of outliers below the 5th compared to above the 95th centile (p < 0.01), an uneven outer data spread. The ratio of sitting height to standing height was also plotted on the Danish standard [p < 0.01) and outer data spread (p < 0.01) for females was seen  and outer (p < 0.01) data spread. For the sitting height to standing height ratio both the inner data spread (p < 0.01) and outer data spread (p < 0.01) were significant.When standing height was plotted on the WHO height standards there was no significant difference between the inner or outer data spread for both the male and female groups see Fig. . A plot een Fig. . This isCorrected standing and sitting height and corrected sitting height to standing height ratios were calculated. This was done by adding the calculated height loss using the different formulae to the original measured data and then replotting on the appropriate standards. For the Stokes method, only the formula for double curves was used, averaging the size of the two curves as in this series there were not enough truly single curves to make analysis of this group meaningful .The table of results Table  shows thBland Altman analysis showed that the mean difference between all formulae was less than 3 cm (2.97 cm) in both males and females with 95 % limits of agreement no greater than 5.52 cm .This is the first time that growth standards have been used to reference height loss and corrected height loss in scoliosis to our knowledge. The World Health Organisation (WHO) standing height standards  were choA sitting height standard is not published by the WHO but the Danish standard used here is easily accessible . The DanIn clinical practice, patients commonly ask about the height that will be regained following scoliosis surgery. It is very difficult to estimate this pre-operatively as the exact end result of scoliosis correction is dependent on intraoperative factors. The best that can be done is to estimate the amount of height loss secondary to the presence of the scoliosis. This can be achieved pre-operatively using a variety of published formulae that have been detailed here –5. Post-The first attempt to calculate height loss caused by a scoliosis was by Bjure et al. . They deA new formula was proposed by Kono et al.  in 2000.In 2003, Ylikoski  measuredStokes  publisheHwang et al.  retrospeThe Hwang formula  is the oThe assessment of which of the formulae gives the most valid calculation of height loss secondary to scoliosis has been performed using a test of equal or given proportions on the spread of data points above or below the median or 50th centile line, and outside ± 2 z-scores or the 5th and 95th centile . This haThe uniform spread of data points across the WHO standing height standard, even with a loss of height caused by a scoliosis, demonstrates that the spread of data between ± 2 standard deviations from the median is too large to demonstrate the generally small changes in height caused by most scoliotic curves. This is because the loss of height caused by the scoliosis when viewed as a fraction of total body height in the standing position is small and makes little difference to the whole. In this series the mean height loss across all formulae was 3.38 cm for females and 2.86 cm for males. When expressed as a fraction of sitting height, the change is more obvious and can be seen on the sitting height standard. This then suggests that both sitting and standing height standards should be used to chart height in those with scoliosis to identify the subtle loss from the spinal curve, agreeing with the previous literature .Using the definition of most valid as no significant differences in the number of data points for either the inner or outer data spread when plotted on the sitting to standing height ratio, it is seen that the Kono  or StokeBland-Altman analysis for all pairs of formulae allows a comparison of the differences between them . In thisFour of these formulae for calculating loss of height were compared against a measured loss of spine height on an individual basis by Tyrakowski et al. . The ‘trThe assumption behind the use of growth standards in this setting is that the scoliotic spine is the same length as the non-scoliotic spine when all other variables are equal when at the same chronological age. This may be flawed as there is some evidence to suggest that scoliosis is an effect of an ‘over long’ spine or rapid early growth . FurtherThe height loss seen in the presence of scoliosis is best documented on sitting height and sitting to standing height ratio growth charts. This height loss can be calculated pre-operatively and the most valid result will be obtained with the formulae described by Kono et al.  or StokeEthical approval from the Research Ethics Committee has been obtained for this study reference 15/EM/0283.