This study was carried out in Pickering emulsions stabilised solely with silica particles of different hydrophobicity. We correlate, for the first time, the properties of the emulsions with the conversion of the reaction, thus gaining an insight into the impact of the particle wettability and particle concentration. When comparing two emulsions of different type with similar stability and droplet diameter, the oil‐in‐water (o/w) system displayed a higher conversion than the water‐in‐oil (w/o) system, despite the conversion in both cases being higher than that in a “classic” two‐phase system. Furthermore, an increase in particle concentration prior to emulsification resulted in an increase of the interfacial area and hence a higher conversion.Pickering emulsion systems have emerged as platforms for the synthesis of organic molecules in biphasic biocatalysis. Herein, the catalytic performance was evaluated for biotransformation using whole cells exemplified for the dehydration of  n‐octanaloxime to n‐octanenitrile catalysed by an aldoxime dehydratase (OxdB) overexpressed in E. coli is carried out in Pickering emulsions stabilised solely with silica particles of different hydrophobicity. The effects of particle wettability and particle concentration are evaluated with respect to the conversion of the reaction.The dehydration of  Enzymes are efficient biocatalysts that, due to their high chemo‐, regio‐ and stereo‐selectivity, play an important role in the synthesis of organic molecules under mild and sustainable conditions.2, 3 E2O3 or Fe2O3)[To improve biphasic solvent systems for enzymatic transformations, co‐solvents,or Fe2O3)27, 30n‐octanaloxime to n‐octanenitrile in Pickering emulsions stabilised with silica particles of different wettability using E. coli cells containing the aldoxime dehydratase OxdB (Scheme 2+) heme.[Aliphatic nitriles are attractive compounds from an industrial point of view as they are widely used as solvents and key intermediates for the large‐scale production of commodity chemicals such as surfactants as well as fine chemicals, agrochemicals and pharmaceuticals. Here we report the enzymatic dehydration of B Scheme . This pr2+) heme.39, 402 particles loaded with Ru.We correlate the properties of the emulsions with the conversion of the reaction, thus gaining an insight into the impact of the effect of the particle wettability and particle concentration. Silica particles possessing various silanol contents (%SiOH) spanning from 100 % SiOH (most hydrophilic) to 15 % SiOH (most hydrophobic) were used. Despite the fact that the particle wettability can be tuned to stabilise the desired emulsion for the biphasic reaction, this parameter has not been addressed systematically so far. In fact, to the best of our knowledge, in the literature there is no such study that compares the conversion of a biocatalytic reaction in both emulsion types (o/w and w/o) with the same particle type by gradually changing the particle hydrophilicity. However, we acknowledge the work of various authors who addressed this topic. Zhang et al. prepared two particle types of different hydrophilicity by grafting methyl groups on the surface of hydrophilic TiOn‐Octanaloxime and n‐octanenitrile are not surface‐active at the n‐dodecane‐water interface  whereas E. coli cells can stabilise some oil droplets in water at a concentration higher than 0.025 wt. % . E. coli cells contain both hydrophilic and hydrophobic moieties at the surface as the outer membrane is composed mainly of lipopolysaccharide, phospholipid and protein.E. coli cells containing a lyase for the stereoselective carboligation of benzaldehyde to (R)‐benzoin.E. coli cells containing OxdB of 0.025 wt. % was selected to prepare emulsions.E. coli cells containing OxdB and n‐octanaloxime in the emulsion constant . The concentration of silica particles is 1.14 wt. % in all cases. Photos were taken one month after preparation  and so particles are thought to be irreversibly adsorbed at interfaces. Therefore, when creaming/sedimentation and coalescence are halted, emulsions remain unchanged for a long period of time. Hence no difference regarding the emulsion stability was observed between the emulsions after 48 h and 1 month.Emulsions were then prepared with silica particles of different hydrophobicity by keeping the concentration of n Figure  and the Vmax is the theoretical maximal turnover rate. Km is defined as the substrate concentration where Vmax/2 is reached and depends on the type of enzyme and substrate. This means that even if the substrate concentration increases above approx. 2 x Km, an increase of the turnover rate will not occur anymore. The observed decrease in the rate with time in all our experiments could be due to either one or a combination of the following reasons: (i) deactivation of the enzyme , (ii) insufficient diffusion of the substrate from the organic to the aqueous phase and (iii) low amount of substrate available . As observed in Figure E. coli cells containing OxdB might not be available for the reaction. In the control experiment, the aqueous phase containing the enzyme was homogenised for 2 min at 13 000 rpm and the two phases were left to stand undisturbed as the emulsions were also not stirred while the reaction was taking place. The conversion reached with other controls (with/without homogenisation of the aqueous phase and with/without stirring) is shown in Figure S8. Stirring enhances the diffusion of the components and high conversions are achieved as shown in Figure S8(a) by comparing the blue and grey points with the yellow and orange points. The conversions obtained with and without homogenisation of the aqueous dispersion containing the cells are similar both for the stirred (grey and blue points) and for the non‐stirred (yellow and orange points) case. Therefore, homogenisation does not seem to affect the activity of the enzyme. However, when the two phases are homogenised together the conversion is very low for both the stirred and non‐stirred case ). This suggests that the organic phase deactivates considerably the enzyme during homogenisation. It is noteworthy that this deactivation effect does not occur when the two phases are homogenised in the presence of silica particles as they protect the enzyme from the organic phase by forming a protective layer of particles at the oil‐water interface. n‐Octanaloxime is slightly soluble in water. Therefore, another control experiment was carried out in a monophasic system, where n‐octanaloxime was placed as a solid in an aqueous phase containing the enzyme (0.44 g L−1) and was stirred at 1000 rpm. The conversion after 1, 4 and 24 h was as low as 2.9, 5.7 and 9.7 %, respectively. All these controls highlight the advantages of carrying out the reaction in an emulsion system, in terms of achieving high conversions and protecting the enzyme from the organic solvent.A second set of emulsions was prepared to study the conversion of the reaction therein. As described in the experimental section, samples from the emulsion cream were withdrawn at different times and the conversion of the reaction was measured by gas chromatography (GC). In all cases, the conversion increases with time but the reaction becomes slower at long time Figure . In enzyt=48 h versus %SiOH  for two partially hydrophobic silica particles. By increasing the particle hydrophobicity (lower SiOH content) particles protrude more into the oil phase of the w/o Pickering emulsion. As a result, the fraction of the particle surface in oil is higher for more hydrophobic particles. This could explain the increase of the conversion detected in the w/o emulsions by increasing the particle hydrophobicity. E. coli cells containing the enzyme in the water droplets may be more protected from the organic environment with 15 % SiOH silica particles than with 25 % or 51 % SiOH silica particles. The average droplet diameter in this case would not be the main factor governing the extent of the conversion as the diffusion of organic molecules across the interface is expected to be high. This is not the case for the o/w emulsions as creaming is observed in some cases. As a result, some E. coli cells containing the enzyme dispersed in the aqueous phase released will not be accessible for reaction.Furthermore, an initial rate was calculated as described in the ESI (section 1.2.5). As shown in Figure n‐octanaloxime and n‐octanenitrile are slightly soluble in water . Similar conversions were measured in the aqueous phase which could imply that the partitioning of the two molecules between the organic and the aqueous phases is similar. As expected, the areas corresponding to both molecules in the aqueous phase were very low compared to the ones measured in the organic phase so they have been neglected for the analysis. The conversion of the reaction in the emulsion cream after 48 h (filled circles) is compared with that obtained from the organic phase separated by centrifugation (empty circles) after the same period of time ). In general, the two values are in good agreement and they match completely for the emulsion prepared with silica particles of intermediate hydrophobicity (no aqueous or organic phase resolved).Forty‐eight hours after emulsion preparation, emulsions were broken by centrifugation, recovering the two incompatible phases together with a thin layer of emulsion cream that could not be broken completely. The conversion of the reaction was measured in both phases by GC. The conversions measured in the organic phases separated from the emulsions prepared with silica particles of different hydrophobicity are shown in Figure S9(b) (empty circles). This is referred to as the work‐up of the organic phase (WU OP). The conversion of the reaction was also measured in the aqueous phase as both E. coli cells with the silica particles but it could also indicate that the confinement of the biocatalyst in water droplets of a w/o emulsion could reduce the conversion compared to the o/w system where the biocatalyst diffuses freely through the continuous phase. Moreover, as pointed out previously in the w/o emulsion prepared with 51 % SiOH silica particles, the particles protrude more into the organic phase. On the contrary, in the o/w emulsion prepared with the 65 % SiOH particles, particles are slightly more wetted by the aqueous phase. Therefore, one would expect less deactivation of the enzyme in the o/w emulsion as the interaction of the enzyme with the organic solvent is slightly lower and hence a higher conversion can be achieved. However, none of these hypotheses has been proven yet.As mentioned in the introduction, this is the first time that the conversion of a biocatalytic reaction is followed in Pickering emulsions prepared with the same particle type (silica) by progressively increasing the particle hydrophobicity. One of the reasons that could explain the absence in the literature of a systematic study could be that it is difficult to prepare two emulsions differing in type with a similar stability and average droplet diameter. Zhang et al. prepared two emulsion types with similar droplet diameters (ca. 60 μm) but different stabilities to study the hydrogenation of benzene to cyclohexene.E. coli cells containing OxdB and different silica particle concentrations  were prepared with 65 % SiOH silica particles  , following the same pattern as the plot for the conversion in the emulsion after ) Figure . After 4Finally, the total interfacial area in emulsions prepared at different concentrations of 65 % SiOH silica particles was calculated as described in the ESI. Figure n‐octanaloxime can be added to carry out the next cycle after homogenisation. The recyclability study was evaluated for 3 subsequent cycles in an emulsion stabilised with partially hydrophobic silica particles (65 % SiOH) and the conversion of the reaction measured after 1 hour is ≈6.5 % for all the cycles as shown in Table S5. The appearance of the emulsion and the average droplet diameter do not change significantly after each cycle. Hence this system can be recycled several times with no significant loss of activity, highlighting the positive effect of silica particles to protect the enzyme from the organic solvent.Another important advantage of carrying out the reaction in Pickering emulsions is that once the reaction is completed, the two phases can be separated by centrifugation. Thus, this elegant type of phase separation also enables an efficient recycling of the aqueous phase containing the biocatalyst (which often is difficult or even impossible when “standard emulsions” or “classic” two‐phase systems are used in biotransformations). The organic phase containing the product can be removed easily and a fresh organic phase containing n‐octanaloxime to n‐octanenitrile in Pickering emulsions stabilised with silica particles of different hydrophobicity using E. coli cells with OxdB enzyme. To the best of our knowledge, this is the first time the conversion of a reaction is followed systematically in emulsions stabilised by the same particle type but of increasing hydrophobicity. The highest conversion after 48 h was accomplished in an o/w Pickering emulsion stabilised with silica particles of intermediate wettability. This precise control in tuning the particle hydrophobicity enabled the preparation of two kinds of emulsion of different type but with similar stability and droplet diameter. By comparing the conversion of the reaction in these emulsions, it appears to be higher in an o/w (92 %) than in a w/o (65 %) emulsion. However, in both cases the conversion was higher than that measured from a control without homogenisation. Finally, by increasing the particle concentration upon the preparation of the Pickering emulsions, emulsions with smaller average droplet diameters were prepared. The consequent increase in the total interfacial area resulted in an increase in conversion. Furthermore, in contrast to many traditional emulsions or two‐phase systems used in biotransformations, this Pickering emulsion system could be recycled several times without significant loss of enzyme activity, proving the positive effect of silica particles in protecting the enzyme from the organic solvent.Pickering emulsions have been demonstrated to be a promising tool to perform catalytic reactions, thus representing an attractive alternative to the typical biphasic system. Here we reported the enzymatic dehydration of The authors declare no conflict of interest.As a service to our authors and readers, this journal provides supporting information supplied by the authors. Such materials are peer reviewed and may be re‐organized for online delivery, but are not copy‐edited or typeset. Technical support issues arising from supporting information (other than missing files) should be addressed to the authors.SupplementaryClick here for additional data file.


It shows that first zero-point usually falls at around 8 and 5 second window length respectively, while second zero-point usually falls between 16 and 24 and 8–16 s respectively. The value for the first zero-point can remove simple measurement errors when data are recorded once every few seconds. The value for the second zero-point corresponds well with what is known about physiological response of heart to changing load.Time series are a common data type in biomedical applications. Examples include heart rate, power output, and ECG. One of the typical analysis methods is to determine longest period a subject spent over a given heart rate threshold. While it might seem simple to find and measure such periods, biomedical data are often subject to significant noise and physiological artifacts. As a result, simple threshold calculations might not provide correct or expected results. A common way to improve such calculations is to use moving average filter. Length of the window is often determined using sum of absolute differences for various windows sizes. However, for real life biomedical data such approach might lead to extremely long windows that undesirably remove physiological information from the data. In this paper, we:  Specifications TableSignals from sensors usually contain some amount of noise that can have a negative impact on further data analysis. Usually, such signals are pre-processed with a form of a low-pass filter to remove unwanted elements. One of the typical and simplest low-pass filters is a moving average filter. In this filter a window of a certain length is continuously applied through the signal and value for the current time step is substituted for average of current and adjacent steps. The length of the window is decided either using domain's rule of thumb or by using Sum of Absolute Differences (SAD) approach. However, SAD approach might lead to too large window length for some biomedical signals, e.g. heart rate.In the regular SAD approach, a point is identified at which the SAD curve flattens. This point is then selected as the window length. We propose an extension to the SAD approach with use of third gradient, also called jerk, of the SAD curve. Zero-points of the third gradient become suggested window lengths. First point would be considered a conservative value, which preserves most information, but might also not remove all the noise. While second zero-point would be considered a liberal value, which guarantees that most noise is removed while still preserving most important information.In SAD approach, a set of filtered versions of the signal is calculated for varying window lengths. Then each of filtered signals is subtracted from the original one, point by point. Absolute values of each point difference are then summed. This results in set of sums, each for different window length. These sums when plotted form a curve that usually flattens from a certain window length, suggesting that there is no further filtering effect for longer windows. This window length is then selected as the optimal for filtering the signal for further analysis.However, SAD approach might lead to too large window length for some biomedical signals, e.g. heart rate. As we demonstrate in The loss of information is clearly visible in the examples. In In In While it might seem that a general shape of the heart rate curve is preserved after applying MA filter with a window length based on regular SAD approach, we observe that many significant details are lost. This observation is consistent with typical physiological response of heart rate that should be visible in less than a minute g(x), its derivative can be defined as its rate of change with respect to variable x. If function g(x) is a function of one variable, its derivative is the same as partial derivative, that can be calculated as a dot product of the gradient of the function g(x) and unit vector uu is equal to 1. It leads us to conclusion that in case of discrete function g(i) with step h = 1, its derivative is equal to the gradient of this function.If we consider continuous function g(i) is calculated as presented in The first gradient of the discrete function h, denoted by 2)O(h. In case of regular sampling of the discrete time series g(i) with h = 1 the gradient of g(i) can be calculated as shown in The calculated gradient may be burdened with an error defined by the limiting behavior of a step function g(i) can be considered as the third derivative of g(i), should g(i) be a continuous function. It can be understood more intuitively by comparison to the third derivative of position in physics, which is known as jerk, the time derivative of acceleration We can detect finer changes in the behavior of SAD curve using zero-points of the third gradient of the curve. The third gradient of a discrete function g(i) can be expressed by Calculation of zero crossing point of the third gradient of function If the subsequent zero-points are located less than 3 time-steps apart, then we suggest combining them with the resulting value calculated as a ceiling of an arithmetic mean of these zero-points .(4)iz−c=TS. First, in addition to the input signal, we specify N as a maximum length of SAD curve. In line 1 we create an empty list that will later hold values of sum of absolute differences for different window length of a moving average filter. These values are calculated in lines 2 and 3.In In line 4 we calculate third gradient of the SAD curve. Definition of gradient function is provided earlier in this section and implementation of such function is available in major data analytics tools, e.g. in NumPy. Finally, in line 5, we extract only the sign of third gradient values and check for change in this sign. We select first three such places in the curve. The algorithm can be further extended by combining zero-points located closely to each other.Later, in method validation, we demonstrate that the first zero-point of jerk corresponds to 5–8 s long window for moving average filter. It is therefore safe to assume that any signal variation in that range would be related to simple measurement errors when data are recorded once every few seconds. Second zero-point of jerk corresponds to 16–24 s window for moving average. In that range physiological effect should already be visible As a result, there are at least two values that could be applied in further processing. We suggest two possible approaches to how these values can be used. First approach uses one of these two zero-points of jerk as the window size for a moving average filter and any further calculations are performed in a standard way on the filtered signal. Second approach is approximate and combines the two values together with an uncertainty factor inspired by the interval type-2 fuzzy set footprint of uncertainty Two of the most typical cumulative measures used to evaluate the condition of a person based on the heart rate trace are: (1) longest period above threshold value of HR, and (2) area under the plot for HR greater than the threshold value. We present how these measures are formally calculated, before using them for validation.thr of HR we define S to be the set of periods ms for the time series HR, where each ms is a series of subsequent values from HR having values HR(i) ≥ thr  ≥ thr . Total n a set L :(7)lmax=In 1l – length of the longest segment above the threshold for sampling time equal to 1st zero-point of jerk2l – length of the longest segment above the threshold for sampling time equal to 2nd zero-point of jerkWe define length adjusted by the uncertainty factor as given in a = 11 – distance between 0 and point P1b2 – distance between points P1 and P2b1 and P2 – window sizes corresponding to 1st and 2nd zero-point of jerkPWe define uncertainty factor in 1P=5, 2P=22, 1=950l, and 2=1298;l we have 1b=5 and 2b=17 and consequently:For example, for 1A – area under the HR curve above the threshold for Moving Average filter with window length equal to 1st zero-point of jerk.2A – area under the HR curve above the threshold for Moving Average filter with window length equal to 2nd zero-point of jerk.We define area adjusted by the uncertainty factor in 1= 66,253A, and 2= 57,314A we obtain:Using values from the example with We demonstrate the validity of the proposed method in three different ways. First, we show that most important characteristics of the heart rate curve are preserved, when using first or second zero-point of third gradient of SAD as the window length for a MA filter. We also make a connection between values of the zero-points and physiological phenomena related to heart rate development. Subsequently, we demonstrate improved results in calculating longest period over a given threshold value, based on the filtered data. Finally, we discuss an impact of MA filtering on integral of heart rate over a given threshold value. It is another common statistic, that might be impacted in a non-obvious way by filtering.We observed earlier, in We applied the modified SAD approach using zero-points of third gradient, as defined in the previous section, to the same PAMAP2 data. First three zero-points were calculated. For the first zero-point values ranged from 5 to 8 s for different persons. For the second zero-point values ranged from 16 to 24 s. For the third zero-point values ranged from 21 to 32 s.It is interesting to notice that all these zero-points lie with the physiological response range for the heart rate Based on these observations, first and second zero-points seem to be of most interest. With first zero-point being a safe choice. While the second zero-point is potentially optimal, balancing smoothing effect with detail preservation.In We can notice, that in each case the shape of the heart rate curve is well preserved. All characteristic points that were misrepresented using standard SAD approach, are represented correctly with the modified SAD approach.In In In The proposed approach was further validated using PhysioNet dataset “Simultaneous physiological measurements with five devices at different cognitive and physical loads”. Results for three example subjects are presented in Window length based on the 2nd zero point of 3rd gradient of SAD curve ranged from 8 to 16 s. The resulting heart rate signal preserved all the major variations of the original, while removing the noise.Based on the shape of the heart rate curve, we conclude that using modified SAD approach as proposed in the previous section, preserves characteristic points including shape, absolute value, and relative values.Heart-rate related measures are one of the most commonly used markers in monitoring performance, fitness and fatigue of a person during physical effort HR series above the threshold value thr = 105 bpm are presented in HR series above the defined threshold value for the second zero-point of the third gradient. This increase is equal to 90%, 36% and 35% respectively. Including the uncertainty factor UF defined in the previous section, this increase was lower, equal respectively to 8%, 1.7% and 1.8% for persons 1, 2 and 5, as showed in For an average middle age person zone 2 of the heart rate, corresponding to the middle exercise intensity, is a typical aerobic target. The bottom value of heart rate in zone 2 is equal to the 60% of a maximum heart rated. We approximate this value to be equal to 105 bpm for average young to middle-aged person. The results of calculations of the longest period of In case of person 1 we can observe a drop in the heart rate around 2600s., however the heart rate drops slightly below the threshold, by about 1–2 bpm. The behavior of the HR curve suggests that this decrease is insignificant and should not influence the period of heart rate above the threshold of 105 bpm. In case of the original signal and signal filtered using window length corresponding to the first zero-point of the third gradient, this decrease in the heart rate is reducing the period of the HR above the threshold by almost half, from about 43 min to 22 min, what can be crucial during training of a middle aged person.Cases for person 2 and person 5 are similar, in terms of a slight and short decrease of heart rate, to person 1. In both cases the HR drop below the threshold is by 1–2 bpm and should not be considered in the determination of the value of the longest period above 105 bpm. The increase of this period for persons 2 and 5 for filtered signal with window length equal to the second zero-point is not as large as in case of person one, but still significant, equal to around 5 min.For persons 3 and 4 the proposed method did not influence the length of the longest period of heart rate above the threshold. Both HR curves show moments of decreased activity, lasting for few seconds, before the considered period in case of person 3, and after the considered period in case of person 4.It is clearly visible that the longest period above the threshold value calculated for window length equal to 230 is significantly longer in case of persons 1, 2, 3 and 5. For person 1 this value is very similar to the result calculated for the window length corresponding to the second zero-point of the third gradient. However, in case of persons 2 and 5 these values cannot be taken into consideration in analysis of the heart rate in terms of person's performance and fatigue. As we can observe on the first plot of The method was also validated using Physionet dataset “Simultaneous physiological measurements with five devices at different cognitive and physical loads”. The threshold was set to 95 bpm to evaluate the heart rate signal recorded during the period of five-minute walking on the treadmill at a moderate speed. As it can be observed in The longest period above the threshold value calculated for window length equal to 540 for the three analyzed signals is presented in One of the parameters recommended in analysis of serial measurements in medical research is area under the curve. It can be interpreted as a kind of weighted average of the analyzed signal and can be used for finding and evaluating patterns among groups of monitored patients Obtained results show that proposed method of filtering does not affect significantly the area under the HR curve and this parameter can be used in analysis of the time series. On the other hand, we can see that filtering affects the amplitude of the signal and applying Moving Average filter with long window is not recommended as it significantly affects the area under the curve. This problem might manifest often when measuring heart rate under a load lasting more than several minutes. Due to aforementioned cardiac drift, average heart rate will increase with time even without changes to the exercise load. It will result in the time series of heart rate becoming non-stationary. What will lead to distortion of measurements if a Moving Average filter with long window is applied.The area under the HR plot and above the threshold value, as well as the total area under the HR plot, were evaluated also for the PhysioNet Simultaneous Physiological Measurements dataset and are presented in The values of the total area under the HR plot have slightly decreasing tendency as the size of the window increases, however this decrease is not very significant for the filters based on the first and second zero-point of the third gradient. It is much greater in case of the window size of 540 and is equal to 20–30%, as it can be seen in Sum of absolute differences is an established approach to selecting window length for a moving average filter. The underlying properties of a process generating the data might, however, render results of such approach detrimental to further analysis for some signals. Biomedical signals, such as heart rate, are a good example, since underlying physiology is often well understood. As we demonstrated regular SAD approach results in long windows that undesirably remove physiological information from the data.The method outlined in this paper provides a formalized alternative to the regular sum of absolute difference approach. It extends it with calculation of zero-points of third gradient, sometimes called jerk, to detect finer variations in the shape of SAD curve.We demonstrated validity of this approach on the PAMAP2 Physical Activity Monitoring Data Set, an open dataset from the UCI Machine Learning Repository, as well as on the PysioNet Simultaneous Physiological Measurements dataset. It showed that first zero-point usually falls at around 8 and 5 second window length respectively, while second zero-point usually falls between 16–24 and 8–16 s respectively. The value for the first zero-point can remove simple measurement errors when data are recorded once every few seconds. The value for the second zero-point corresponds well with what is known about physiological response of heart to changing load The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.


Fish adaption behaviors in complex environments are of great importance in improving the performance of underwater vehicles. This work presents a numerical study of the adaption behaviors of self-propelled fish in complex environments by developing a numerical framework of deep learning and immersed boundary–lattice Boltzmann method (IB–LBM). In this framework, the fish swimming in a viscous incompressible flow is simulated with an IB–LBM which is validated by conducting two benchmark problems including a uniform flow over a stationary cylinder and a self-propelled anguilliform swimming in a quiescent flow. Furthermore, a deep recurrent Q-network (DRQN) is incorporated with the IB–LBM to train the fish model to adapt its motion to optimally achieve a specific task, such as prey capture, rheotaxis and Kármán gaiting. Compared to existing learning models for fish, this work incorporates the fish position, velocity and acceleration into the state space in the DRQN; and it considers the amplitude and frequency action spaces as well as the historical effects. This framework makes use of the high computational efficiency of the IB–LBM which is of crucial importance for the effective coupling with learning algorithms. Applications of the proposed numerical framework in point-to-point swimming in quiescent flow and position holding both in a uniform stream and a Kármán vortex street demonstrate the strategies used to adapt to different situations. Furthermore, a unique energy-saving behavior termed Kármán gaiting is observed in rainbow trout and other fishes when swimming behind a bluff body in the flow, which is characterized by large-amplitude lateral motion of the body occurring at a low frequency5. In addition, fish may exploit the vortices shedding from its leading one or its fins to improve its swimming performance8, of which the propulsion mechanism can be further revealed by separating the drag and thrust9. In nature, fish are able to achieve the above mentioned behaviors in a very quick and efficient way, with which current man-made vehicles cannot compete. Therefore, it is important to achieve them in numerical simulation, with which researchers are able to understand the design concepts of fish, and to put these concepts into man-made vehicles.It has long been observed that fish can adapt to different environments and achieve their goals optimally. These adaption behaviors are essential for survival since they allow a fish to obtain and save energy as well as avoid risks. A typical example of adaption behavior is prey capture, in which the fish is trying to reach a target with given time . Another important behavior that has been observed in many fishes is rheotaxis12. Among them, visual signals from the eyes and hydrodynamical signals from the lateral line system are the most commonly used, allowing fish to perform a variety of adaption behaviors14.The mechanisms underlying these adaption behaviors are complex and have not been fully understood. The cooperation between the sensory system, the neural system and the muscles, which forms a precise and robust feedback control system, is of primary importance. The sensory system is responsible for continuously collecting information about the environment which is input into the neural system so that the fish can update its knowledge of its surroundings in real time. Based on this information, the fish may change its swimming kinematics via the muscles to achieve its goals. During over 400 millions years of evolution, a variety of sensory systems have emerged in different fishes based on proprioceptive, visual, tactile, olfactory, electric and hydromechanical signals16 based on the correlation between the pressure on the fish body and the position with respect to an object or a point in the flow field. The effect of a specific action in the control problem is typically predicted by using a simplified flow dynamics model . However, the development of a robust and accurate controller is still a challenging problem, due to incomplete flow models and nonlinear historical influence of past actions. Given the difficulty of acquiring a reliable policy to reproduce adaption behaviors, swimmers in numerical simulations are often forced to swim in preferable configurations, thus making them not entirely free swimming20.Significant effort has been directed towards reproducing the adaption behaviors. Feedback control design has been adopted in robot fish21 or the collective motion of fish23 and dipole swimmers24, autonomous thermal soaring of UAVs26 and birds28, lift generation of UAVs32, and the navigation of microswimmers33. The method has two remarkable advantages. The first advantage is that the swimmer does not need to possess any prior knowledge of the environment. Instead, it only needs to sample the information about the environment through trial and error and so there is no need to simplify the flow dynamics. The other advantage is that the influence of the historical states can be easily taken into consideration. Therefore, the correlation between action and its effect can be accurately captured even when there is a delay between them and there are measurable historical impacts from historical actions.In order to address the challenge in developing robust and accurate controllers, a novel control method based on reinforcement learning (RL) has been proposed to study bio-inspired swimming and flying problems including the individual34. Here the environmental flow information is updated by using an immersed boundary–lattice Boltzmann method (IB–LBM) which makes excellent use of the advantages of the lattice Boltzmann method (LBM) and the immersed boundary method (IBM)41. Compared to traditional numerical methods based on the Navier–Stokes equations, the IB–LBM is more efficient43 and is a promising alternative in combining with reinforcement learning methods. The IBM is a methodology for dealing with boundary conditions at interfaces based on meshes that do not conform to the shapes of the immersed boundaries. In the IBM, the mesh generation is very easy even for complicated geometries. The mesh movement and mesh regeneration are not necessary for flows involving moving boundaries and fluid–structure interaction (FSI) problems. Therefore, it is very convenient to handle cases involving topological change of the computational domain, complicated geometries and large movement of boundaries48.A challenge of the method is that in order to obtain a robust control policy, the learning agent must repetitively explore a large number of different possible actions in many environment states. Thus, an efficient way to obtain the environmental flow information is of crucial importance for the agent to learn in a reasonable time, which is a great challenge for numerical simulationsIn this work, a deep recurrent Q-network (DRQN) is coupled with an adaptive-mesh IB–LBM FSI solver for the simulation of the FSI system mimicking fish adaption behaviors including prey capture in still water, rheotaxis in a uniform flow and Kármán gaiting in a Kármán vortex street. It should be noted that we recognize each component in the computational framework is not new. However, the combination of DRQN and adaptive IB–LBM FSI solver provides a very efficient tool to study the fish behaviours in complex environments. In addition, a new mathematical model of fish-like swimming is developed with amplitudes and frequencies altered smoothly every half period to implement highly maneuverable motions for the swimmer swimming in highly complex and dynamic flow environments. The feasibility and efficiency of the combined DRQN and IB–LBM method will be demonstrated by applying it in fish behaviors in three typical cases.The rest of this paper is organized as follows. Numerical models, IB–LBM and DRQN are introduced in “49 as shown in Fig. L is the body length, and The shape of the 2D swimmer model here is reconstructed from the cross-section of a troutn-th half cycle, the mid-line lateral displacement is determined byt is the time, h is the waveform function described by51. It allows the swimmer to change its periods, amplitudes and wavelengths smoothly and arbitrarily every half period. Therefore, the swimmer model is able to choose appropriate combinations of different kinematics to achieve different maneuvering movements such as accelerating, decelerating and yawing, which enables the fish to handle complex and fast-changing environments.The motion of the body includes three parts as shown in Fig. entclass1pt{minimax, y) according to:m is the mass of the fish, The translational and rotational motions of the swimmer are determined by FSI in the global coordinate system  in modeling lid-driven cavity flows, and found that the CPU time of each step for the LBM is about 1/3 of that for the FDM, indicating the LBM is more efficient than the FDM in modeling fluid dynamics, which is of crucial importance for the coupling with the reinforcement learning method as each learning application normally requires thousands of simulation cases. The FSI process implemented by the IB–LBM is shown in Algorithm 1.Yoshino et al.57. One of the most successful methods in reinforcement learning is one-step Q-learning. In this study, the DRQN58 is used where a one-step Q-learning is coupled with a three-layer long-short-term-memory recurrent neural network (LSTM-RNN).Deep reinforcement learning combines reinforcement learning with an artificial neural network to approach human-level control in complex real-world problems18. During this process, the agent must be able to sense a defined set of parameters representing the state of the environment (denoted by s) and take actions (denoted by a) to affect it. Each action is assessed with a scalar number called the reward (denoted by r) whose value indicates whether the agent moves towards or away from the goal by taking the action. In order to achieve the goal, the agent must seek actions that maximize its expected cumulative rewards in the long run  which is defined as,n-th state and action, n-th action, 57. The principle in Q-learning is that the agent explores different actions in different states and evaluates the actions with Q, so that when the state reoccurs, the agent will choose the optimal action to achieve its goal.Q-learning describes a general process of an agent learning how to achieve a goal during prolonged and continued interaction with its environment by trial and errorQ, which can generalize past experience to new situations57. In this work, an LSTM-RNN composed of three layers of 64 LSTM cells and a linear output layer is adopted. In order to find the optimal action-value function, the neural network is iteratively updated by minimizing the temporal difference errorQ. This can be achieved by updating the network weights via gradient descent methods,59.Q-learning suffers from the classical “curse of dimensionality” problem, where the data and computational resource required grow exponentially with the dimensionality of the state and action spaces. Deep reinforcement learning has partly resolved this problem by approximating the action-value function with a neural network 57. The replay memory is used to store large numbers of quadruplets Q. This technique breaks the correlation between the samples to avoid local optimization57. The sizes of the replay memory  in Eq. . It is un of each action, the agent detects a state 60 is used to select actions, with which the agent chooses the optimal action  with probability The detailed interaction procedure is summarized in Algorithm 2 where the agent-environment interaction is broken into 23, this work incorporates the fish position, velocity and acceleration into the state space in the DRQN; and it considers the amplitude and frequency action spaces as well as the historical effects.It should be noted that compared to existing models of learning for fish54. Here we further provide application-specific validations by focusing on the frequency of vortex shedding from a cylinder in a uniform flow and the swimming speed of an anguilliform swimmer in a quiescent flow. The cases are conducted with 20 computational cores on a workstation with Intel Xeon CPU E5-2650 and OpenMP.The current flow solver has been validated in previous publicationsf of the Kármán vortex street by varying the Reynolds number U is the incoming fluid velocity, D is the diameter of the cylinder, and A uniform flow over a stationary cylinder is conducted to determine the frequency 63 is shown in Fig. St, The Strouhal number mentclass2pt{minimmentclass2pt{minim68T is the waving period. Body length of the fish L, fluid density T are chosen as the characteristic values. To compare with the result in Ref.68, the parameters are selected as: Here an anguilliform swimmer swimming in a quiescent flow is conducted to validate the capability of the current fluid solver for modelling a self-propelled swimmer. The half thickness of the swimmer is described as68. It is noted that the balanced swimming velocity of the present study is smaller than that of Kern and Koumoutsakos68 and Case b (with divergence-free correction of body motion) of Gazzola et al.69, but agrees well with Case a (without divergence-free correction of body motion) of Gazzola et al.69. As the divergence of body motion does not affect the learning process considered in this work, and thus is not corrected in order to save computational costs.The forward velocity Three applications of the coupled DRQN and IB–LBM are conducted to demonstrate the effectiveness of this approach for the investigation of fish behaviors in different flow environments: point-to-point swimming in a quiescent flow mimicking prey capture behavior, position holding swimming in a uniform flow mimicking rheotaxis behavior and position holding in a Kármán vortex street behind a half-cylinder mimicking the Kármán gaiting behavior. The simulations are conducted with 20 computational cores on a workstation with Intel Xeon CPU E5-2650 with OpenMP.L is placed in a circular area with radius O from any position within the circular area and arbitrarily given orientation. This goal is reflected by defining a reward asO.Here we apply the coupled approach to the point-to-point swimming of a sub-carangiform swimmer in a quiescent flow. The swimmer of length The swimmer propels itself by periodically generating a travelling wave propagating from head to tail, as defined by Eqs.  and 3)..3). In o21, Novati et al.22 and Verma et al.23 . However, ignoring the flow information will make the learned policy inaccurate as shown in Fig. The state is an important component in the DRQN. Theoretically, it should include the information of the swimmer and the ambient flow. The information of the swimmer includes the body waveform, position, pitch angle, velocity, angular velocity, acceleration and angular acceleration of the body. The flow information includes the flow velocity and pressure in the whole flow field. The historical evolution of the flow should also be considered. Therefore, it is impossible to consider the flow information as a simple definition of the state. One way to resolve this problem is to consider the information of the swimmer only as that in the work of Gazzola et al.71. Therefore, it is reasonable to use these quantities to define the state.Here, we propose a method to consider the influence of the flow information in the states without having to deal with the complexity of the flow. Considering the flow is developed from the historical actions and fish dynamics, it is partially reflected by the dynamics and actions of the swimmer in the past time. If the whole historical dynamics and actions are considered in the states, the flow information is naturally included. However, tracking the whole historical dynamics and actions is memory and time consuming and not necessary since the far history only has minor influence on the flow dynamics at current instant. Our simulations show that only considering the historical dynamics and actions of the fish in the last 4 periods is enough to capture the flow dynamics. In order to further reduce the complexity, accelerations are not considered in the state. The state is thus defined by a tupleThe simulation is performed for a Reynolds number of L and 5L and the initial orientation O (episode 338). After learning for 545 episodes, it successfully finds a tortuous path to reach the center O. However, at episode 3890, it has learned how to directly swim towards its destination. This is further demonstrated by Fig. The learning process is divided into a series of episodes. In each episode, the initial position L, 2L, 3L and 4L. In the other 4 cases, L while Figure Figure see Fig. a. Then isee Fig. b and smasee Fig. c to perfsee Fig. e and rigsee Fig. damplitudsee Fig. f.Figure Here we apply the coupled approach to the rheotaxis swimming of a sub-carangiform swimmer in a uniform flow. Its goal is to hold position in a circular area of radius In this case, the swimmer is able to change both the wave period and amplitude every half swimming cycle. The period action base is defined as Note that the information of the position The simulation is performed for a Reynolds number of O of the swimming area with its initial orientation angle The swimmer is initially placed in the center mentclasspt{minimamentclasspt{minimamentclasspt{minimaFigure The lateral movement of the tail when the swimmer is holding position is presented in Fig. 72. Furthermore, a fish in the Kármán vortex street selectively explores the flow and swims back and forth when slaloming around the incoming vortices73, which makes the encountered flow field more variable and unpredictable.Here we apply the coupled approach to the position holding behavior in a Kármán vortex street. The Kármán vortex street is an example of a drag wake, characterized by a repeating pattern of swirling vortices. It is a complex but mostly predictable flow environment. The abundant vortices make the fluid dynamics in different areas highly diverse and unsteady and there is always a certain amount of unpredictable variation in the vortex behaviors73 conducted a series of experiments to observe the kinematics of live rainbow trouts in the Kármán vortex street, in which the fish were placed in the wake behind a D-shaped cylinder. They found that the midline kinematics of the fish could be represented as a superimposition of four midlines generated by four motion components: lateral translation, body bending, body rotation and head motion, whose contributions were respectively Liao and Akanyeti50. The Strouhal number of the vortex street is L. The fish is trained in a rectangular area of 73.A D-shaped cylinder of diameter 72 who found that nearly The hydrodynamic forces exerted on the fish are also included in the state to better capture the dynamic nature of the flow field. In order to reduce the complexity, body rotation and head motion are not considered, which is based on the observation by Akanyeti and LiaoThe simulation is performed for a Reynolds number of L to 2.5L. Figure 50 in Fig. 50 in live rainbow trout, the simulation-predicted area where the swimmer tends to stay overlaps the majority of part of that observed in experiment.The fish is initially placed in the mid-line of the swimming area with its initial distance between the head and the cylinder randomly varying from 1.550. The Reynolds number of the cylinder in our simulation is 300 compared with 18,000 in the experiment. The resultant tail-beat frequency agrees quite well with that of the experiment but the tail tip amplitude is slightly lower. The undulation wave speed is also slightly slower than that in the experiment due to the smaller wavelength. It should be noted that the typical wake of the D-shaped cylinder in the high Reynolds numbers as observed in experiment is the well-organized turbulence vortex street (see Ref.17), which is the foundation of the successful comparison between experiment and simulation as shown in Fig. The averaged undulation kinematics for 50 successful cases are shown in Table  see Ref., which iThe vorticity contours and the pressure distributions on the fish surface at different instants when the fish is holding its position in the vortex wake are shown in Fig. 4.4 Fig. a and b, .04 Fig. c and d, .52 Fig. e and f, =56 Fig. e and g, However, the fish could occasionally get trapped in the low pressure area of the vortex center. If the swimmer is not able to properly synchronize with the vortices, it would move downstream with the vortex and lose its stability. In this case, the fish must find a way to escape from the vortex in order to hold position for a long period. Figure .88 Fig. a and b, .88 Fig. c and d a.88 Fig. e and f, .84 Fig. g and h a.84 Fig. i and j, 50. Highly asymmetrical corrective undulation is performed when fish is trapped in the vortices, which enables the fish to escape from the vortex center and hold its position or maintain its stability.The fish adaption behaviors in complex environments have been numerically studied. A recurrent Q-network is first coupled with an immersed boundary–lattice Boltzmann method to simulate the adaption behaviors of a fully self-propelled smart swimmer. Three different behaviors are studied with this swimmer: point-to-point swimming in a quiescent flow, rheotaxis swimming in uniform flow and position holding swimming in a Kármán vortex street. The swimmer utilizes only the position, velocity or acceleration information extracted from the environment to learn to achieve specific tasks. By considering the historical information, the swimmer learns suitable policies to achieve different tasks, demonstrating that deep reinforcement learning is able to extract useful characteristics from flow structures with various complexities. During the point-to-point swimming, the fish performs rapid turning to face the target and then swims directly to it with different initial distances and orientation angles. During rheotaxis swimming, the fish rapidly aligns its body with the uniform flow and holds position for more than 200 periods. Two types of wake vortex patterns are identified for rheotaxis swimming. The vortex patterns produce jet flow in different directions in the wake to facilitate a dynamic balance of the hydrodynamic forces. During position holding in a Kármán vortex street, the fish utilizes the ambient vortices to achieve balance and save energy. The robust position holding in the Kármán vortex street only happens in a specific flow area which is in reasonable agreement with the experimental observation of Liao


The principle of least effort has been widely used to explain phenomena related to human behavior ranging from topics in language to those in social systems. It has precedence in the principle of least action from the Lagrangian formulation of classical mechanics. In this study, we present a model for interceptive human walking based on the least action principle. Taking inspiration from Lagrangian mechanics, a Lagrangian is defined as effort minus security, with two different specific mathematical forms. The resulting Euler–Lagrange equations are then solved to obtain the equations of motion. The model is validated using experimental data from a virtual reality crossing simulation with human participants. We thus conclude that the least action principle provides a useful tool in the study of interceptive walking. In classical mechanics, equations of motion can be generated from Maupertuis’s principle of least action or the closely related Hamilton’s principle3. Theories of modern physics, including quantum mechanics and general relativity, also have formulations in terms of principles of least action5. Such formulations mostly adopt calculus of variations in which the dynamics selected by the theory, e.g. the path taken by a particle, is governed by the stationarity of the suitably defined action. In the context of classical mechanics, the action is given by the time integral of the Lagrangian along the path of motion.Principles of least action play a fundamental role in many areas of physics. They were preceded by Fermat’s principle or the principle of least time in geometrical optics6, and subsequently a diverse range of phenomena such as crowd behavior7 and even mental effort8. These studies have found predictable patterns in distributions arising from human behaviors, which individually are variable and unpredictable. However, it is reasonable to expect that there should typically be a tradeoff between effort and other quantities. Therefore, a more complete description of human behaviors may plausibly be provided by a least action principle in which effort is one component. In this report, we take this approach and propose a principle of least action, which is used for modeling human walking behavior.In the study of human behavior, these principles have inspired the principle of least effort, which was used to explain the power-law form of the rank-frequency distribution of words in the English language9, and human bipedal locomotion in particular has a long evolutionary history which resulted in a uniquely economical cost of transport10. Human walking exhibits a parabolic curve when the cost of transport is plotted against speed; the optimum walking speed is related to the pendular mechanism of human walking12. Here we are concerned with trajectories of interceptive walking15. It is widely observed that during acts of interception, certain paths tend to be taken, despite the inherent variability of human action. This study presents a quantitative model that characterizes a wide range of such individual walking trajectories based on a few key assumptions, thus capturing the essential features of interceptive walking. In doing so, we ignore the specific biomechanics16 and gait patterns19 of walking, which are themselves subjects of active study.The study of locomotion requires an integrative approach23. Finally, we discuss the meaning of the model and fitting parameters.In the following sections, we first propose the Lagrangian mechanics of interceptive walking by defining a Lagrangian and solving the resulting Euler–Lagrange equation, which yields the path of stationary action. Inspired by the principle of least effort, we postulate that the Lagrangian consists of effort as well as a quantity we call security. Two specific forms of the Lagrangian are considered and their respective equations of motion are derived. Then the equations of motion are verified by fitting the solutions to positional time series data from a virtual reality crossing experiment. The experiment simulates a road crossing situation in which a pedestrian should cross between two moving vehicles, hence intercepting the gapT and potential energy U, we here consider a Lagrangian of the formE denotes the effort and S the security in walking. It is known that the rate of metabolic energy consumption during walking on a level surface is proportional to the walking speed squared27. We thus assume that the effort for walking increases in proportion to the square of the walking speed v, and write, up to multiplicative and additive constants:E dimensionless. It turns out to correspond to the maximum walking speed (see below).Taking inspiration from Lagrangian mechanics, where the Lagrangian is defined as a is taken into account by a function g(a) (again up to multiplicative and additive constants), which we presume to be a function with zeros at On the other hand, security is related to the motivation of the pedestrian to reach a goal, while avoiding danger, and likely to depend on the walking speed and acceleration. With regard to the speed, it is reasonable to assume that the pedestrian should feel safe when she/he can cross the gap at high speeds. Measuring the speed in units of g(a) below. The difference between Eqs. (We discuss two different possible choices for een Eqs.  and 3)  g(a) bely. Regarding effort, this means that we are assuming a uniform and level walking terrain; Eq. (27), but this is not considered here. As for security, the pedestrian would surely feel less safe in the middle of the crosswalk than elsewhere. By imposing boundary conditions, however, we are already placing a positional constraint; we thus reason that the sense of safety is fully accounted for by the speed at which the pedestrian is passing through the gap  in Eq.  that yield analytic solutions of the simplified Euler–Lagrange equation [Eq. (Before we discuss the specific form of ) in Eq. , we derim of Eq. . We thusL on v is illustrated in Fig. a in the quadratic case is plotted with a blue curve in Fig. The simplest convex function one can consider is the quadratic equation mentclass2pt{minimmentclass2pt{minimEquation  has unstEquations  and 13)13) obey L on a in the logarithmic case is plotted with a red curve in Fig. g(a) is asymmetrical and steeper on the left side. Plugging into Eq.  with four magnetic counters that track movements. A Velcro belt connected to the treadmill was worn for suppression of vertical and lateral movements, and a handrail was placed for safety. Each participant wore a commercial virtual reality headset connected to a standard desktop PC. The headset portrayed a realistic view of a typical crosswalk in Korea in y from the crossing point taken as the origin and is recorded to generate positional time series. The initial position Figure We fit Eqs.  and 17)17) to thWhen Eq.  was fit There were individual variations in the slope mentclass2pt{minimmentclass2pt{minimWe have presented a model for goal-directed human walking behavior, based on a principle of least action. The approach can be considered as a generalization of the principle of least effort, incorporating another term called security. Walking behavior results from the assumption of three simple terms making up effort and security. The resulting equations have been found to fit experimental data from a virtual reality road-crossing experiments. While the equations were derived from a Lagrangian, an equivalent Hamiltonian formulation can also be constructed see .28. However, we presume that the form of Eq. . Accordingly, there are individual variations in the endpoint the participant chooses. In addition, we expect that physiological differences lead to differences in constants One may note the limitations of using a treadmill, which may change walking behavior and also constrains the participant to walk in a straight line. Additional walking simulations have been done in which the participants walk freely in a room, with sensors used to detect their positions. The results were again consistent with the model (data not shown). However, an additional dimension is added due to the freedom in the walking direction. There were no interesting features in the dynamics of the angle, which was generally held constant. The choice of the angle exhibits also individual variations; this is beyond the scope of the current model.a| in place of a in Eq.  affecting the trajectory and also in that it produces an analytical solution for the entire walking trajectory, which is possible due to the simplicity of the walking task. Guy et al. instead simulate collision avoidance in crowds by restricting the direction of movement based on the environment at each simulation step. The least action model may be extended to include interactions with other pedestrians and walking directions; this is left for future study.We note that our model has similarities with other models of pedestrian behavior, e.g., in Guy Supplementary Information.


Clinical use with prostate gold fiducial markers is presented.To present an open‐source software (Projections of regions of interest (ROI) in the reference frame of the electronic portal imaging device are computed offline for different gantry angles before the first treatment fraction. During treatment delivery, the contrast of portal images is automatically adjusted using a histogram equalization algorithm. The projections associated with the current gantry angle are then superimposed on the images in real time. This allows the therapist to evaluate if the imaged structures of interest remain within their respective contours during treatment delivery and to potentially interrupt the treatment if deemed necessary. The spatial accuracy of the method was evaluated by imaging a ball bearing phantom in a set‐up where the position of the projected ROI is highly sensitive to gantry angle errors. The visibility of fiducial markers during one fraction of seven different volumetric modulated arc therapy (VMAT) prostate treatments is characterized.The geometric validation showed a negligible systematic error μ < 0.1 mm for the position of the projections. The random errors associated with the time accuracy of the gantry angle readout were characterized by standard deviations σ ≤ 0.6 mm. The VMAT clinical treatments showed that the fiducial markers were frequently visible, allowing for a meaningful clinical use.The results demonstrate that the method presented is sufficiently accurate to be used for intrafraction monitoring of patients. The fact that this method could be implemented on many modern linacs at little to no cost and with no additional dose delivered to the patients makes this solution very attractive for improving patient care and safety in radiation therapy. For example, medical linacs are now equipped with MV and kV imaging systems as standard and techniques such as portal imaging, kV radiography, or cone‐beam computed tomography (CBCT) are typically used at the beginning of each treatment fraction to minimize interfraction positioning errors. During treatment delivery, intrafraction positioning errors may occur due to internal organ or patient motion. Margins are usually applied to the target volume during treatment planning to account for these effects. Motion management systems can also be used to monitor the patient position during treatments. Available solutions include optical surface imaging,The intrafraction monitoring systems mentioned above have one or more disadvantages. First and foremost, the costs associated with additional hardware, such as optical, ultrasound, and RF implants monitoring systems, can be a disincentive, while integrated magnetic resonance (MR)‐Linac solutions are simply out of reach for most radiation therapy centers because of their high costs. Kilovoltage imaging techniques give additional radiation doses to the patient which may pose a health risk, especially when many fractions are delivered.Cine‐MV imaging has been the subject of many publications.https://github.com/CHUSRadOncPhys/FluoMV) and instructions for installation and use are provided.The purpose of this work is to present an open‐source software that uses superimposition of contours of projected regions of interests (ROIs) on cine‐MV images to monitor intrafraction motion of patients with Elekta linacs (Elekta Limited). A validation of its spatial accuracy and a presentation of its clinical use with patients treated for prostate cancer using volumetric modulated arc therapy (VMAT) with implanted gold fiducial markers are also presented. By making our software open source, we hope to give clinicians a useful tool to assess the accuracy of intrafraction positioning of patients in their clinics. The software can be downloaded on GitHub  and has been tested on Synergy and Infinity linac models. The software includes a ROI projection module that can be run on a quad‐core CPU or on a NVIDIA graphics processing unit (GPU) for faster computation, and an acquisition module that runs on the computer connected to the EPID. The EPID is an amorphous silicon detector panel (XRD 1640 AL5 P) that produces images of 1024 × 1024 pixels with a digital resolution of 16 bits per pixel. The EPID is located at a distance of 160 cm from the radiation source and has a field of view of 25.6 cm × 25.6 cm at the linac isocenter.2.AThe projections are computed from the ROIs defined by the radiation oncologist or the planner in a treatment planning system (TPS) and exported as a DICOM RTSTRUCT file. To compute the two‐dimensional projections of a three‐dimensional (3D) ROI, the linac geometry and the coordinates of the plan isocenter provided by the DICOM RTPLAN file are used. The first step of the projection algorithm is to create a 3D voxel grid with coordinates that matches those of the planning CT images. This grid contains binary values, where all voxels have a value of 0 except those inside the ROI to be projected. Secondly, for a given gantry angle and for each group of 2 × 2 detector elements of the EPID,For a given patient, this projection process is done only once, before the first fraction of the treatment, for every desired ROI, at intervals of 0.5° over 360°. The data are saved in a file accessible by the computer that controls the EPID. At the time of treatment, the therapists use the graphical user interface to enter the patient identification number, select the prescription, and the contours they want to visualize during treatment delivery. Finally, the projected contours are superimposed on the live MV EPID images during treatment delivery. Since the projections are precomputed, the time efficiency of the projection algorithm has no effect on the real‐time display of the contours during the treatments.2.BXISL.dll dynamic link library provided by the EPID manufacturer to directly control the detector and read in real time the frame buffer of the EPID frame grabber board. The acquisition mode is set to Free running which corresponds to the situation where the panel sends out images continuously to the frame grabber at a frequency equal to the inverse of the image integration time. The integration time is set to the shortest time available, 433 ms, and a timestamp corresponding to the time when the panel readout is completed is assigned to each image. The timestamp is based on the QueryPerformanceCounter function of the Microsoft Windows API and has a resolution < 1 μs.The software used to visualize the clinical treatments is installed on the Elekta iView computer located in the control room of the linear accelerator. The software includes two acquisitions modules that are executed in parallel on different processor threads: the image acquisition module and the iCom messages acquisition module. The image acquisition module uses the iCOMClient.dll provided by the linac manufacturer to read the iCom messages sent continuously by the linac. Information on the linac state, gantry angle, dose rate, and the nominal beam energy is acquired at a rate of approximately 4 HzThe iCom acquisition module uses the dynamic link library 2.CImage processing has three purposes: (a) to produce images with minimal artifacts by calibrating the pixels response, (b) to produce images that are registered in the same reference frame as the precomputed ROI projections, and (c) to adjust the image contrast to better differentiate the structures. The pre‐processing steps consist of a subtraction of a dark frame acquired prior to irradiation, followed by a pixel gain correction, an image translation, a spatial median filtering, and a pixel binning. The pixel gain calibration file and the panel position correction table used for image translation are loaded in memory at the opening of the software and are specific to each linac. The gain calibration file consists of flood field images acquired at different dose rates for a given nominal beam energy. The panel position correction table contains the average displacements of the panel in the AB and GT directionsOnce the preprocessing of the image is completed, histogram equalization is performed on pixels of value superior to a threshold in order to obtain a good contrast in the radiation‐field region and thus be able to differentiate the structures of interest. By default, this threshold is 70% of the maximum gray‐level value of the image but can be adjusted during the delivery with a scroll bar if the contrast is not deemed good enough. Pixels below this threshold are given a value of 0 and all the others have their gray‐level value redistributed between 0 and 255. Once the contrast is adjusted, the precomputed projected ROIs are superimposed using the same colors specified in the TPS to be easily recognized. In addition, a checkbox list of the names of the ROIs is included in the software to allow users to choose whether or not to display any ROI at any time during the treatment. The whole process of image preprocessing, contrast adjustment, and superimposition of contours is sufficiently fast to be executed between two consecutive image readouts.c=0 at t=0 and ends at column c=511 at t=433  ms. For the linac B‐side section, the readout starts at column c=1023 at t=0 and ends at column c=512 at t=433  ms. The exact readout time tc of a column is as follows:TS is the timestamp of the image, and αc=c when c∈0,511 and αc=1023-c when c∈512,1023. For each ROI, we choose to approximate the image acquisition time as the readout time of the column corresponding to the center of the ROI projection in the image: cx. The temporal correspondence between an image and the iCom messages is therefore an iterative process that depends on cx, which itself depends on the gantry angle. For each new image acquired, the first step consists of linearly interpolate the gantry angles of the iCom messages to the timestamp TS of the image. Then, the coordinate of the centroid of the ROI projection for this angle is read and Eq. (t(c) and the ROI projection corresponding to this new gantry angle is superimposed on the EPID image.To determine which projection is to be displayed on the image, the gantry angle at which the image was acquired must be known. As a first approximation, a gantry angle can be assigned to each image by linearly interpolating the gantry angles of the iCom messages to the timestamp of the image. However, this method neglects the effect of the time order of the readout of the columns of the panel detector elements which can be significant when the ROI projected contours are off‐centre and the gantry rotation speed is high. Physically, the panel is divided laterally (in the AB direction) into two independent sections that have their own electronic reading system. The columns of the panel are parallel to the GT axis. The two sections are read synchronously, column by column, from the outside to the inside of the panel.2.Dθ, the radiation source is at position Sθ=SAD·sinθ,SAD·cosθ in the X‐Z plane, where SAD is the source‐to‐axis distance. The projection of the object located at O→= can be found by calculating the vector:The accuracy of the position of the projected ROIs on the EPID images depends on many factors. The resolution of the treatment planning images on which the contours are initially defined, the angular resolution of the precomputed projections, and the resolution of the EPID images on which the ROIs are projected are fundamental limitations of the method. Other factors can affect the accuracy of the projections, such as the accuracy of the gantry angles associated with the images. Geometrically, gantry angle errors have an effect mainly in the AB direction. The magnitude of this effect depends on the position of the object relative to the radiation source. A geometrical analysis of the accuracy of the positions of the projected ROIs in the AB direction for different gantry angle errors and for different object positions is presented in Section u^ is the unit vector along the u axis and:where β is positive when -90∘≤θ≤90∘ and negative otherwise.The angle The accuracy of the projections also depends on the reproducibility of the panel position with respect to the gantry angle. To characterize this effect, an 8‐mm‐diameter radio‐opaque ball bearing (BB) phantom was fixed to the treatment table and positioned at the linac radiation isocenter. The EPID was then deployed at a gantry angle of 0°. Two 360° arc beams of 1000 monitor units with a field size of 10 cm × 10 cm were irradiated in the clockwise and the counter‐clockwise direction, respectively. This sequence was repeated five times producing ten measurements total given the two rotation directions. During irradiation, acquisitions were performed using the EPID acquisition module and the iCom acquisition module to assign a gantry angle to each image. Finally, the pixel coordinates of the center of the BB was found for each image which is by definition the central position of the panel in the ROI projection module. The results are presented in the Section To evaluate the overall accuracy of the method, an end‐to‐end test was performed using a BB phantom. First, a CT scan of the phantom was obtained with a transverse resolution of 0.5 mm × 0.5 mm and a slice thickness of 1.0 mm. It was then imported in a treatment planning system. A treatment plan was generated with two 360° clockwise arc beams of fixed field size of 24 cm × 24 cm. The monitor units were adjusted in order to have a constant angular speed of 1.5°/s for the first arc beam and 4.8°/s for the second arc beam, which correspond, respectively, to the minimum and maximum gantry speed of the VMAT plans in our clinic. The plan isocenter was positioned with an offset of 10 cm relative to the center of the BB in the lateral direction and with an offset of 5 cm in the longitudinal direction. In theory, this configuration is one of the most sensitive to gantry angle errors, mainly because of the lateral shift, as shown in Fig. 2.ETo demonstrate the usefulness of the method in clinical situations, images were acquired with the software during VMAT treatments of seven patients treated for prostate cancer. The patients already had three gold fiducial markers  implanted in the prostate for pretreatment CBCT image guidance. The distribution of the periods of visibility of the fiducial markers in the images was then generated. The planning CT images had a transverse resolution of 1.2 mm × 1.2 mm and a slice thickness of 3 mm. All the patients were treated with 6 MV dual arc VMAT treatment plans with a dose of 2 Gy per fraction. The plans were generated with the TPS Monaco version 5.11.02. The ROIs of the fiducial markers were manually drawn in the treatment planning system by the planners and a 3D isotropic expansion of 5 mm of those ROIs was performed to be used as tolerance margins for intrafraction motion. All the patients were positioned using CBCT as pretreatment image guidance. The registration of the CBCT images was done with the planning CT images by matching the fiducial markers. The default threshold setting for the histograms equalization was used in all cases. Results are presented in Section 33.AX,Y,Z) correspond to the lateral, longitudinal, and vertical axis in the reference frame of the treatment room with the linac isocenter as the origin. According to Fig. ε is the gantry angle error. The results show that the projection error increases with increasing lateral distance of the object from the isocenter. For objects located at the edge of the panel, a gantry angle error as low as 1° can cause an error of up to 2.5 mm in the projections. Figure Figure On Figs. Therefore, if the case simulated was a clockwise arc with a systematic positive gantry angle error, the projection error would always represents a contour that is ahead in time relative to the actual position of the object, even though the sign of the projection error changes. However, the same figure could also represent a contour that lags behind the actual position of the object if the arc was counter‐clockwise with a positive systematic gantry angle error.3.BFigure 3.CFigure 3.DAn example of two images acquired during VMAT treatments is presented in Fig. 4The method for intrafraction monitoring presented in this work has some advantages over the other techniques discussed in the introduction. Most importantly is that the software is freely available and the technique does not require any additional hardware not already available on Elekta linacs equipped with an EPID model XRD 1640. The software source code is available under a MIT license and instructions for installing and using the software are provided. The technique also has the advantage of not giving additional dose of radiation to patients since the treatment beam is used for imaging.For clinics interested in using the software, we expect clinical implementation to be done without major difficulty by following the instructions provided in the project repository. At the moment, the software has been tested on Synergy and Infinity Elekta linac models. The actual version of the software takes a lot of time for the calculation of projections if it is executed on a CPU, so it is highly recommended to get a GPU and use the GPU version for a routine clinical use. The software should be compatible with all treatment planning systems because only DICOM files are used. Once the pixel gain and panel position calibrations are completed, we recommend that the user carry out the commissioning test presented in section It is known that the poor contrast of MV images compared to kV images limits the capability to differentiate the different structures present in radiological images. This is especially true for limited field of view like those of collimated treatment beams where there can be a lack of anatomical landmarks to help identify which structures are present in the images. However, when used with gold fiducial markers and with proper contrast enhancement techniques, the contrast is generally good enough to discern the markers in the image as shown in Figs. There are some limitations to the method proposed in this work. The first one is the limited size of the EPID. Indeed, to avoid damaging the electronics of the detector, all treatments using beams with a field size larger than the detector are not eligible to this method. For this reason, when the projections are calculated with the ROI projection module before the treatments, each control point is verified in order to guarantee that the primary radiation field stays within the imaging area of the detector, with a small security margin, by using the leaves and jaws positions and the collimator angles contained in the DICOM RTPLAN. Another limitation is that the information is available in the beam's eye view only, which means that the position of the structures in the direction of the beam axis is not observable. The main effect of a small displacement in this direction would be a dose variation of a few percent in these structures due to a change to their distance relative to the radiation source and/or a change in their radiological depth. However, this would have much lesser clinical and dosimetric consequences than missing the target, which is why the beam's eye view geometry is preferable for 2D imaging.One more limitation of the method is that the monitoring is limited to the moments where the leaves and jaws do not hide the structures the user is trying to locate. Nevertheless, our results showed that for prostate cancer patients treated using VMAT plans in our clinic, fiducial markers position can be verified at regular intervals throughout the entire duration of treatments. When the leaves or jaws do hide a structure of interest, the superimposed contours give the operator a visual cue that this structure is supposed to be hidden at that moment. Still, this interplay effect can cause a small delay before the therapist realizes that a motion of the structure occurred. Also, even with the automatic adjustment of image contrast using histogram equalization, there are still some cases where the images are not easily interpreted. In situations where only one marker is visible, there may be uncertainty in determining the contour to which the marker belongs. However, if the position of the markers changes slowly from one image to another, successive appearances and disappearances (behind the MLC) of a marker at the same location can help to identify the marker and its associated contour. When the displacement of the markers is large between images, the correlation between the trajectory of a marker and the trajectory of a contour can be used to identify the reference contour. Nevertheless, this uncertainty does not usually last long because it is common to see more than one marker (34% of the time on average according to Table Despite the listed limitations, the proposed method contributes to improve the quality and safety of treatments. Preliminary clinical use of this method proved to be useful in our institution for VMAT treatments of prostate cancer with implanted fiducial markers. Indeed, prostate gland motion was frequently observed using this tool and in some cases the treatment beam was interrupted. Our criterion for interrupting a treatment was that at least one fiducial marker systematically touches or crosses over its tolerance contour during several successive images. Treatments were resumed after a short pause or, when necessary, by performing a CBCT and repositioning the patient. These results are not presented here since it is not in the scope of this study to characterize the movements of the prostate during treatments. We also believe that this tool can be useful when deciding which PTV margins are adequate. Because the tolerance margins for intrafraction motion are fixed by the planner and because the treatment can be manually stopped when the markers move outside those margins, the intrafraction motion contribution to the PTV margins equation can be fixed.This technique could also be applied to other treatment sites. For anatomical regions where a significant contrast exists between the target and the normal tissues, such as lung, the technique could potentially be used without fiducial markers. This would be especially useful for lung SBRT treatments as they are longer to deliver, therefore increasing the risk of the target getting out of the tolerance margins. The common practice of performing intrafraction CBCTs5We introduced a free open‐source software designed for intrafraction monitoring which uses projected ROIs contours in a beam's eye view geometry. We validated the accuracy of the technique and its clinical usefulness in a standard VMAT treatment delivery for prostate cancer patients with implanted fiducial markers. This technique has the major benefit of potentially improving patient care with very little upfront cost and no additional delivered dose.No conflicts of interest.


A helimagnet is a chiral magnet in which the direction of the magnetic moment spatially rotates in a plane perpendicular to the propagation vector. The sense of the rotation known as spin helicity is a robust degree of freedom of matter and may provide a new concept of magnetic memory if it can be electrically controlled and detected. Here we show that the helicity can be controlled by magnetic fields and electric currents in an itinerant helimagnet MnP. Second-harmonic resistivity measurements allow us to read out the controlled helicity. In contract to an insulating multiferroic magnet, in which spin rotation was shown to be controllable by an electric field, we achieve helicity manipulation by using an electric current in the conducting helimagnet. The controllability of the spin helicity may pave the way to new method of realizing magnetic memories based on the spin internal degrees of freedom. The spin helicity in helimagnets may be exploited in magnetic memory applications if electrically controllable and detectable. Here, helicity manipulation driven by an electric current and detection by second harmonic resistivity measurements in an itinerant helimagnet MnP is demonstrated. Recently, chirality has been attracting attention in magnetism and materials science, and some spiral magnetic structures show chirality. There are two types of spiral magnetic structures. One of them is a helical structure  in which the ordered direction of the magnetic moment spatially rotates in a plane perpendicular to the propagation vector  interaction12. The sign of the polarization is determined by the sense of the spin rotation, which can be controlled by an electric field. The magnitude of the ferroelectric polarization depends on the angle between the spiral plane and the propagation vector; the polarization shows a maximum when the propagation vector is parallel to the spiral plane (cycloid structure) and vanishes for the perpendicular configuration . These are related to the magnetic symmetry. In cycloid structure, the spin-rotation direction is unchanged by a mirror operation parallel to the spiral plane and by that perpendicular to the propagation vector, but it is reversed by a mirror operation perpendicular to both the former two mirrors, being consistent with polar symmetry  spin structure15. The net magnetization direction of the conical spin structure depends on the sign of the product of spin helicity and electric current  by using a focused ion-beam technique  sample previously reported by Shiomi et al.21. The resistivities are similar to each other, indicating that sample damage due to the microfabrication is minimal for the present sample. Figure 23. The ferromagnetically ordered phase (FM1) is stabilized even at zero magnetic field above 60 K, in which the magnetic moments align along the c-axis. We observed discontinuous changes in resistivity accompanying the hysteresis , there emerges a fan structure (FAN), in which the magnetic moments are within the ac plane, and the angle between the magnetic moment and the propagation vector spatially oscillates along the propagation vector  when the fan-conical transition field was traversed , observed after the poling with ±jp and ±Hp. While the magnetic structure is history-dependent in the low-field range at 51 K, the helimagnetic state is stable in the field-decreasing process  − ρ2f(−H))/2. As shown in Fig. jac, being consistent with the magnetochiral origin. Figure jp. jp = 1 × 109A/m2, suggesting that the volume fraction of controlled helicity is nearly unity.To further examine the electrical magnetochiral effect and helicity control, we extract the intrinsic odd-function component of 29. In MnSi, the magnetochiral signal shows an enhancement around the helical–paramagnetic transition temperature, but it is suppressed by applying a large magnetic field28. The origin of this was ascribed to the chiral magnetic fluctuation in the vicinity of the phase boundary. Enhancement of the magnetochiral signal around the phase boundary was also observed in CrNb3S6, while some components with different origins seem to coexist29. In this case, the magnetochiral signal increases more sharply as the ferromagnetic phase transition is approached from a lower temperature. Then, it rapidly decreases above 52 K owing to the reduction of helical volume fraction, because the inversion symmetry is unbroken in the ferromagnetic state. It should be noted that the magnetic field dependence of a-axis16. Similar sharp enhancement at a low magnetic field unrelated to the magnetization is observed in the paramagnetic state of MnSi28, which may be a characteristic of the chiral fluctuation-induced magnetochiral signal. Importantly, the strong chiral fluctuation in the vicinity of the phase boundary should make the magnetic structure electrically susceptible, which should more readily enable controllability of the spin helicity.In Fig. 2. Since control of a chiral object with a magnetic field and an electric or thermal current was not demonstrated previously, the applicability to a wide range of magnetic and chemical chiral objects should be examined. While antiferromagnetic spintronics has been attracting much attention recently, the present result opens up the possibility of magnetic memory based on helical magnets because the helicity degree of freedom has now been shown to be controllable. Nevertheless, there are still a few issues to be resolved in future to realize such application. One of them is instantaneous helicity switching. While we achieved helicity control by using the poling procedure just below the transition temperature for the present microfabricated single crystal, practical devices need instantaneous helicity switching without any adjustment of magnetic field and temperature. In principle, an electric current with high enough density should be able to control the helicity instantaneously. Such a high-density electric current should be realized in the form of a thin-film sample. In addition, the voltage signal for a helicity probe becomes larger and more easily detectable for a thin-film sample. There is one report of thin-film fabrication of spiral magnets with a transition temperature above room temperature30. Another issue is the use of high magnetic fields. In this work, we applied a magnetic field to control and to detect helicity, but it cannot be used in practical electronic devices. Instead of applying a magnetic field, a spin-polarized current may be used for control and detection of the helicity. In fact, some theories suggest coupling between the helicity and spin current32. A junction with a spin-polarized ferromagnet or spin Hall material  seems useful for zero-field detection of the spin helicity.We have demonstrated control of the helicity  in a helical magnet MnP by using an electric current and a magnetic field, which satisfies the symmetrical rule that the external stimulus for chirality control should be collinear polar and axial vectorial fields with the same time-reversal symmetry (both odd or both even)−1) at a rate of 0.5 mm h−1 for 21 days.We grew a single crystal of MnP, utilizing a Bridgman method. A silica tube with Mn and P was placed in a Bridgman furnace and slowly heated to 1200 °C within 14 days and kept at that temperature for 4 days. After that, the tube was moved toward the low-temperature region in a temperature gradient  technique (see Supplementary Fig. We extracted a microscale thin rectangular piece with an approximate size of 10 × 20 × 1We measured the first and second-harmonic ac resistivities with an electric current frequency of 14.3 Hz in a superconducting magnet.Supplementary InformationPeer Review File


Following publication of this article , concernThe authors noted that incorrect images are reported in PLOS ONE’s Data Availability policy that was in place at the time of the article’s submission.In addition, the Data Availability statement for this article reads “All relevant data are within the paper.” However, the primary data are not included with the article and the authors noted that except for the data in PLOS ONE Editors issue this Expression of Concern.In light of the above concerns, the The Data Availability statement is updated to: Image data for Figs S1 File(ZIP)Click here for additional data file.S2 File(ZIP)Click here for additional data file.