Most dynamic systems are controlled by discrete time controllers. One of the main challenges faced during the design of a digital control law is the selection of the appropriate sampling time. A small sampling time will increase the accuracy of the controlled output at the expense of heavy computations. In contrast, a large sampling time will decrease the computational power needed to update the control law at the expense of a smaller stability region. In addition, once the setpoint is reached, the controlled input is still updated, making the overall controlled system not energetically efficient. To be more efficient, one can update the control law based on a significant fixed change of the controlled signal (send-on-delta or event-based controller). Like for time-based discretization, the amplitude of the significant change must be chosen carefully to avoid oscillations around the setpoint  or an unnecessary increase of the samples number needed to reach the setpoint with a given accuracy. This paper proposes a novel non-linear event-based discretization method based on inter-events duration. We demonstrate that our new method reaches an arbitrary accuracy independently of the setpoint amplitude without increasing the network data transmission bandwidth. The method decreases the overall number of samples needed to estimate the states of a dynamical system and the update rate of an actuator, making it more energetically efficient. With ever faster and ever cheaper digital computers, the control of dynamic systems has shifted from analog to digital controllers. Critically, the majority of discrete-time control laws assume that the sampling rate of the discretization process is constant. There is currently a discrete-time equivalent for the vast majority of continuous-time control theory principles, from the continuous proportional-integral mechanical “governors” of Maxwell  to the mThe selection of the appropriate sampling rate depends both on the open-loop system dynamics and on the desired dynamics of the controlled system. A system with fast dynamics needs a high sampling rate to ensure the stability of the controlled system at the expense of higher computational power. Moreover, the controlled input of a dynamical system is traditionally updated at each time step independently of the error amplitude. When a controlled system is in a stable configuration at the setpoint, there is obviously no need to sample the data, update the controller and the actuator. Indeed, doing so is not energetically efficient.In the early 60's, this lack of efficiency combined with lower computational power spurred the development of adaptive discrete-time sampling methods  because the data transfer must have the same representation as the system data. Therefore, if one wants to control a system with a high accuracy on a wide range of value, such a control system would need a high bandwidth network to transmit the values of the signal.In this paper, we propose a new framework and theory for event-based control that are circumventing the problem mentioned above while preserving the benefit of the event-based approach. We formalize a generalized discretization method that stems from the principle of neuromorphic event-based cameras . Using linear event-based discretization, if the setpoint lies between two intervals then no event will be generated (no event will be generated after t4 as the system is stable between two samples). To overcome this problem, the simplest method is to compute the step size such that the setpoint is an integer multiple of the step size. It is worth noting that when the setpoint lies between two samples, it corresponds to a bias β in the linear function presented in S to control, we assume that a control law C has been designed to ensure that the output y of the system converge toward a setpoint r, stabilizing  the controlled system. Both signals r and y can be multidimensional. Using this representation, the error signal ϵ is equal to the difference between the system output y and the setpoint r. + computed from the control law C, can be defined as the basin of attraction around the equilibrium ϵ = 0 such that:Given a generic finite dimension, non-linear continuous-time system h be the “event generator” continuous function that computes the time of the next event ti+1 from the value of ϵ at ti.Let h must tend to zero when the error tends to zero. Mathematically, this can be written has a limit:The key principle of our event-based discretization method is that the duration between two events generated by Equation (4) can be equivalently rewritten into:h if (4) is true.Equation (6) is the first requirement that must be met by ti} using a monotonically increasing function g applied to ϵ(t) such that an event will be triggered when the difference of g between two events is equal to one:As the class of functions that satisfy (4) is very broad, this section presents a method to build these functions. One can compute a time series {g is monotonically increasing. However, it must be noted that if g is monotonically decreasing, the same reasoning can be applied and the time series becomes:For the rest of the paper, we postulate that g−1 exists , one can predict the value of the error at which the next event will be triggered:As This discretization process is presented in −1 exists by definition of Equation (9). Therefore, one can write the function h defined in (4) as:In addition, ϵg can be extracted from (6):From this relationship, a supplementary condition on g is a function of reals, only ±∞ is a solution of (12).As Using the proposed discretization, one can reconstruct the time course of the original error signal using the events:H(x) represents the Heaviside step function:in which As the series (13) converges toward 0, then:t0)) and on the time interval estimation (ti − ti − 1). In this paragraph, the effect of uncertainties on these values is analyzed.Up to now, our formulation of the events generation assumes that there is no measurement uncertainties on both the initial value of the error (ϵ(t0), influences the estimate of ϵ(tN) and thus affects the convergence of the series. We will postulate that we have an infinitely accurate measurement of the time interval between two events.First, we will analyze how an uncertainty on the initial estimate of the error, ϵ representing the true initial value, δ0ϵ representing an uncertainty on the true initial value, one can rewrite (13):with ϵ(The first terms of recursion (17) are:g is monotonically increasing, it follows that i − 1. Then for i, Because andThis result shows that, independently of the initial error on the measurement, the estimate of the error converges toward zero.i on the time value ti is drawn from a uniform random distribution In this section, we will analyze the effect of an uncertainty on the measurement of the time interval between two events. If one assumes that the uncertainty σϵ(t), using the rectangular function Π:Using (13) and (22), one can extract δFrom (24), one can evaluate the bounds of the error if one supposes that all the uncertainties are equal to either ς or −ς:t0) = ϵ0, the limit of (25) representing the upper (lower) bound of δϵ(t) can be computed using (15):If ϵ(0. Equation (26) shows that the boundaries of the uncertainty on the error signal is only a function of the clock accuracy. Therefore, the proposed non-linear discretization can be used to reach an arbitrary precision of the controlled state, provided that the user has access to an infinitely accurate clock.Similarly, the other bound is equal to ςϵAs the goal of the paper is to demonstrate the usefulness of the discretization method to control a system, we postulate in the following sections of the paper that the user has designed a control law such that the controlled system is globally asymptotically stable.g(ϵ). We demonstrated in section 2.1 that a candidate discretization function g must fulfill three conditions:In the rest of the paper, we use a logarithmic function as event discretization function g : ℝ+,* → ℝx ↦ g(x)          g must be continuous and strictly monotonic on ℝ+, * (and thus invertible)b of ϵ(t) for g such that:Selecting a logarithmic function with a base g on ℝ+. The effect of the base of the logarithm on events generation will be presented in section 3 Using (27), one can compute a time series {ti}:it is straightforward to demonstrate that the logarithm satisfies the three conditions on pi, is added to the time series (28) to express if ϵ(t) has increased or decreased between two samples of the series:A polarity information, b > 1 in the rest of the paper. But the same relationships used to generate an events series for b > 1 can be derived if 0 < b < 1:For the sake of simplicity, we will use t) can change sign between two events, we added a third piece of information to each event representing a change of sign between the current event and the prior one: si.Finally, as ϵ in which ti represents the event time, pi represents its polarity  and a Boolean si (0/1) representing the fact that the sign of the signal changed between the ei−1 and ei.Therefore, an event t) usingUsing these notations, one can construct the time course of the original signal ϵ(H(x) represents the Heaviside step function:in which ti can be written using a geometric recursion:The goal of this section is to demonstrate that the proposed logarithmic discretization can be used to reach a stable point with an arbitrary accuracy. Using (29), the value of the error signal at time t0):From (35), it is clear that from any finite value ϵ(Equation (36) shows that one can reach an arbitrary precision using the logarithmic discretization proposed in this section. The arbitrary precision at which the error is considered null (therefore at which the system has reached the setpoint) is a design parameter of the control law.rn:Theoretically when ϵ reaches zero, the number of events goes to infinity and the duration between two events reaches zero. In addition, even if the accuracy of the clock is very high, practically the computations needed to evaluate the ratios (29) and (30) can take some time. Therefore, for practical implementations, we define the refractory period as the minimum time between two events that the system can generate. These limitations put some constraints on the overall system as the ratio between the past error and the current one could be crossed multiple times during the refractory period. To counter this issue, we added a last parameter to the event representing the number of times the threshold has been crossed between two events rn:Equation (33) can be written to include The presented discretization method allows us to benefit from the event-based representation in the control context. In the experiments section, we present its application to the control of a second order dynamical system.The proposed event-based non-linear discretization and the resulting control laws are tested on two systems different systems. The first one is a classical second order dynamical system while in the second example we stabilize an inverted pendulum on a cart.This section presents the control of a second order dynamical system with 0.1 and 0.01 s as time constants. The second-order system is controlled by a proportional-integral-derivative controller  without changing either the discretization or the PID parameters. As the absolute error at the end of the simulation remained mostly constant with increasing amplitude, the relative error is decreasing.T) is applied to the cart, the pendulum is moved out of its equilibrium position. An event-based proportional-integral-derivative (PID) controller stabilizes the pendulum in an inverted position through a control force F applied to the cart. The model of this dynamical system is presented in the Appendix in We have implemented a numerical simulation of an inverted pendulum put on a cart. When a perturbation  controller with a proportional gain of 100, an integration gain of 2 and a derivative gain of 10. We compared a traditional discrete PID controller with an event-based version of the PID controller. The base of the logarithmic discretization function was set to 1.05 for the event-based control law simulations and the refractory period of the sensor was set to 1 ms.This simulation shows that the frequency of events generation is sensitive to the amplitude of the error  and to the amplitude of the time derivative of the error . In In this set of simulations, we showed that the system could reject a perturbation applied to the cart and keep the pendulum stable while decreasing the number of times the controlled input is updated by a factor 12 compared to an equivalent time-discrete PID with a 1 ms sample time. We also showed that we can keep the pendulum stable with the same controller parameters while updating the force applied to the cart every 50 ms instead of each time an event is received, reducing drastically the burden put on the actuator.K of the state-space feedback law was computed using the linear-quadratic regulatorAfter the controllers reject the perturbation and stabilize the pendulum angle, the third row in In this final set of simulations, we stabilized both the inverted pendulum angle and the cart position. The designed system rejected the perturbation and stabilized both the pendulum angle and the cart position. As for the PID simulations, the force applied to cart was updated every 50 ms and we showed that we could keep the system stable.This work proposes a new non-linear event-based discretization method and the associated proportional-integral-derivative and state-space feedback control laws. The key novelty of the new event generator function lies in the principle that the time between two events must decrease when the discretized signal  tends to zero. Contrary to current event-based discretization/control schemes  can be much higher than the number of bits used to transmit an event . As a result, the design of a control scheme based on the new method of this paper can increase the controlled signal dynamic range without increasing the network data bandwidth needed to transmit the information between the different components of the controlled system.Our results demonstrate that the new method combines the advantages of analog continuous time systemsit can reach an arbitrary precision with a very high dynamic rangewith the advantages of event-based controlno events are generated when the error is null.In addition, the amount of data transmitted between active parts of the control system can be smaller than the memory needed to store the signals value. This makes the overall system more efficient energetically.All datasets generated for this study are included in the manuscript and the PD, S-HI, and RB conceived of the presented idea. PD developed the theory and performed the computations. All authors discussed the results and contributed to the final manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Ferritin and apoferritin nanoparticles adsorbed on a mica substrate exhibited a change in their size by varying the pH value of the surrounding medium. Lowering the pH value of the solution led to a decrease in size of the nanoparticles whereas a successive increase of the pH value increased the particle size again. The pH dependent change in size could be related to the dis- and reassembling of the protein shell of ferritin and apoferritin. Supplementary imaging by bimodal magnetic force microscopy of ferritin molecules accomplished in air revealed a polygonal shape of the core and a three-fold symmetry of the protein shell providing valuable information about the substructure of the nanoparticles.Ferritin, a protein that is present in the human body for a controlled iron storage and release, consists of a ferrihydrite core and a protein shell. Apoferritin, the empty shell of ferritin, can be modified to carry tailored properties exploitable for targeted and direct drug delivery. This protein shell has the ability to dis- and reassemble depending on the pH value of the liquid environment and can thus be filled with the desired substance. Here we observed the dis- and reassembly process of the protein shell of ferritin and apoferritin  Depending on the environment and the ferritin subtype, ferritin features a high thermal stability up to 77 °C4. As other proteins5, apoferritin, the empty protein shell of ferritin, can be used to transport various substances, such as drugs for medical applications8. The empty shell of ferritin can be used as a platform for the synthesis of nanoscale materials9. This protein shell is known to dis- and reassemble depending on the pH value of the surrounding medium10: the protein shell is stable in acidic environments down to pH values of 3.4, whereas the core is stable down to pH 2.111. Below pH 3, the protein shell starts to disassemble. However, this process is reversible by the subsequent increase to higher pH values turning apoferritin to an almost ideal candidate as carrier for drug delivery, i.e. loading with a substance and subsequently release it. This loading occurs in a random fashion: the substance to be enclosed inside the protein shell is step-wise surrounded by the reassembling shell. It is possible to adsorb ferritin onto a substrate12 to study dis- and reassembling of the protein shell using sedimentation-velocity measurements10 and small angle x-ray scattering13. However, these studies do not provide a direct observation of the dis- and reassembling process and hence direct evidence, instead the conclusions had to be drawn based on subsequent transformation of the data. In particular for scattering methods, the observed data reflects the protein state in the reciprocal space and must be subsequently transferred to real space by means of simulations based on specific assumptions. Here, we report on the direct in situ observation of the pH dependent dis- and reassembly process of ferritin and apoferritin nanoparticles using atomic force microscopy (AFM) based on images that provide the size of individual particles in real space in liquid solutions of various pH values. Additionally, we examine the electrosteric interactions between the tip of an AFM cantilever, the ferritin nanoparticle and the substrate at different pH-values to validate the size interpretation of the molecules. Exploiting the coexistence of mechanical tip-shell forces and the magnetic tip-core interactions with bimodal magnetic force microscopy16 allowed us to resolve substructures of the molecules, such as the three-fold symmetry of the protein shell and a polygonal shape of the core indicating its crystalline condensed state.Ferritin is an iron-storage core-shell-protein present in almost all living organisms. Ferritin has an outer diameter of approximately 12 nm containing iron in the form of ferrihydrite in its approximately 8 nm large core. In the body of animals, ferritin is utilized to store, transport and release iron in a controlled manner for the oxygen transport, indispensable to life, and can be exploited for different medical17, muscovite mica18 and apoferritin19 depending on the pH-value of the medium is depicted. The resulting charge interactions between the AFM tip, nanoparticle and SiO2 or mica for solutions in different pH-values are summarized in Fig. The isoelectric point is the pH value of a liquid solution, at which the surface of an immersed material or molecule does not carry an electrical net charge. For proteins, this is also the pH value for best solubility. In Fig. z-piezo to move downward while scanning across an (apo)ferritin molecule whereas the actual size of the molecule forces the z-piezo to move up simply because of the topographical feedback that keeps the applied setpoint force constant. Thus, it is reasonable to believe that the height and hence the size of the ferritin nanoparticles at pH values in the range of 2.5 < pH < 4.8 is slightly underestimated by the apparent topography. If we detect a decrease in particle size in this range while lowering the pH value, then this decrease is an upper limit and might be even stronger in reality, compared to height changing values measured at pH < 2.5 or > 4.8 caused by the variation of the electrostatic interactions. This effect has to be taken into account when evaluating the development of the size of nanoparticles when changing the pH value of the solution.As can be deduced from Fig. Figure In Figure To study the pH dependent size behaviour of apoferritin nanoparticles, topographical analysis by AFM in solutions of different pH values was performed in the same manner as for ferritin described in the previous section. The Apoferritin nanoparticles show a higher density on the substrate and a higher initial height deviation, than compared to the ferritin samples. The pH value of the solution was adjusted in the following order: pH 6, pH 2, pH 3, pH 4 und pH 6. Figure A general change in size of the apoferritin nanoparticles by reducing the pH value is clearly apparent when comparing Fig. As for ferritin nanoparticles, it can be deduced that with decreasing pH value, the average height of the nanoparticles decreases, from approximately 11 nm at the first step of the measurement (pH 6) to approximately 8 nm in the second step (pH 2). The standard deviations of the size distribution displayed in Fig. 20. However, the core and shell substructure of ferritin molecules could not be resolved by this technique. In the following we show that by using bimodal magnetic force microscopy16, we can separate magnetic from mechanical cantilever responses of the mixed tip-sample interactions. This allowed us to image substructure features of the shell and the core of a single ferritin nanoparticle in great detail.As demonstrated in the previous section, the pH dependent dis- and reassembling of ferritin and apoferritin particles can be directly visualized using atomic force microscopy. First high resolution studies on the protein shell of ferritin were performed by Ohnishi using contact mode AFMi.e. first (a) and second (b) eigenmode amplitude images as well as the first (c) and second (d) eigenmode phase images and a three-dimensional representation of the topography rendered by the colour code of the first and second eigenmode phase 21, which presents itself in our study as a (common) polygonal geometric structure. The analysis of the symmetry visible in the second eigenmode amplitude image was conducted in Fig. 20. By the choice of the experimental parameters  it is determined whether the first or the second eigenmode channels are mainly dominated by magnetic or mechanical forces. Hence, the polygonal structure visible in Fig. 23. A threefold symmetry of the protein shell exposed to the topside, i.e. opposite to the substrate, is clearly visible in the image taken by MFM  amplifying the mechanical contrast between the tip and the ferritin shell. For the measurement parameters used in this study such as free amplitude of the first eigenmode (A1 = 23 nm) and relative setpoint (ASP = 16 nm) as well as the relatively large free amplitude of the second eigenmode (A2 = 16 nm) the observables of the first eigenmode are more prone for magnetic forces and the observables of the second mode are dominated by mechanical forces, which is in contrast to previous studies20 where substantially different parameters were applied, in particular a small second eigenmode amplitude. Thus, the first eigenmode ferritin particles formed most likely developing from surrounding protein fragments, which remained in the liquid from previously disassembled protein shells. Due to the lack of a constant liquid flow in the electrochemistry cell, there is no reason to assume that the dissembled protein shell fragments were flushed away from the vicinity of the sample surface. Several authors26 assume that the higher-order structure of the protein shell is disassembled upon pH value decrease of the environment of the particles. However, the individual protein chains should remain intact. In the current study, conclusions about the structure of individual proteins cannot be drawn. Using atomic force microscopic measurements and further improving the resolution, it might pave the way to visualise the behaviour of individual proteins during the formation and degradation of the protein shell.In conclusion, the pH dependent dis- and reassembly process of ferritin and apoferritin proteins can be directly visualized and studied in detail by an 22 and the appearance demonstrates that the protein capsule was mainly intact.The bimodal excitation of a magnetic cantilever allowed us to simultaneously map the shape of a ferritin molecule, its mechanical as well as magnetic properties. In our experiments, these two properties could be clearly separated in the amplitude and phase channels of the different eigenmodes. The results suggest that the contrast found in the phase image of the first eigenmode mainly results from the magnetic interactions between the ferrihydrite core of the ferritin particle and the tip providing a defined polygonal structure as a result of the polycrystalline condensed state. Additionally, the contrast in the amplitude image deduced from the second eigenmode revealed a three-fold symmetry in the center of the ferritin nanoparticle arising from the mechanical interaction of the tip with the shell. This symmetry is in very good agreement with the calculated ferritin structure27 has been a great challenge so far due to the low-Q environment and unstable magnetic coatings, in particular on tiny magnetic structures such as present on nanoparticles.In future, bimodal magnetic force microscopy in combination with a liquid environment of changeable pH value would allow for the observation of the dis- and reassembling process in great structural detail due to the simultaneous interplay of magnetic and mechanical forces between tip and sample. The methodology could open up new pathways to study drug release for targeted drug delivery in cancer research. However, magnetic force microscopy performed in liquids2 large piece of a Si(100) wafer . Size measurements with AFM were initiated 5 minutes after the samples preparation to provide sufficient time for the nanoparticles to settle on the substrate. For the apoferritin sample preparation, 10 µL of the apoferritin stock solution , purchased from Merck KGaA as well, were drop casted onto the substrates (mica or Si(100)) and 200 µl deionized water were dropped above for further dilution. Size measurements by AFM were initiated after 5 minutes settling time. To observe the dis- and reassembly process of the molecules, particular attention was paid to avoid exsiccation of the sample after preparation to keep the tertiary structure of the protein shell intact.To prepare ferritin samples, a two-step dilution of the stock solution was conducted. 10 µL of the purchased ferritin solution (10 mg/mL in 0.15 M NaCl) cationized from horse spleen  were diluted into 1 mL deionized water  and homogenized for 5 min at 800 rpm at 22 °C . 10 µL of the prepared solution were again diluted into 1 mL deionized water and homogenized repeating the procedure. 200 µL of the final solution were drop casted onto a freshly cleaved muscovite mica  substrate or a 1 × 1 cm28 could be observed with the low force we adjusted. A structural change of e.g. the protein shell that would alter the observed height by changing the pH value seems unlikely to be the major origin of the size increase/decrease because of the relatively large size change we observe. The cantilevers used were SNL-A , exhibiting a typical force constant of 0.35 N/m as determined by the thermal noise method29 and a nominal resonance frequency of 65 kHz in air.For the size analysis of the nanoparticles, PeakForce Tapping AFM measurements were carried out in a closed, heated electrochemistry cell  mounted in an ICON AFM from the same manufacturer. Topographical images were taken using an oscillation amplitude of 300 nm at a driving frequency of 500 Hz, far less than resonance frequency of the cantilever in the solution to avoid resonance effects. The maximum force exerted to the sample was adjusted to 500 pN. The force applied is rather high and it can affect the analysis based on height measurements. Since the applied force was kept constant for all images such an effect applies to the whole imaging series and there is no change for the outcome of the experiment. Based on our observations we exclude particle migration or particle movement during the pH change because no traces that are typical for particle movement30. Height profiles of individual particles can be found in the supplementary material  or ammonia  to deionized water before filling the solution into a syringe. No additional ferritin or apoferritin nanoparticles are in the syringe-solutions. Using pre-heated (37 °C) syringes filled with the measuring solution of different pH values and a syringe pump Aladdin-1000 , the pH value inside the electrochemistry cell was changed after each measuring step through connection tubes attached to the inlet of the cell. The pH stability of the solution in the cell during the measurement was confirmed at the outlet of the cell. The samples were kept in a liquid environment at all times. Due to the low number of particles, the size of the ferritin could be measured by drawing a cross-sectional profile through the particle and extracting the average height over a 3-pixel line with respect to the substrate level. The size of the apoferritin nanoparticles was analysed by an automated particle analysis toolal Figs.  and S3 aal Figs. .31 was combined with magnetic force microscopy in a single-pass technique. Using a magnetic tip, it is feasible to separate long-range magnetic from short-range mechanical tip-sample interactions in air19 and in liquid environments20. To this end, we measured a ferritin sample in air using a Cypher S  AFM with a magnetic Multi75-G cantilever  exhibiting a nominal force constant of 3 N/m and a nominal resonance frequency of 75 kHz of the fundamental flexural eigenmode in air. First and second cantilever eigenmodes were simultaneously excited by a photothermal excitation system and the respective amplitude and phase shifts of both modes were mapped. We used a scan speed of 1 Hz, free oscillation amplitudes of A1 = 23 nm and A2 = 16 nm and a setpoint amplitude ratio of Asp/A1 = 16 nm/23 nm = 70% to obtain images.To study the substructure of ferritin nanoparticles bimodal force microscopySupplementary dataset


Machine learning is rapidly gaining importance in radiology. It allows for the exploitation of patterns in imaging data and in patient records for a more accurate and precise quantification, diagnosis, and prognosis. Here, we outline the basics of machine learning relevant for radiology, and review the current state of the art, the limitations, and the challenges faced as these techniques become an important building block of precision medicine. Furthermore, we discuss the roles machine learning can play in clinical routine and research and predict how it might change the field of radiology. Machine learning is a rapidly evolving research field attracting increasing attention in the medical imaging community. Machine learning in radiology aims at training computers to recognize patterns in medical images and to support diagnosis by linking these patterns to clinical parameters such as treatment or outcome. These methods enable the quantification of disease extent and the prediction of disease course with higher precision than is possible with the human eye.Two recent advances have further accelerated the development of machine learning in radiology. First, the acquisition volume of medical imaging data is accelerating. Worldwide, during 2000–2007, an estimated 3.6 billion radiologic, dental radiographic, and nuclear medicine examinations were performed per year . MedicalHow can machine learning serve as a tool to perform automated quantitative measurements in radiological routine?Can machine learning contribute to research by expanding the vocabulary of patterns we can exploit for diagnosis and prognosis?Can we effectively expand the evidence on which machine learning relies from controlled studies to large-scale routine imaging data?What are the roles of machine learning in current radiology?Here, we review the current state of the art and the possible roles machine learning can play in medical imaging covering clinical routine and research. After summarizing the basics of machine learning, we discuss the most pressing challenges and structure the review around four core questions:These questions are connected to a number of challenges, tackled by research at the interface of machine learning and radiology. They range from the availability of data and expert annotations, to the exploitation of partially unstructured data acquired during routine for learning, and the coping with noise in both data, as well as annotations.In conventional analysis software, computers are programmed to carry out instructions specified by the developer. While machine learning consists of algorithms implemented in standard frameworks, its approach toward data analysis is different. Machine learning algorithms learn from examples and replicate the observed behavior—typically a mapping from an input to an output—on new data. The basic principles of such models are illustrated in Fig. For instance, training data can consist of example magnetic resonance imaging (MRI) volumes depicting segments of the breast. For each example, there is a label indicating whether there is a lesion or not in the volume. The algorithm learns a mapping from the input (volume) to the output (label).supervised learning since a large number of expert annotations in the form of correct labels are necessary. The resulting model is called a classifier since it classifies the input into a discrete set of possible classes . After training, the classifier can process new input data and produce an output in the same categories as it has learned during the training phase. The first main component of these models is feature extraction or the mapping of raw input data such as lesion location or delineation to a feature representation, typically in the form of a vector. Features capture relevant characteristics of the raw input data. Constructing informative features for specific classification tasks was long the focus of research and has only recently shifted to being solved by algorithms. The second component is the mapping model, such as a classifier in the case of categorical output , or regression models (time to recurrence) in the case of continuous output.This paradigm is called unsupervised learning does not rely on expert labels for the training examples. Instead, it processes a large number of unlabeled examples and seeks to find structure in the data. It can be in the form of clusters, groups of examples that are similar and clearly separable from other groups. This cluster model can then be used to assign cluster memberships to new data. Bone texture patterns that can be identified repeatedly across a population are such an example [manifolds that reflect the fact that often different variables do not occur in arbitrary combinations; for instance, age, weight, and height in children.By contrast,  example . A diffesupport vectors.A range of algorithms for classification and regression have been investigated over time, all of them sharing the aim of learning a mapping from a complex input space to a label or scalar variable. Nearest neighbor approaches predict labels based on the distance of the feature representation to those representations with a known label. Support vector machines  have proensembles of classifiers: bagging and boosting. Both use a large number of relatively simple models, so-called weak learners trained on parts of the data, and aggregate their estimates on new data, by mechanisms such as voting. During training, boosting builds a cascade of weak learners, each trained on data that previous learners were performing poorly on. An example is AdaBoost [The insight that large heterogeneous training data are often beyond the capacity of single classifiers led to two approaches relying on AdaBoost . BaggingAdaBoost . The latThe critical contribution of random forests was the ability of the algorithm to work with very large numbers of features, even if a portion of them is not informative, and to identify those that carry information during training. Algorithms that identify or construct relevant features based on examples were shown to outperform approaches that rely on the careful design of descriptors capturing relevant image information.Most recently, deep learning has emerged as a powerful approach in machine learning. It uses neural networks with large numbers of layers to perform both feature construction and predictor learning simultaneously . ConvoluThe automation of repetitive tasks, the enabling of radiomics, and the evaluation of complex patterns in imaging data not interpretable with the naked eye.The discovery of new marker patterns and disease signatures in combined imaging and clinical record data, and the linking of these signatures to disease course and prediction of treatment response.The computational analysis of imaging data using machine learning is about to fundamentally change our ability to understand disease, risk, and treatment. It will likely serve as the basis for the development and implementation of novel treatment strategies as well as the individualized, effective, and early delivery of care and preventive medicine. Linking research in machine learning and medicine will act as a powerful driver in this endeavor. Interdisciplinary collaboration is the key to creating better models to predict disease course and risk in individual patients, to forecast the response to a growing landscape of possible treatments, to deliver early prevention, and to make possible the optimal care for each individual. We will discuss two important roles of machine learning in radiology:Machine learning can automate repetitive tasks, such as lesion detection, and the quantification of patterns such as textures that are hard to discern reliably with the human eye.One example of a tedious task is the detection, counting, and measurement of lesions in large-field-of-view imaging data before and after chemotherapy to evaluate response. Accuracy has a high impact on the prognosis and further treatment of the patient. Nevertheless, the task itself does not require a high level of experience but rather a high level of concentration. Additionally, finding a lesion and classifying it as responsive or not responsive to treatment is not sufficient, as a quantitative measurement may provide a more accurate description of the patient’s condition. There are already several software packages available for clinical implementation . Using tRadiomics, or the high-throughput extraction and analysis (or mining) of radiological imaging data [ing data , is a sting data . Howevering data  and osteing data .In addition to quantifying known patterns, machine learning enables the discovery of new patterns, which can serve as candidate markers for diagnosis, outcome prediction, or risk assessment.The diagnosis of disease patterns in radiology relies on the identification of imaging patterns that have proven to be of diagnostic and/or prognostic relevance. One prerequisite of such an approach is that the patterns must be defined precisely enough for trained radiologists to be able to recognize them with low interobserver variability. The reliable recognition of disease patterns in radiology is a challenging task and requires the systematic analysis of images. A good example of this is the diagnosis of the usual interstitial pneumonia (UIP) pattern, which is one of the most important patterns in diffuse parenchymal lung diseases. The diagnosis of a UIP pattern relies on the presence of reticular abnormalities and honeycombing in a basal and peripheral predominance . In addiMachine learning enables us to utilize imaging information that is not recognizable for the human eye and thus new disease patterns and predictive markers can be discovered. In lung imaging, machine learning is a promising approach to support the CT diagnosis of a UIP pattern and other patterns of diffuse parenchymal lung diseases and to predict the course of these diseases . The potBone diseases are one example, where bone density is linked to outcome such as fracture risk but does not capture the rich variety of trabecular architecture linked to different diseases. Unsupervised learning is able to identify patterns that can be extracted repeatedly across individuals and even scanners . SimilarThe amount of available medical data has been growing exponentially over the past few decades, with approximately 118 million CT or MRI examinations per year in the EU . We routClinical routine data are a particularly valuable source of real-life patient histories and connected imaging data capturing disease course and treatment response across a wide variety of individual disease paths. Using routine data is challenging, since they are not acquired in a systematic fashion as during clinical trials, but instead with the sole purpose of individual patient care. Furthermore, parts of these data are unstructured, rendering their use for modeling challenging. Currently, only a fraction of routine imaging information is exploited for research and the development of models. While there has been a tremendous advance in image acquisition technology during the past five decades, the interpretation of imaging data has hardly changed. Radiologists focus their reports and summarize individual findings based on specific questions provided by referring clinicians . The information content of imaging modalities such as three-dimensional volumes with submillimeter slice thickness by far exceeds the capacities of visual assessment by humans. The complex multivariate relationship between clinical information  and imaging information is not sufficiently reflected in current studies. Large parts of multivariate information that is critical for personalized medicine are discarded, restricting evidence to small study populations rather than exploiting real clinical populations that are currently largely underexploited. There are two reasons for this: on the one hand the lack of technology to use heterogeneous and partly unstructured routine data for machine learning, and on the other hand the stumbling blocks of data management. Machine learning on this scale makes the integration of partially unstructured clinical information from patient records and imaging data necessary [ecessary , 14, butThe increasing use of computer-assisted diagnosis software in radiology raises fear that such software might eventually take over the job of radiologists. There is no doubt that 5, 10, and 20 years from now, the job of radiologists will be very different. In a rapidly evolving specialty such as in radiology, this is to be expected. In fact, 20 years ago, the task of radiologists was different from what it is today and rapid technical developments are a hallmark of this field. Yet, the prime impact of machine learning will not be in replacing, but rather in enabling radiologists to use more powerful prediction models, and to rely on more comprehensive patterns that are informative for the individual disease course and treatment outcome. The radiologist’s role will be to develop such diagnostic paradigms, and to integrate these paradigms with overall patient care.The primary role of machine learning will not be limited to the automation of specific analysis tasks, but will be critical in improving individual care by discovering new marker patterns and rendering these patterns useful. We will have powerful computational models that will enable a more accurate prediction of individual disease course, and which will forecast a response to treatment from observations such as imaging data and disease history. These models will facilitate the selection of personalized treatment strategies more effectively based on an assessment of each patient in light of an unprecedented scale of evidence from thousands of disease and treatment histories, together with models that translate these observations into an accurate prognosis.As we become able to detect and quantify complex but subtle patterns in observations at an earlier stage, diagnostic categories might change. Through an understanding of the link between distributed signatures and prognosis, we will discover novel disease and response phenotypes, which, in turn, will contribute to research for novel treatments.Until then, a number of hurdles must be overcome. Research communities that advance machine learning and medical disciplines such as radiology must collaborate more closely, learning to trigger algorithmic methodology by clinical questions and to ask new clinical questions whose answers will be made possible by computational algorithms. We will need to find methods to deal with heterogeneous multimodal data and the means for the efficient and possibly automated curation of such data. Algorithms will have to identify prognostic feature patterns in complex imaging data, and robust models will need to be able to predict, simulate, and assess the certainty of their output at the same time. While these challenges are significant, an increasingly joint effort by researchers across varying fields makes the solution to such challenges plausible, and the future of medical imaging might change more rapidly than we expect today.Machine learning facilitates the exploitation of patterns in imaging data and patient records for more accurate and precise quantification, diagnosis, and prognosis.The computational analysis of imaging data using machine learning will change our ability to understand disease, risk, and treatment.Machine learning allows for the automation of repetitive tasks, the enabling of radiomics, and the evaluation of complex patterns in imaging data not interpretable with the naked eye. It leads to the discovery of new marker patterns and disease signatures in imaging and clinical record data, and the linking of these signatures to disease course and prediction of treatment response.Radiologists will be able to use more powerful prediction models and to rely on more comprehensive patterns providing information on the individual disease course and treatment outcome.


A theoretical approach to the consistent full quantum description of the ultrafast population transfer and magnetization reversal in superconducting meta-atoms induced by picosecond unipolar pulses of a magnetic field is developed. A promising scheme based on the regime of stimulated Raman Λ-type transitions between qubit states via upper-lying levels is suggested in order to provide ultrafast quantum operations on the picosecond time scale. The experimental realization of a circuit-on-chip for the discussed ultrafast control is presented. One of the main problems of modern quantum physics is the transfer and storage of quantum information. This is why the development of quantum logic protocols using the fast state control of promising qubits and registers is of great importance. Superconducting artificial atoms based on Josephson junction circuits underlie a number of developments in algorithmic and adiabatic quantum computers, artificial metamaterials, and quantum neural networks. Hence, they seem to be very promising for studies of novel types of fast quantum-state control or initialization –15. In tIndeed, in the case when the resonance carrier (“filled”) pulses are used for the qubit state preparation and control, the characteristic Rabi-period appears to be in the nanosecond range, which is caused by the small energy separation and near-degeneracy of the atomic levels. For this reason an important task is to decrease the time of the operations with meta-atoms  to improve the speed of quantum computations as well as the relationship between decoherence time and characteristic duration of basic logic gates. One of the possible ways to solve this problem for different types of superconducting artificial atoms –7 is to In this paper we suggest a method for the ultrafast control of the population dynamics and population transfer between the qubit states in superconducting meta-atoms by unipolar pulses using the regime of stimulated Raman Λ-type transitions between them via upper-lying levels. The possibility to perform the simplest “Not”-operation (bit-flip or π-rotation of the Bloch vector of the qubit state around x-axis) on the picosecond scale and even faster is demonstrated. The advantages of unipolar pulses working in the Λ-type configuration in comparison to direct population transitions are established. The experimental realization of the theoretically developed scheme is worked out and described.This paper is organized as follows. In the next section we present our model and discuss the developed theoretical approach. In the section “Results and Discussion” we firstly show the theoretical results obtained for the scheme with blocked direct transitions between qubit states and demonstrate an ultrafast Raman Λ-type “Not”-operation stimulated by a unipolar magnetic pulse (first subsection). Further, we study the influence of additional upper-lying levels of the studied superconducting meta-atom on the time of the obtained ultrafast qubit control. We include an additional fourth level in the consideration and analyze the possible time of the population transfer. Then we analyze the interaction of the superconducting meta-atom with the unipolar pulse for allowed direct transition between qubit states and show this case to be irrelevant (second subsection) and finally we present the experimental realization of our suggested promising scheme (third subsection). At the end we give the conclusions.EJos, EJos and αEJos, α > 0.5. The third junction can be replaced by a superconducting quantum interferometer, which allows one to adjust α using an external magnetic flux. So, we can apply magnetic fluxes ΦX and ΦZ in order to control the barrier height and potential asymmetry respectively. Below, we show that in this way we can control the speed of transition between individual states, which is extremely important for the presented concept.In this section we develop a theoretical approach on the interaction of a multilevel superconducting meta-atom with unipolar magnetic field pulse. The considered model seems to be fairly general since it takes into account the multilevel structure of a real superconducting system and can be applied to describe the dynamics of qubit states in different types of superconducting artificial atoms. However, to be specific, we will consider the well-known flux qubit scheme: a superconducting ring with three Josephson junctions  of the corresponding states with direct transition |φ1⟩ → |φ2⟩ being blocked:and obtain a set of coupled differential equations for the probability amplitudes ij are the magnetic matrix elements of the allowed transitions and for simplicity we suppose all of them to be of the same value: μ31 ≈ μ32 ≈ μ ≈ 106 arb unit. Further, we solve the system |2. The obtained dynamics of the system is compared to that calculated for the case in which the transitions between all the considered states are allowed and found as solution of the following system of equations:Here, μe system  under thSubsequently, the time-dependent population transfer between the qubit states will be compared for both cases. Our goal is to provide an ultrafast transfer of population from one qubit state to another and to establish the regime when the population of the upper level is explicitly equal to zero because of the coherent trapping of population in the two qubit states due to the Raman Λ-type transitions induced by the magnetic field pulse.First we analyze the possibility of the “Not”-operation induced by the unipolar magnetic pulse in the case when the direct transition between the qubit states is blocked. Under such conditions only Raman Λ-type transitions take place and are responsible for the transfer of population from the lower qubit state to the upper one. The system  is solveIt should be noticed that since we consider a unipolar magnetic pulse which has no carrier frequency, Ω is just a designation.Then we introduce new variables:Regarding the new variables, the system  can be rThe solution of where γ is the quasi-energy of the system  and can V induced by a magnetic field is rather strong due to the huge value of the magnetic dipole moment. For these reasons Usually for real superconducting systems Δ ≪ ℏΩ and the coupling between qubit states  corresponding to γ(0) = 0, which is found to beIt is interesting to analyze the quasi-energy wave function 3⟩ is equal to zero. Such a regime is referred to as a coherent population trapping known for real atomic systems in optical electromagnetic pulses [3⟩ is detected.Such a wave function corresponds to the case when the whole population is trapped in the qubit states and the population of the upper state |φc pulses . If such1⟩. We focus on the possible population transfer to the qubit state |φ2⟩. Using the quasi-energies  between the qubit states, or rather, some mean level with energy  and the upper third level. It can be seen that the time of the first |φ1⟩→ |φ2⟩ transition appears to be much shorter than the period TΩ and further decreases with a growing degree of coupling  is normalized by TΩ = 2π/Ω. While this period is in the sub-nanosecond range, the time of the considered gate appears to be up to two orders of magnitude lower. Thus we really achieve very fast gates with switching times of the order of picoseconds, which is dramatically faster than that obtained in the case of a carrier magnetic pulse resonant to the direct transition between spin states. Since in the experimental realization of such a scheme it is rather difficult to provide explicitly the required duration of the magnetic pulse, the required time accuracy providing 98% of transition efficiency is plotted in V. It can be seen that the possible deviation interval decreases with the degree of coupling. But at the same time, the characteristic time of the “Not”-operation also decreases such that the relative required time accuracy remains the same at no less than 6% of TNOT. This accuracy level seems to be achievable and can be provided in real experiments. It can be seen from x- and Oz-axes , this allows us to realize a full set of single-qubit operations through the influence of a unipolar pulse. However, in this paper we focus on the “Not”-operation gate.As a result, the characteristic time of the population transition decreases dramatically with increasing coupling strength without loss of efficiency which is illustrated in V = 10. The time of the first peak of the population transfer appears to be even shorter in comparison to the result of Further, we have analysed the influence of the additional upper-lying levels of the studied superconducting meta-atom on the time of the obtained ultrafast population transfer. We include an additional fourth level into our considerations, and this level has a higher energy than the third one. The corresponding dynamics of population of the levels is presented in Let us compare the results obtained in the previous section with the case when the direct transition between the qubit states is allowed. Such an unusual situation is possible in an artificial Josephson atom . We will1⟩→|φ2⟩ transition is blocked now we observe slower population dynamics with the characteristic time determined not by the separation between the qubit levels Δ, but by the period TΩ = 2π/Ω, which corresponds to the range of nanoseconds or at least to the sub-nanosecond range for real superconducting systems. Hence, the observed population transfer appears to be significantly faster than in the case of resonant Rabi transitions but dramatically slower than in the scheme with blocked direct transition. It should be noticed that the increase of the magnetic field strength (or the coupling between levels) does not influence the characteristic time of the gate at all.The corresponding dynamics of the population of this state is presented in 1⟩→|φ2⟩ transitions it is not possible to speed up the population transfer significantly.With the increase of the energy gap Δ the maximal possible value of the population of the upper qubit state starts to decrease. This means a drop of transition efficiency, while the characteristic transition time remains almost the same see . Thus, i of the system chosen earlier as an example. Here  is the Coulomb energy of the system and  is the Josephson energy. The results of the calculations of the energy eigenvalues and the matrix elements of the operator of the generalized coordinate are presented for different values of the fluxes setting the working point , transitions are allowed only inside the well. In the intermediate case (α = 1–1.2) there is a mode when the |φ1⟩→|φ2⟩ transition is prohibited, but |φ1⟩→|φ3⟩ and |φ3⟩→|φ2⟩ are allowed (Λ-area). In Here, the value ΦAn experimental issue of the proposed picosecond-time qubit control are the difficulties related to generation and application of the required pulses with minimal parasitic thermal perturbation of the quantum system. The first practical approach dates back to implementation of ballistic fluxon qubit readout –28. The 0, are implemented on a different chip that is physically located at a higher temperature stage in the cryocooler. The chips with classical and quantum circuits are connected by using the standard multi-chip module (MCM) technology [An improved version of the classical interface was proposed in a recent work . Here, tchnology . ControlA simpler solution could be utilization of adiabatic superconducting logic (ASL) circuits broadly used, e.g., to control qubits in D-Wave Systems quantum processors . While A±, of the two Josephson junctions asThe sum of the Josephson and inductive energies in the system can be written through the total and difference phases, φl and m are the normalized values for the shoulder inductance  than the best values achieved for direct “Rabi spin-flip”. Up to now, in the Rabi-technique using carrier radio pulses with a carrier frequency of 0.1–100 GHz the achieved gate times appear to be no shorter than tens of nanoseconds due to small energy separation and near-degeneracy of the spin qubit levels . This peAnother important aspect of our suggested technique is the control over the parameters of the experimental scheme with the possibility to block the required transitions and to adjust the energy spectrum and matrix elements to their optimal values obtained from our theoretical consideration. Switching the regimes from the simple Λ-scheme to the specific case of a Δ-atom is shown and the possibility to fulfill the required conditions for the ultrafast control by unipolar picosecond pulses is demonstrated. In addition, in our experimental scheme the parameters of the magnetic field can be tuned and the optimal magnetic field strength and duration are shown to be reproduced exactly. Experiments on controlling the state of the transmon by different trains of SFQ-pulses –2042 hav20It should be emphasized that the proposed ultrafast population transfer between selected levels is important not only for the acceleration of the population transitions in superconducting qubits. It also plays a principal role in ultrafast state initialization for algorithmic quantum computers and quantum neural networks as well as in the fast control of the magnetic properties of media from Josephson meta-atoms.


To develop a novel approach to accurately verify patient set up in proton radiotherapy, especially for the verification of the nozzle – body surface air gap and source-to-skin distance (SSD), the consistency and accuracy of which is extremely important in proton treatment.Patient body surfaces can be captured and monitored with the optical surface imaging system during radiation treatment for improved intrafraction accuracy. An in-house software package was developed to reconstruct the patient body surface in the treatment position from the optical surface imaging reference capture and to calculate the corresponding nozzle – body surface air gap and SSD. To validate this method, a mannequin was scanned on a CT simulator and proton plans were generated for a Mevion S250 Proton machine with 20 gantry/couch angle combinations, as well as two different snout sizes, in the Varian Eclipse Treatment Planning Systems (TPS). The surface generated in the TPS from the CT scan was imported into the optical imaging system as an RT Structure for the purpose of validating and establishing a benchmark for ground truth comparison. The optical imaging surface reference capture was acquired at the treatment setup position after orthogonal kV imaging to confirm the positioning. The air gaps and SSDs calculated with the developed method from the surface captured at the treatment setup position (VRT surface) and the CT based surface imported from the TPS were compared to those calculated in TPS. The same approach was also applied to 14 clinical treatment fields for 10 patients to further validate the methodology.The air gaps and SSDs calculated from our program agreed well with the corresponding values derived from the TPS. For the phantom results, using the CT surface, the absolute differences in the air gap were 0.45 mm ± 0.33 mm for the small snout, and 0.51 mm ± 0.49 mm for the large snout, and the absolute differences in SSD were 0.68 mm ± 0.42 mm regardless of snout size. Using the VRT surface, the absolute differences in air gap were 1.17 mm ± 1.17 mm and 2.1 mm ± 3.09 mm for the small and large snouts, respectively, and the absolute differences in SSD were 0.81 mm ± 0.45 mm. Similarly, for patient data, using the CT surface, the absolute differences in air gap were 0.42 mm ± 0.49 mm, and the absolute differences in SSD were 1.92 mm ± 1.4 mm. Using the VRT surface, the absolute differences in the air gap were 2.35 mm ± 2.3 mm, and the absolute differences in SSD were 2.7 mm ± 2.17 mm.These results showed the feasibility and robustness of using an optical surface imaging approach to conveniently determine the air gap and SSD in proton treatment, providing an accurate and efficient way to confirm the target depth at treatment. The advantage of proton radiotherapy is that along proton beam path, high dose could be tailored to cover the target, while spilling very little dose to normal tissues beyond the target [Wilson theorized that fast protons could be used for treating deep-seated tumors while sparing adjacent normal tissues due to the unique physical properties in 1946 [Currently, volumetric imaging, using cone-beam computed tomography (CBCT) or CT-on-rails, and 2D orthogonal kilovoltage (kV) imaging are the available options for image guidance in proton therapy to ensure patient position before treatment –7. AlthoThe optical surface imaging systems have become widely available to radiation oncology departments and have been routinely used for the purpose of patient setup and motion management. With this type of system, the patient body surface position can be acquired and/or monitored during set up and treatment for improved accuracy . It is cA Mevion S250  proton system has been clinically used to treat patients since 2015 in our department. The proton system is a single-room, gantry-mounted passive scattering proton therapy machine, equipped with a 6D robotic couch and a pair of orthogonal planar kV X-ray imagers for image guided patient setup . It inclAn in-house Matlab program was developed to reconstruct surface from the surface information captured by AlignRT. First, this program renders the 3D surface by extracting vertices of a triangular mesh from AlignRT 3D model file (.obj). The position information of these vertices are then used to calculate the distance between the end of a compensator and all the surface points in the treatment area. The shortest distance from patient surface to the end of compensator inside the treatment field would be the air gap, as shown in Fig. In order to validate this method, CT images of a mannequin , along with the CT body surface (CT surface) in the AlignRT system, was exported from the AlignRT system and imported into the in-house software package. The CT body surface that was imported to the in-house software package from the AlignRT system was also used to compute the air gap and SSD for the purpose of validating and establishing the benchmark accuracy of the developed methodology. Theoretically and ideally, the computed air gap and SSD derived from the imported CT body surface, compared to the corresponding air gap and SSD in the treatment planning system, should be exactly matched. However, due to digital round-off, they could slightly differ from each other. An example of comparison of beam’s eye view (BEV) in TPS, the CT body surface and VRT body surface is shown in Fig. The developed methodology was also retrospectively evaluated using clinical data. In our clinic, the AlignRT system is routinely used for patient setup and motion monitoring at proton treatment. The VRT body surfaces are routinely acquired for the patient treatments to which the AlignRT system is applied. The evaluation was conducted for 10 clinical proton patients. Among the 10 patients, 5 were liver patients, 2 were prostate patients, 2 were mediastinum patients, and 1 was a breast patient. In the clinical practice involving the AlignRT system in our clinic, before treatment, one set of patient body surface is always captured after the patient position (relative to the isocenter) is corrected and verified with the orthogonal kV image pairs. The acquired body surfaces for the 10 clinical cases were used to derive the values of the air gap and SSD for each of treatment fields using our method, and were compared to corresponding TPS values, using the same approach as above with the mannequin phantom. The values of the air gap and SSD were only calculated for the treatment fields with anterior angles since patient posterior surface could not be captured by the AlignRT system due to the camera arrangement.The result comparison of the air gap and SSD for the phantom is presented as a box and whisker plot in Fig. Figure The values of the air gap and SSD calculated from the CT body surfaces with the developed methodology and calculation program agreed well with the corresponding TPS values within about 0.5 mm, validating the methodology with the digital round-off accuracy of about 0.5 mm. The values of the air gap and SSD calculated from the phantom VRT surfaces with our program are also in agreement with the TPS values within about 2 mm, validating the feasibility of utilizing this program to estimate the air gap and SSD with an accuracy of about 2 mm. The deviations calculated from the VRT body surface of the phantom were larger than those calculated from the CT body surface of the phantom, mainly caused by the limitation of the camera imaging accuracy compared to CT images. For the clinical patients, the result deviations derived from the VRT body surfaces were larger, which reflects the fact that the target depths may have exhibited some changes at treatment. The accuracy of the target depth calculations from the air gaps and SSDs can be dependent on multiple factors, including the patient setup accuracy, breathing motion, patient anatomical change, among others. In Fig. The nature of how proton beam deposits energy along its beam path makes it essential to keep the radiological equivalent path length of proton beam consistent through a course of treatment. Otherwise, significant dose deviation may take place, leading to underdosing the target volume or delivering unplanned dose to organs at risk. However, the conventional 2D orthogonal kV imaging IGRT technology may help on the alignment of the target but is often inadequate to detect the target depth change along the proton beam path. Although 3D volumetric IGRT technologies, such as CBCT, may serve for the both purposes, the availability of the volumetric IGRT technologies to the existing proton centers is still limited. Recently, our group has published our experience using automatic measurements of proton air gap based on orthogonal kV imaging with radiopaque wires . The curThe optical surface imaging-based approach for the derivation and verification of the air gap and SSD has limitations in the setting of radiation treatment using posterior fields due to the configuration of the AlignRT camera systems. The three AlignRT cameras are normally mounted on the ceiling in the treatment room and can only capture the patient’s surface from above. The surface that cannot be captured by the cameras would not be able to be used for air gap or SSD calculation. Additionally, based on the captured surface image, one can visually determine if the surface of interest for a posterior beam is blocked by the tabletop or not and whether it is suitable for the calculations of the air gap and SSD. Therefore, the air gap and SSD calculation for some posterior beams may not be feasible with this method if the surface area cannot be captured by the cameras. In the current study, posterior fields were excluded, and only anterior fields were presented for both the phantom and patient studies. At our center, at least 70% of the treatment fields come in anteriorly, as the posterior fields would go through couch and possibly couch rails. Additionally, with the proton machine in our clinic, the patient image guided setup couch position is not necessarily the treatment couch position. The patient setup is normally conducted with the couch at 270-degree position. After the patient positioning is confirmed by the 2D orthogonal imaging system, the couch is returned to the treatment position . The patient body surfaces were also captured by the AlignRT system at the setup position (couch at 270 degrees). Ideally, the surfaces should be captured at the treatment couch position.Although the methodology was developed using the AlignRT system, the principle of the methodology can be applied to any other optical surface imaging system used in radiation oncology clinic. Additionally, the principle can also be applied to the radiation treatments using the conventional photon or electron beams for the clinics where volumetric IGRT technologies are not available. This proposed method would be of value for centers where their proton machines are not equipped with CBCT or treatment CT capability and it provides a sensitive measure to trigger additional evaluations and measures to ensure proper treatment and quality. However, the clinical tolerances for different action levels are site dependent as well as dependent on the planning range uncertainty margins and PTV margins, which may vary from institution to institution. Future studies on the tolerance levels are needed to explore the optimal action and measure when the deviations are detected.We have developed an optical surface imaging based methodology to automatically derive the values of the air gap and SSD for proton treatments. The methodology was evaluated and validated using both phantom and clinical patient data. The results show that it is feasible and robust to use the optical surface imaging approach to conveniently determine the air gap and SSD in proton treatment, with acceptable accuracy. This approach provides an objective and efficient way to confirm the physical target depth at treatment, in order to ensure desired target coverage and normal tissue sparing.


We propose and demonstrate a new imaging technique to noninvasively see through scattering layers with the aid of a spatial light modulator (SLM). A relay system projects the incoherent light pattern emitting from the scattering layer onto the SLM. Two coded phase masks are displayed, one after another, on the SLM to modulate the projected scattered field and the two corresponding intensity patterns are recorded by a digital camera. The above procedure helps to achieve two goals. Firstly, since the coded phase masks are digitally synthesized, the point spread function of the imaging system can be engineered such that the image retrieval becomes more reliable. Secondly, the two recorded intensity patterns are subtracted one from the other and by that the background noise of the recovered image is minimized. The above two advantages along with a modified phase retrieval algorithm enable a relatively easier and accurate convergence to the image of the covered object. Different imaging techniques have been developed in the past to see through scattering layers which can be broadly classified into invasive and noninvasive categories depending upon whether the point spread function (PSF) of the system is measured or not. In the case of invasive techniques, it is necessary to have prior information about either the PSF of the imaging system containing the scattering layer or the phase image of the scattering layer14. Besides being invasive, these techniques are not suitable for imaging through temporally varying turbid media, like fog, or a blood flow, whose PSF varies with time. Furthermore, a high level of reconstruction noise was noticed16 in some of the recently developed 2D and 3D imaging techniques using cross-correlations between the PSF and object intensity response. Consequently, different techniques were developed to suppress the reconstruction noise by engineering the phase masks17, and statistical averaging18, which are not always practical in the case of impenetrable scattering layers. Other deconvolution techniques based on Richardson-Lucy iterative algorithm19 and inverse filter20 are invasive as well, as they require to measure the PSF in order to retrieve the image of the object.Imaging through scattering layers enables seeing objects present behind media such as biological tissues29. In some of these noninvasive techniques, the intensity of the scattered light is autocorrelated and from this autocorrelation, the image of the concealed object is reconstructed using a phase retrieval algorithm31. While the noninvasive technique can retrieve the image without knowing the PSF, the success of the phase retrieval algorithm is highly statistical as the outcome of the phase retrieval method is dependent upon the initial guess of the iterated matrix. Consequently, the phase retrieval algorithm must run several times with different initial random patterns before the optimal result can be obtained24. One of the main requirements for the phase retrieval based noninvasive imaging techniques to be successful is that the phase variation of the scattering layers must be chaotic such that the autocorrelation of the system PSF yields a sharply peaked function with a constant distribution as a background. This requirement highly limits the applicability of the technique. In a formal notation, for incoherent illumination and for object intensity O, the intensity distribution recorded by the camera is O * IPSF, where IPSF is the system PSF and ‘*’ is the sign of 2D convolution. In24, it is suggested to compute the autocorrelation (O * IPSF) ⊗ (O * IPSF), where ⊗ represents the 2D correlation operator. The same autocorrelation can be written as (O ⊗ O) * (IPSF ⊗ IPSF)32, Under the assumption that (IPSF ⊗ IPSF), is equal to a delta function plus a constant (δ + Constant), the autocorrelation of the camera output is approximately (O ⊗ O + Constant) and hence the object distribution can be retrieved by a constant removal procedure and a phase retrieval algorithm. However, the assumption that (IPSF ⊗ IPSF) ≈ δ + Constant is sometimes problematic. First, the autocorrelation of a positive chaotic PSF has positive non-constant background distribution with double the size of IPSF. Second, the width of the delta-like function in certain setups is dependent on the properties of the scattering layer and hence the approximation to a narrow delta-like function is limited only to certain scatterers. Because of these two effects, the approximation (IPSF ⊗ IPSF) ≈ δ + Constant is sometimes not established well, and hence the input of the phase retrieval algorithm is far from the ideal (O ⊗ O) Consequently, the output of the algorithm might not converge to the required image of O.Noninvasive imaging techniques have been developed recently to see through scatterers without knowing the PSF of the system or the phase function of the scattering layersO * [IPSF1− IPSF2] is bi-polar with negligible bias term. Consequently, the background of the autocorrelation of the PSF is much lower than the background obtained from the autocorrelation of a positive function regardless whether this later background is constant or not. These two improvements in the original method bring the autocorrelation of the system response closer to the autocorrelation of the hidden object. Thus, the proposed method increases the chances of the phase retrieval algorithm to converge to an image, which is more similar to the original object.In this study, we present a new approach to extend the applicability of the phase retrieval based noninvasive imaging techniques for imaging through various types of scattering layers. The main concept of the proposed method is to project the scattered light emitted from the scattering layer on a computer-controlled coded phase mask (CPM) which is used as a second scattering mask. By doing that, we intend to achieve two goals. First, since the second mask is synthesized digitally, we have better control on the properties of the combined scattering medium, such that a better approximation to the delta function can be obtained from the autocorrelation of the PSF. Second, in our proposed procedure, two camera shots, with two different CPMs, are captured and subtracted one from the other. Hence, the obtained superposed distribution 33 critically illuminates an object using a lens L1. The light diffracted from the object is incident on a diffuser used as the scattering layer and located at a distance of zs from the object. A relay system projects the light pattern obtained at a distance of zr from the diffuser onto a phase-only spatial light modulator (SLM). The relay system can either be constructed from two identical lenses L2 and L3 with an overall magnification of 1, like in our experiment or if required, from different lenses with a magnification different from 1. The phase mask displayed on the SLM is a modulo-2π phase addition of the CPM, synthesized using modified Gerchberg-Saxton algorithm (GSA)34, and a quadratic phase mask (QPM) with the focal distance f used as a diffractive lens for focusing the light on an image sensor. The light modulated by the SLM is incident on the image sensor located at a distance of zh from the SLM.The optical configuration of the proposed setup is shown in Fig. f. Furthermore, the illuminating sub-system and the relay are omitted since these components have no influence on the mathematical analysis. We choose to represent the object intensity distribution The goal of the following mathematical formalism is to obtain the expression of the intensity distribution over the sensor plane for any arbitrary object and diffuser functions. Knowing this distribution might improve our understanding of the parameters which contribute to the quality of the reconstructed images. A simplified scheme for calculating the intensity over the sensor plane is given in Fig. zh) = 1/f is fulfilled between the object and the sensor planes, where u = zs + zr and the distances zs, zr and zh are defined in Fig. 35. The various shifts of the second and third Delta functions are obtained from ray analysis35 through thin prisms which their transparent function is a linear phase. For the entire points of the object and for the entire linear phases composing the phase masks, the intensity on the camera is a convolution between the three series, as follows,Assume the imaging Eq. (1/u) +  = f(αx). Therefore, the intensity on the camera is,Ao is a constant. Based on Eq.  and their corresponding autocorrelation profiles (IPSF ⊗ IPSF) are depicted for different σ, as shown in Fig. IPSF ⊗ IPSF) for different values of σ for the above two cases are shown in Fig. A point object is introduced into the simulated system, and its complex amplitude propagates through the different optical components of the optical configuration shown in Fig. 32. Such convolution does not yield a PSF with a different bandwidth than that of the scattering layer. Hence, the smallest feature size of the PSF, and the width of the autocorrelation peak in the lensless case are equal to the feature size of the diffuser and inversely proportional to σ. The conclusion from this observation is that in the lensless case24 the assumption (IPSF ⊗ IPSF) ≈ δ + Constant is valid for relatively strong scatterers only. On the other hand, Fig. IPSF obtained on the camera plane is a speckle pattern in which the feature size of the speckles is dependent on the sizes λ, zh, D but independent on σ. Since the feature size of the speckles dictates the width of the autocorrelation peak, this width is independent of σ. Equation , one can estimate the limiting window of the size  on the object plane of the iterative algorithm.The spectral magnitude of the object is fed into the phase retrieval algorithmn in Eq.  divided The phase retrieval algorithm shown in Fig. λ = 532 nm) is passed through a rotating diffuser. The rotating diffuser was constructed using a thin polycarbonate sheet rotated by a DC motor to minimize the spatial coherence of the incoming light. From the diffuser, it was collected by a refractive lens L1 to critically illuminate the object. Initially, we had used a pinhole with a diameter of 100μm in the object plane to validate our assumptions and later, we used a more complex object. The light diffracted by the object passed through a constant ground glass diffuser  with 1500 grit and a thickness of 2 mm, located at a distance of about 5 mm from the object. The diffused light was projected on a phase-only reflective SLM  by a relay system comprising of two identical refractive lenses L2 and L3 with a diameter of 5 cm and focal length of f = 10 cm. The light emitted from the diffuser further propagated a distance of zr = 16 cm before reaching the relay system, which projected the light onto the SLM plane. A polarizer P placed beyond L3 was used to polarize the light along the orientation of the SLM active axis to maximize the modulation efficiency. A quadratic phase mask with a focal length of 6.9 cm was multiplied with CPM synthesized using GSA and both were displayed on the SLM. The reflected light from the SLM was recorded by an image sensor  located at a distance of zh = 11.7 cm, from the SLM.The experimental setup for the proposed method is shown in Fig. µm diameter, three cases were studied. In case 1, only a quadratic phase mask was displayed on the SLM and the intensity pattern (I1) at the sensor plane was recorded. In case 2, CPM with a scattering degree of σ = 0.2 was displayed with the QPM and the intensity pattern (I2) was recorded. In case 3, a second independent CPM was combined with the QPM and another intensity pattern (I2b) was recorded. A bipolar pattern (I3 = I2 − Ib2) was calculated and stored. The patterns and their autocorrelation for the three cases are shown in Fig. In the first experiment with the pinhole of 100 In the next experiment, the digits ‘2’ and ‘6’ from group 4 of United States Air Force (USAF) resolution target were tested. The objects were placed one by one at the object plane behind the static diffuser. First, only QPM was displayed on the SLM and the intensity patterns shown in Fig. Next, two intensity patterns corresponding to two different CPMs both with σ = 0.2 were recorded, where the hidden object is the digit ‘6’. One pattern was subtracted from the other to yield the bipolar pattern. The bipolar pattern indeed reduces the bias term from the autocorrelation as is shown in Fig. The same phase retrieval algorithm was applied on the autocorrelation of the bipolar matrix. The reconstruction results for both the objects ‘2’ and ‘6’, with 25 different initial random are shown in Fig. We have proposed and demonstrated a noninvasive technique to image through a scattering layer, using a two user-controlled phase coded masks with two camera shots. The explanation to the success of the two-shot method is as following; in each intensity pattern measured by the camera after passing through the scattering media there is a dominant bias function, which does not contain any information on the hidden object, but on the other hand, obscures the relatively small signal, which does contain the information about the object autocorrelation. The difference between the two measured intensity patterns created from two independent, but statistically similar CPMs, contains only negligible remains from the bias function, because there is not much difference between the two bias functions. After the elimination of the bias, the autocorrelation of the difference between the two measured intensities mostly contains the information of the covered object autocorrelation as is indicated by Eq. , and hen24 in which the background is removed digitally under the assumption that its distribution is constant, the width of the background can be calibrated such that its distribution can be constant, at least over the area of the observed objects.The method can be easily applied for imaging through different types of scatterers by accurately engineering the autocorrelation profile of the PSF, using suitable QPM to determine the width of the autocorrelation peak and proper CPMs to determine the width of the background distribution. It is important to keep the width of the autocorrelation peak as small as possible and its sharpness as high as possible. As the peak is sharper, the input to the phase retrieval algorithm is more similar to the autocorrelation of the observed object, and hence, the final outcome for the algorithm is more similar to the object. The width of the background distribution is not significant in our method because the background is removed in any case. However, for single shot methods24 by several aspects. First, in the proposed method there are two camera shots captured through two different independent CPMs. That makes the overall two scattering media to be in a weak correlation between them (in comparison to their autocorrelation peaks). Two shots, in comparison to a single shot, add more information about the hidden object to the algorithm and have the effect of averaging and consequently the effect of noise reduction. Moreover, unlike24, our method of background removal from the autocorrelation is not limited to a constant background only. Any background function can be removed as long as the statistics of CPMs and of the scattering layers remains the same between the two camera shots. The technique of background removal in our case is done only by subtracting the two camera patterns obtained from the two independent CPMs. Therefore, there is no need to crop the autocorrelation with any kind of digital window in a trial and error procedure before the phase-retrieval algorithm24.The method is different than the method proposed in24 is that the proposed system is equipped with a diffracted lens displayed on the SLM in arrangement that makes the PSF equal to the magnitude square of the Fourier transform of the scatterer product. Unlike24, this arrangement guarantees that the sharpness of the PSF autocorrelation peak is dependent on the diameter of the system aperture and independent on the scattering degree of the scatterers.Another difference from the lensless system of24. However, for SLM with refresh frequency standard of 60 Hz, an acquisition of 30 frames per second can be regarded as a real time imaging. Therefore, a video of dynamic hidden object can be recorded assuming the iterative phase retrieval process of the image reconstruction is performed off-line as many times as needed on the entire pairs of captured raw patterns. The averaging on the many outcomes from the phase retrieval algorithm is also done off-line without slowing the double-shots capturing process.The proposed technique needs two exposures and apparently is slower than single-shot methodsWe believe that the proposed technique will be useful for imaging through scattering layers with lesser complexity and time compared to the existing techniques. In this study, the method is only applied for 2D imaging. Therefore, additional studies are required to improve the system performances toward 3D imaging systems.