Fourier ptychographic microscopy (FPM) is a new, developing computational imaging technology. It can realize the quantitative phase imaging of a wide field of view and high-resolution (HR) simultaneously by means of multi-angle illumination via a light emitting diode (LED) array, combined with a phase recovery algorithm and the synthetic aperture principle. However, in the FPM reconstruction process, LED position misalignment affects the quality of the reconstructed image, and the reconstruction efficiency of the existing LED position correction algorithms needs to be improved.This study aims to improve the FPM correction method based on simulated annealing (SA) and proposes a position misalignment correction method  using an improved phase recovery strategy.The spectrum function update strategy was optimized by adding an adaptive control factor, and the reconstruction efficiency of the algorithm was improved.The experimental results show that the proposed method is effective and robust for position misalignment correction of LED arrays in FPM, and the convergence speed can be improved by 21.2% and 54.9% compared with SC-FPM and PC-FPM, respectively.These results can reduce the requirement of the FPM system for LED array accuracy and improve robustness. Traditional optical microscopes are limited by the space-bandwidth product (SBP)In an FPM system, LEDs at different positions on the LED array generate plane waves with different incident angles to illuminate the sample. The spatial position of each LED directly determines the position of the corresponding LR image in the frequency domain;In our previous study, we improved the conventional FPM reconstruction algorithm, which is referred to as the AA algorithm.The remainder of this paper is organized as follows. We introduce the basic model of the FPM and LED array position model in Sec. 22.1First, we establish a position model for the LED array. As shown in Here, 2.2A flowchart of the proposed method is shown in The following are the specific steps and process of algorithm implementation:Step 1:Initialization: Before running the iterative algorithm, take the corresponding LR image illuminated by the central LED and perform Fourier transform to obtain the initial HR spectrum guess Step 2:Use the pupil function to intercept the information in the initial HR spectrum of the sample, which is equivalent to low-pass filtering, and then inverse Fourier transformed to the spatial domain to generate the target complex amplitude image Step 3:In each iteration, the angle corresponding to each LED is processed  before completing one iteration of the algorithm. However, in the initial iteration, we generally use bright-field (BF) images because they contain relatively more low-frequency information, which is more important for image reconstruction; BF images are less affected by noise, and more accurate parameters can thus be obtained at the beginning. Therefore, we first iterate over the images with low Then, in the FPM reconstruction process, the SA algorithmThe simulated target complex amplitude image is then: A smaller value of Step 4:After updating the position of the frequency aperture A Fourier transform is then performed on the updated LR target image to obtain the updated HR spectrum after replacing the amplitude: Step 5:To reduce the effect of abrupt changes in the value of the pupil function on the spectrum update, the ratio is added to the spectrum update process, defined as Then, the adaptive control factor The value of Step 6:Repeat steps 2 to 5 for different LEDs on the LED array until all spectral information within all sub-apertures have been updated  curves of the HR intensity image and the phase image reconstructed by the SC-FPM, PC-FPM, and AA-C algorithms are shown in 3.2The simulation results verified the effectiveness of the AA-C algorithm. To further verify the performance of the algorithm, we tested it using image data collected from actual experiments. The USAF resolution target was selected as a sample, and the experimental parameters were consistent with those used in Sec. To further validate the effectiveness of the algorithm, we experimented with biological samples. Select human tumor cells were selected as a sample, and an The results of the reconstruction of human tumor cells using different algorithms are shown in Next, we selected human blood smears as samples for the experiment. A 4This study presents improvements to the FPM correction method based on the SA algorithm and proposes a position misalignment correction method  with an improved phase recovery strategy that effectively improves the image reconstruction quality by adding an adaptive control factor and optimizing the spectrum update strategy during the reconstruction process, which does not add to the complexities of the calculation and also has a robust convergence performance. Through simulations and using the USAF resolution target, human tumor cells, and blood smears as samples for the experiment, the effectiveness and robustness of the proposed AA-C algorithm for LED array position misalignment correction were verified. The proposed method accelerated the convergence speed and reduced the effect of LED array position misalignment on the FPM reconstruction quality. Compared with other advanced position correction methods, it can obtain a better reconstruction effect and improve the robustness of the FPM system. In addition, because the imaging efficiency of the FPM requires further improvement, we can combine LED multiplexing to further improve the imaging efficiency of the FPM and achieve faster and higher-quality high-throughput quantitative phase imaging. This will be the direction of our work in the future.


Many animal aggregations display remarkable collective coordinated movements on a large scale, which emerge as a result of distributed local decision-making by individuals. The recent advances in modelling the collective motion of animals through the utilisation of Nearest Neighbour rules, without the need for centralised coordination, resulted in the development of self-deployment algorithms in Mobile Sensor Networks (MSNs) to achieve various types of coverage essential for different applications. However, the energy consumption associated with sensor movement to achieve the desired coverage remains a significant concern for the majority of algorithms reported in the literature. In this paper, the Nearest Neighbour Node Deployment (NNND) algorithm is proposed to efficiently provide blanket coverage across a given area while minimising energy consumption and enhancing fault tolerance. In contrast to other algorithms that sequentially move sensors, NNND leverages the power of parallelism by employing multiple streams of sensor motions, each directed towards a distinct section of the area. The cohesion of each stream is maintained by adaptively choosing a leader for each stream while collision avoidance is also ensured. These properties contribute to minimising the travel distance within each stream, resulting in decreased energy consumption. Additionally, the utilisation of multiple leaders in NNND eliminates the presence of a single point of failure, hence enhancing the fault tolerance of the area coverage. The results of our extensive simulation study demonstrate that NNND not only achieves lower energy consumption but also a higher percentage of k-coverage. In recent decades, there has been a significant interest in studying the spectacular phenomenon of animal collective movement. Such collective movement can be observed in various biological events, including the migration of bacteria ,2,3, theThe collective movement of animals inspired many models ,14,15,16The analysis and modelling of collective movement in animal aggregations have had a profound impact on the development of decentralized control mechanisms for self-deploying mobile sensors . In a neBy employing the nearest neighbour rule, the collective movement within the network can converge all the mobile sensor nodes and guide them along the same trajectory based on their initial headings. The concept of convergence in mobile nodes over a period of time is stated in , while aBuilding upon the findings presented in ,23,25,26Although Cheng algorithm ,22 proviIn this paper, we propose a distributed node deployment algorithm, called Nearest Neighbour Node Deployment (NNND), to achieve blanket coverage in a mobile sensor network. Leveraging the advantages of parallelism, NNND facilitates the creation of multiple sensor motion streams, with each stream heading towards a different region of the designated area. To ensure cohesion within each stream and eliminate the risk of a single point of failure, a leader is adaptively elected for each stream. This utilisation of multiple leaders enhances fault tolerance and incorporates collision avoidance mechanisms. As a result, the algorithm effectively reduces the total distance travelled by sensors, thus minimizing energy consumption. It especially holds high importance as optimal coverage with a limited number of sensor nodes is a significant topic in industry and academia . FurtherThe rest of the paper is organised as follows. A Wireless Sensor Network (WSN) includes a set of sensor nodes All the sensors in the area are assumed to be mobile sensors and can move freely across the area without any restrictions, i.e., there is no obstacle in the area.The sensors are capable of accurately determining their positions within the area and effectively communicating their information to neighbouring sensor nodes.The sensors are aware of their environment and can detect when they cross the predefined borders within the designated area.In every time step The values of the aforementioned parameters are defined as Δi−jThe All sensors possess the capability to detect the boundaries of the Region of Interest and determine the orientations of their tangents. In the case of vertical lines, the assumed orientation is equal to The sensing range of every sensor node, In addition, the relation between At each time period We adopt the following commonly used assumptions from the literature ,22,23 toThe Nearest Neighbour Node Deployment (NNND) algorithm is a distributed algorithm designed for mobile sensor nodes to achieve blanket coverage in an area. Our algorithm incorporates the nearest neighbour rule and control laws proposed in ,22 to faIn the algorithms presented in ,22, contTo enhance the coverage efficiency, we introduce two additional lines, i, is initialized at the first step as The NNND algorithm aims to enable sensors to determine their optimal locations in a distributed manner by exchanging information with their neighbouring sensors, denoted as Taking into account the initial step and the initial values of the variables, the NNND algorithm utilizes the nearest neighbour rule and control laws to guide the movement of sensors in subsequent time steps. This movement can be projected in two perpendicular directions: (a) the Stabilizer axis and (b) the Thrust-Drag axis. These directions determine the desired trajectory for each sensor. Thus, the new positions of the sensor nodes are updated based on the resultant force acting on each sensor. The resultant force considers factors, such as the influence of neighbouring sensors, environmental conditions, and control laws.The Stabilizer axis refers to the direction that helps maintain stability and balance in the sensor movement after applying the nearest neighbour rule. It ensures that the sensors stay aligned and coordinated with their neighbouring nodes while navigating through the area. This axis plays a crucial role in achieving a cohesive and synchronized motion among the sensors . On the By considering both the Stabilizer axis and the Thrust-Drag axis, the NNND algorithm enables the sensors to navigate the space while maintaining coordination and making progress towards the goal of achieving full blanket coverage.n sensors, The Stabilizer axis has the same direction Equation . The χiΓ, Square (2 × 2), and Deep (1 × 4). These area formats are chosen to further study the impact of the width and length of the area on sensor movements, sensors’ location and the area coverage. Moreover, for each scenario where area is considered constant we consider a range for the number of sensor nodes in the area to cover the minimum of the minimum and the maximum of the minimum number of sensor nodes for that area. In the following section, the results of our studies for all scenarios are discussed with X width and Y length, with n number of sensors and sensing range The different behaviours of the NNND and Cheng algorithm in regard to the depth of the area are related to their different principals towards sensor movement at each time step. In an area with While the total movement of sensors in NNND algorithm is calculated as:X value by a considerably larger number, which amplifies the total travelled distance. However, in the case of NNND, the total travelled distance is influenced by both X and Y values. Nevertheless, it is the Y values that hold a stronger influence over the final outcome. The described equations verify the behaviour of these algorithms in different areas. It demonstrates that although NNND algorithm has a significantly less total movement of sensor nodes in different shapes of an area, its behaviour is different from the Cheng algorithm as they use different mechanisms for employing sensor nodes.To address the first question, an analysis of Equations  and 33)33) can pIn Finally, the last component of the sensor movement in our simulation study is the wasted walk. As described in the assumptions section of this paper, sensor nodes are able to move freely across the area. Every sensor node in the area can move in four directions. They can move up, down, right and left or a combination of these directions. However, providing blanket coverage requires only specific movements for each sensor node with designated directions from the initial location of the sensor node towards its proper destination. For instance, if a sensor node’s initial location is on the top-left side of the Region of Interest, any movement to the left and up is considered wasteful since it is initially outside the required RoI and does not lead towards the proper location.While achieving blanket coverage is possible even when not all directions align perfectly, it is crucial to closely examine sensor movements for algorithm efficiency and energy usage optimization. The inefficient movement, which is usually inevitable due to various reasons, is referred to as the wasted walk. Analysing the wasted walk provides a better understanding of the proposed algorithm’s efficiency. In The total time consumed by an algorithm or the duration of its deployment to locate sensor nodes in their proper locations within the designated area is another parameter that can affect algorithm efficiency. In The higher deployment time for the deeper areas is not a concern in the Cheng algorithm, as sensor nodes move sequentially and do not wait for their turn to be allocated to their lines. However, as shown in the figure, in most scenarios (wider and square shape area) Cheng algorithm requires more time to deploy sensor nodes in the area. The superiority of the Cheng algorithm dominates the parallel movement of sensors in NNND after some point that the Y to X ratio is higher than the The sensing potential is defined as the total sensing area by existing sensor nodes in the designated area. Generally, the total sensing potential expands by increasing the number of sensor nodes in the area. However, in deployment algorithms such as Cheng and NNND, the total sensing potential depends on the algorithm layout for sensor nodes’ placement. In In both Cheng and NNND algorithms, sensor nodes move in lines with the same static distance between each line. Therefore, they follow the same layout for sensor nodes’ placement. However, the number of sensor nodes on each line is adjustable and varies for different configurations in the NNND algorithm, while it is considered a fixed number in the Cheng algorithm. Based on the existing number of sensor nodes and the length of the area in the NNND algorithm, the number of sensor nodes on each line is calculated as k in the k-coverage, is a legitimate interest. The k-coverage demonstrates that every point within the specified area is covered by a minimum of k different sensor nodes.In the Cheng and NNND algorithms, the coverage percentage is not a valid parameter to evaluate the algorithms, as the final coverage of the requested area is always In order to explore the k-coverage of these algorithms, the thermographic snapshots are considered after the deployment of each algorithm is finished. The thermographic snapshot of the final coverage for 64 sensor nodes in the different shapes of RoI with the same area, 4, is used in the following figures to present the coverage for every point in these RoI and understand the layout of sensor nodes in each algorithm. The thermographic snapshot presents 2-coverage points with dark blue and the maximum 18-coverage point by dark red in the RoI. Also, the assigned colour to other the k-coverage is specified as a range in every figure. In the following three thermographic figures for three different forms of area, , are demonstrated.In The final deployment of 64 sensor nodes in a square-shaped area is shown in Although accommodating the higher number of sensor nodes in a requested area is one of the advantages of NNND, this superiority diminishes as the Y to X ratio of an area increases. The increase in depth of the area decreases the number of sensor nodes on every line. The fewer required sensor node on the line decreases the number of additional sensor nodes on each line. Therefore, the average k-coverage in the deeper area is less than in the wide-shaped and square-shaped areas. In Blanket coverage plays a pivotal role in numerous applications within Wireless Sensor Networks (WSNs). It serves as a fundamental service that is essential for the successful operation of various WSN applications. Distributed blanket coverage algorithms, particularly those based on the Nearest Neighbour rule, have emerged as pioneering solutions in Mobile Sensor Networks (MSNs) taking inspiration from the collective movement observed in animal aggregations. However, the proposed algorithm in the literature often suffers from drawbacks. Firstly, sensors tend to move sequentially and at a slow pace, which leads to unnecessary energy consumption due to excessive movements. This inefficiency arises from the lack of coordination and parallelization in their motion. Furthermore, these algorithms often overlook the importance of considering environmental conditions, resulting in sensor movements that extend beyond the intended area. This lack of environmental awareness can compromise the accuracy and effectiveness of blanket coverage in MSNs.To address these challenges, we have proposed a distributed node deployment algorithm known as Nearest Neighbour Node Deployment (NNND) to provide blanket coverage across the designated area while minimising energy consumption and enhancing fault tolerance. Unlike other algorithms that rely on sequential sensor movements, NNND takes advantage of parallelism by introducing multiple streams of sensor motions, each targeting a specific section of the area. The presence of multiple streams ensures efficient coverage while reducing energy consumption. By adaptively selecting a leader for each stream, the algorithm maintains cohesion and eliminates single points of failure. This approach significantly enhances fault tolerance and incorporates collision avoidance mechanisms, resulting in a network that is more robust and reliable.Through an extensive performance study, we have compared the final k-coverage, total movement, and time consumed by the NNND against those of the Cheng algorithm. The simulation results confirm the advantages of the NNND, as it achieves enhanced fault tolerance while minimizing energy consumption.


Recent advances in synthetic biology are enabling exciting technologies, including the next generation of biosensors, the rational design of cell memory, modulated synthetic cell differentiation, and generic multifunctional biocircuits. These novel applications require the design of gene circuits leading to sophisticated behaviors and functionalities. At the same time, designs need to be kept minimal to avoid compromising cell viability. Bifurcation theory addresses such challenges by associating circuit dynamical properties with molecular details of its design. Nevertheless, incorporating bifurcation analysis into automated design processes has not been accomplished yet. This work presents an optimization-based method for the automated design of synthetic gene circuits with specified bifurcation diagrams that employ minimal network topologies. Using this approach, we designed circuits exhibiting the mushroom bifurcation, distilled the most robust topologies, and explored its multifunctional behavior. We then outline potential applications in biosensors, memory devices, and synthetic cell differentiation.  •Optimization enables gene circuit design with target bifurcation diagrams•The method finds robust topologies and kinetic parameters for bespoke circuit dynamics•Optimization on the mushroom bifurcation shows cross-repression as most robust topology•Robust mushroom topologies show capabilities as biosensor and memory devices Genetic engineering; Developmental biology; Biological sciences tools; Bioengineering Toward building the synthetic cell from the bottom-up, ,,One of the main challenges of synthetic biology is to design and implement gene regulatory circuits capable of complex behaviors in a near-optimal fashion while keeping a minimal design.,a priori knowledge of parameters and steady-state solutions, hampering the integration of bifurcation diagrams within automated algorithms for circuit design. Alternative methodologies based on chemical reaction network theory,,The limited resources of the cell restrict the combination of multiple working circuits in the same organism.(i)Automated design of gene circuits that not only exhibit behaviors compatible with a target bifurcation diagram but are also optimized for specific additional criteria (like metabolic cost) given by sets of functions.(ii)Systematic exploration of minimal topologies compatible with the required behavior.(iii)Robust design i.e. finding robust topologies with respect to parameter perturbations .(iv)More sophisticated design tasks, computing optimal trade-offs between design objectives and finding Pareto optimal topologies with respect to different opposing criteria .In this work we address this last topic and present a method for automated design of gene circuits with pre-specified bifurcation diagrams. The method extends and combines efficient global mixed integer nonlinear programming optimization methods with an innovative procedure for bifurcation detection, allowing the following novel features:,,A multifunctional behavior of special interest for synthetic biology applications is the mushroom bifurcation, named after its mushroom-shaped bifurcation diagram, originally identified in the differentiation of neural stem cells.,,,,,,,In certain aspects, the phenotypic behavior of the mushroom is an extension of “band-detect” gene regulatory networks, which are constructed from incoherent feedforward loops.y of integers, such that j inhibits gene i, j activates gene i and zero otherwise. The mushroom characteristic bifurcation diagram is depicted in inverted mushrooms. Nevertheless, the method introduced in this manuscript allows us to restrict also the particular location of the saddle nodes . Here we apply this methodology for the design of biocircuits exhibiting mushroom bifurcations with particular gene expression levels for the saddle-node bifurcations. Given that the mushroom bifurcation diagram contains four saddle-node bifurcations, we expect that a network containing two pairs of cross-repressing genes A can givodes see C, which odes see .We search for topologies leading to a mushroom bifurcation diagram behavior using an optimization algorithm and a multistart strategy , corresponding to structures A5 and A7. Interestingly, this global topology screen did not return the mushroom topology studied inThe key distinguishing feature for all topologies is the requirement of the signal activating both genes. A screening looking for the existence of mushroom bifurcation diagrams in the presence of a single activating input did not return any successful results, suggesting that the double activating role of the input is a requirement to reproduce the mushroom diagram.Mushroom3D). Unlike exhaustive exploration strategies, our optimization-based approach finds structures fulfilling the target behavior very efficiently, in the order of seconds per run using a standard PC. The most frequent structures found are depicted in With the same optimization strategy, we explore three-dimensional topologies leading to mushroom bifurcations by fixing the number of genes to three. The corresponding bounds for the parameters for the 3-gene network are included in A circuit is considered functional if it shows a mushroom bifurcation. Nevertheless, it is important to in additionally assess its robustness by quantifying how the circuit functionality is kept with respect to perturbations in the parameters. With this aim, we defined different robustness scores based on the size of the parameter space occupied by successful parameter sets see  section.A similar analysis was performed for the successful topologies in 3-node networks. From the point of view of circuit implementation, we are interested in finding a good compromise between robustness and the number of connections. In order to select the best structures we build a Pareto front in the objective space (robustness vs. number of connections) as depicted in B). Finally, the two saddle nodes forming the neck of the mushroom can collide, pinching the neck of the mushroom and producing an isola . This reveals the mushroom’s capabilities as a biosensor, only allowing the activation (transition from OFF to ON) for precise levels of a target signal located in the neck of the mushroom. Furthermore, since the neck of the mushroom is surrounded by two saddle-node bifurcations, the resulting dynamical ghosts will slow down the transition to the ON state.In addition to the mushroom bifurcation, the isola also provides compelling functionalities. In contrast to a genetic toggle switch, transitions out from the isola are irreversible. Hence, once in the isola state the system is able to detect a signal that goes outside of the range of the isola (either low or high), unable to reach back to the isola independently of future levels of the signal C and D. ,in vivo detection of inflammation,,,In this work, we have developed an approach to biosystem design based on the specification of the bifurcation structure (through the location of saddle nodes in the bifurcation diagram) and applied it to the case of the mushroom bifurcation of interest in developmental and synthetic biology. We found gene regulatory networks composed of two and three genes that displayed the desired behavior. In particular, we found that a system based on the genetic toggle switch incorporating a signal activating both genes was sufficient to reproduce a mushroom bifurcation diagram. We also explored the robustness of these circuits and built a Pareto front to capture the trade-off between the robustness and number of connections, identifying the best topologies to implement the circuit using synthetic biology tools. In addition, we explore specific dynamical functionalities of the mushroom that can be exploited to build biosensors with tuneable temporal and precision properties. We extended our exploration to the formation of different bifurcation diagrams, which occur in the parameter space close to the mushroom bifurcation. We showed that a variety of dynamical behaviors can be created that can provide interesting memory and signal-detecting capabilities unattainable with previous memory devices. Interestingly, this variety of behaviors was attained mainly by inducing differences in the degradation rates of the genes of the network, which is a common perturbation tool in synthetic biology,Our results complement the small amount of other existing work on mushroom bifurcation. A recent study explored four incoherent feedforward networks with positive auto-regulation for their potential to display mushroom dynamics.Overall, designing biological systems based directly on bifurcation properties provides a natural tool to explore phenotype-genotype relationships, relating topologies of the network with the available dynamical behaviors. Such an approach allows for the specification of a set of functional requirements without precision of explicit integration of the differential equations. In addition, it provides a tool to design target dynamics that occur in a robust manner making these approaches key to the future of engineering of biological systems where uncertainty dominates. In future work, we will extend the method to encode different types of bifurcations (including the Hopf bifurcation) related to other complex nonlinear behaviors of interest in systems and synthetic biology.Future research on this topic should take into account some limitations on the use of automated bifurcation design. Firstly, the choice of regulatory functions used in the study can significantly impact the results obtained, implying that it is necessary for case-by-case precise biochemical knowledge of the specific promoter and regulatory reactions used in experimental designs of the circuit. Additionally, the study does not account for stochastic effects such as intrinsic or extrinsic noise, which may influence the bifurcation behavior of the system. Finally, other cellular processes, such as competition for protein degradation, or metabolic overload; are not considered, requiring more tailored constraints in the optimization to real-world synthetic biology scenarios.r.perez-carrasco@imperial.ac.uk).Further information and requests related to code should be directed to and will be fulfilled by the lead contact, Ruben Perez-Carrasco . We consider that the rate constants governing the strengths of the active interactions k). As an illustrative example, the three gene system in k are repression strengths and S, given by Hill functions:R is the total concentration of transcription factor, S is the concentration of signal inducer, K is the dissociation constant, and the cooperativity is 2. We assume that repressor proteins bind as dimers . Depending on the specific design scenario, a number of assumptions can be made to reduce the dimensionality of the parameter search space, here we consider x, y in the form given by In this study, we use the superstructure in network in S as bifurcation parameter and, for convenience, we express the dynamics in the compact form:Importantly, previous to the formulation of the design as an optimization problem, we have to select an appropriate bifurcation parameter. We use the concentration of the inducer input S. Here, the bifurcation of interest is the saddle-node bifurcation . In particular, there are 4 saddle-node bifurcations in the mushroom-shaped diagram (as indicated in figure below).Once the dynamics are encoded in the mixed integer form (2), we formulate the conditions for the target bifurcation in terms of the design variables ,,,A parametric condition for saddle-nodes in biochemical networks with mass action kinetics was demonstrated by.Starting from Mushroom-shaped bifurcation diagrams show four saddle-node bifurcation points, see figure in S here denotes a four dimensional vector, being Note that The optimization problem is a Mixed Integer Nonlinear Programming problem (MINLP). Due to its non-convex nature, global optimization solvers are needed to obtain adequate solutions.Mushroom2D and Mushroom3D, respectively.For the screening of 2-gene and 3-gene topologies with mushroom behavior, we use a multistart strategy with 2000 and 10000 runs, respectively. As indicated in the In advanced design applications, it is usually the case that multiple objectives need to be taken into account.subject to:The solution of the above multicriteria mixed integer nonlinear problem is not unique, but a set of vectors representing the best compromises between the  objective functions. This set of best trade-offs is usually known as the Pareto set.The isola-behavior can be also designed automatically, taking into account that at the isola both bifurcation curves of steady-state solutions Computing the Pareto set of optimal designs by solving the above problems efficiently and reliably can be a daunting task due to their non-convexity, arising from their highly constrained, partially discrete and non-linear nature.Here we transform the multicriteria mixed integer formulation (MO-MINLP) into a finite set of single-objective mixed integer (MINLP) problems by adopting an ε-constraint approach.In order to assess the robustness of a topology, we used two different scores based on the covariance and the interquantile range of the successful parameter sets for that topology see . On the


Background: This article proposes a method for analysing the degree of maturity of Health in All Policies (HiAP) among World Health Organization-French Healthy Cities Network (WHO-FHCN) as part of the GoveRnance for Equity, EnviroNment and Health in the City (GREENH-City) project. We focused on the creation or enhancement of health-promoting environments, and more specifically, public green spaces.Methods: We conducted a cross-sectional quantitative study guided by the evaluative framework of the HiAP maturity level developed by Storm et al mixed with a qualitative interpretation. A self-administered questionnaire was sent to elected officials and health department officers in the 85 member cities of the WHO-FHCN in 2017. Subsequently 58 cities were included in the analysis, which was based on a multiple correspondence analysis (MCA) and a hierarchical ascending classification (HAC). Results: Thirty-two criteria among a total of 100 were identified and were used to organize the cities into 8 groups which was then reduced to three profiles among the cities: a less advanced HiAP profile, an established HiAP profile and an advanced HiAP profile. This process allows us to identify 4 dimensions that make it possible to evaluate the level of maturity of cities in the HiAP process, namely: (1) the consideration of social inequalities in health and/or health issues in the policies/actions of the sector studied, (2) occasional intersectoral collaboration, ie, one-off initiatives between the health department and others sectors, (3) the existence of joint projects, ie, common projects between two or more sectors, (4) the existence of intersectoral bodies, in this case on the theme of urban green spaces including an intersectoral committee and/or working groups. Conclusion: Four dimensions which allow to the measurement of the degree of progress in implementing health-all-policies are proposed. With a view to integrating knowledge into public action, this study carried out under real conditions offers a realistic method to evaluate HiAP. Implications for policy makersThe municipalities involved in a HiAP approach will require simple criteria to assess and monitor their practice and achievement. The four dimensions identified in this article may help health sector decision-makers advocate for more consideration of the impact of the wider health determinants. Urban green spaces are used as an example to raise awareness amongst policy-makers of their importance for health. The methodology should be applied to other policy areas such as education policies or urban planning policies.  Implications for the public The determinants of population health go far beyond the scope of the health-care sectors and require cross-sectoral action. Local policies can have a major impact on health and health inequalities. When applied to different sectors, such as urban green spaces, this approach can have a direct impact on health by improving the living conditions of the population, such as local infrastructures for walking, cycling or playing. By paying particular attention to the need of all, especially to vulnerable populations due to their age, physical or social conditions, this approach can create a more inclusive society.2 Health in All Policies (HiAP) is an intersectoral approach to public policy that systematically takes into account the consequences of all political decisions in population health in formulating public policy. It seeks synergies among all different sectors and avoids negative consequences of policies on individual and population health in the interests of equity.5 The objective of this article is to propose, through a cross-sectional implementation study, a method for analyzing the degree of HiAP maturity of cities based on the framework proposed by Storm et al.6 Our study has a particular focus on health-promoting environments, specifically public urban green spaces. The vast majority of the determinants of health are influenced by factors outside the health-care sector.7 The city members of this network are committed to adopting a HiAP approach as a mode of governance.8 Different conceptions of HiAP implies different operational implementation, as well as its level of deployment. Studying the implementation of HiAP is crucial to ensuring all determinants of health are taking into account when developing public policy. This study was conducted in partnership with the World Health Organization-French Healthy Cities Network (WHO-FHCN) in the framework of the GREENH-City project.11 To varying degrees, many countries have taken up this approach and integrated it into their health governance systems. Literature has been published during the last ten years on the theoretical conceptualization of the HiAP approach and models to explain its functioning,13 facilitating factors and its capacity to be sustained over time.14 Recent publications16 show that most of the scientific production is concentrated at the national or regional level, or at least at state or federal decision-making levels. In some countries, studies are beginning to look at the implementation of these top-down strategies at the local level, such as Norway,17 where the principle of HiAP is integrated into the governmental system, or the Netherlands.18 The implementation of HiAP, whether at the national or local level, depends on the extent to which health issues are central, to collaborative activities between services.19 At this level, there is greater proximity between decision-makers and the population, and between decision-makers from different sectors, a proximity likely to promote intersectoral collaboration and therefore action on living environments.20 In theory, the local level is the best place to operationalize HiAP and to promote health and equity.22 Although there are still few published studies, local-level experiences in implementing a HiAP approach have received increasing attention in the last two years.24 Factors supportive of HiAP are known. These includes a shared vision of HiAP among the actors involved, funding to support the approach, ownership and accountability, local leadership and a dedicated team, health impact assessments, and the existence of local health indicators and process indicators.25 However, objectifying effective implementation of the HiAP approach remains a challenge and requires innovative evaluation approaches. Local policies, particularly those defined at the city level, are likely to have a greater impact on health equity.26 Until now, evaluations have used a realist approach, which focuses on characterizing the mechanisms that facilitate the meeting of different sectors , according to the context of their intervention and the actors involved.28 Qualitative description has also been used to report the results in terms of the success of their implementation.12 Van Vliet et al23 point out that current research mainly produces recommendations or proposes narrative evidence of the enlistment of different policy sectors around a given health issue, without giving the keys to their success or failure. The aim of most evaluations is to assess the conditions that facilitate the implementation of cooperative approaches rather than the actual degree of implementation of the approaches.6 At the local level, there is little literature proposing this degree of implementation evaluation. Publications focus only on specific aspects of the HiAP approach, such as its inclusion in urban planning policies that are favourable to health,29 or testify to its deployment by evaluating the integration of decision-makers’ knowledge on equity.30 In our study, we wanted to estimate the level of concrete implementation of HiAP amongst healthy cities members, following the adoption of the Copenhagen Consensus.31 This reaffirms that, in addition to promoting health and well-being, cities must work to create urban environments that contribute to equity and prosperity for their inhabitants.33 This type of a HiAP-based public action is prevalent in the engagement of healthy cities.34 Although by joining the network, cities commit themselves to respecting its ethical principles, the implementation of HiAP is no homogeneous. There is a need to establish a practical approach to assess the level of HiAP among stake-holders such as municipalities. This is particularly true in the case of France because, unlike some other countries, there is no specific strategy at the national level proposing an operational implementation of HiAP.35 Process and outcome evaluations of HiAP are complex. Based on the Storm’s framework, our study aimed to propose a new method for analysing the degree of maturity of the HiAP approach. It measured the WHO network of healthy cities’ commitment to HiAP and to consider where the issue of urban green spaces is situated among their policies.36 Maximizing the positive effects of urban green spaces depends on their design, accessibility, and nature, and minimizing their potential negative effects . The benefits of urban green spaces are now well documented in the scientific literature, but some public authorities still ignore these multiple advantages in their urban plans.39 Urban green spaces are one of the environmental determinants of health for populations living in cities. Our quantitative study measured HiAP among 85 municipalities from the WHO-FHCN.38 were the basis for the design of the collection tool (questionnaire) and also the analytical framework for analysing the degree of maturity of the HiAP approach among municipalities who replied. The conceptual framework developed by Storm et al along with two other sources6 draws on management science, to describe the five levels of HiAP maturity – recognized, considered, implemented, integrated, institutionalized. It is based on the description of 14 characteristics related to the consideration of social inequalities in health issues  the maturity of HiAP in particular, (2) and how health and equity issues were integrated into their green spaces policies. More specifically, we sought to understand:9Whether health was placed at the heart of each municipal sector of activity  or whether health issues are only of interest to the health sector ie, the type of strategies implemented based on Baum et al.6Whether or no municipalities have the conditions to implement HiAP using the Storm et al framework.40 as part of HiAP considerations. Whether municipalities using health-promoting components of green spaces interventions as defined in WHO expert reports The questionnaire contained 100 questions, including both closed and open-ended questions. The questionnaire is available in French in  During the analysis, the open-ended questions were recoded to transform them into quantitative variables  and the number of variables large, we selected only those variables which specifically explained the degree of the HiAP maturity. Based on the classification of variables determined in step 1, we then selected 32 variables that best characterize HiAP maturity see . Here we41 The MCA that we carried out with SAS© enabled us to identify the most discriminating variables between the cities with regard to their implementation of the HiAP approach and to eliminate those that were redundant (because they are correlated with each other). These variables were then used in the HAC to build the profiles. In order to create city profiles reflecting their HiAP, we implemented a multiple correspondence analysis (MCA) coupled with a hierarchical ascending classification (HAC). We opted for a HAC because we did not want to fix in advance the number of homogeneous groups of cities. The statistical analyses created homogeneous groups of cities, ie, those with similar profiles. In order to interpret these profiles, we conducted a qualitative analysis consisting of the analyse of each variables . The aim was first, to understand which characteristics were used to group cities into similar classes. Secondly, to analyse their degree of progress in the implementation of a HiAP approach according to the Storm categories. Out of 85 eligible cities, 72 cities responded to the questionnaires (84% response rate). According to the inclusion criteria, 58 responding cities were included in the analysis see . The HAC based on 32 variables produced the dendrogram shown in  Each city classes replies are listed in  We identified three standard profiles depending on the degrees of maturity in the HiAP approach from less to more advanced. Number of cities per MCA groups and per profile can be found in  This profile includes two classes (A1 n = 14 and B1 n = 5) for which the questionnaire items were partially completed. Intersectoral collaborations and joint projects, when they exist, tend to be with sectors who have traditionally cooperated with the health sector, ie, social/disability services and the early childhood service, sports and housing departments. Respondents indicate that these four sectors also have a focus on health issues and social inequalities in health. This profile includes three classes  that present a low response rate for items related to collaborations. They declare less joint projects with other sectors than the other groups of cities. We have considered that a low response rate indicates less involvement with other services. Joint projects, when they exist, are rather carried out with sectors traditionally oriented towards the health sector, ie, social/disability and the early childhood and sports departments. For class C1, they also existed with the sustainable development and green spaces sectors. Health issues seem to be dealt with at a minimum by the different sectors and issues of social inequalities in health. The two cities of class D1 are close to classes D2 and D3, but it was included in this profile as information was missing concerning the existence of environmental working groups and intersectoral bodies. This profile groups together two classes (D2 n = 7 and D3 n = 6) that are distinguished by the frequency of their positive responses indicating formal intersectoral collaboration  and also in the implementation of joint projects with all sectors, including green spaces. Unlike class D1, classes D2 and D3 also mention the existence of environmental working groups and inter-sectoral bodies, in a majority way for D3 and less so but still present (2nd most frequent response) for the D2 class. To sum up, 19 of the 58 (33%) cities have characteristics associated with a less advanced HiAP profile. In this profile, intersectoral action is rare but health and equity concerns are present. The established HiAP profile group includes 26 cities (45%). These cities are generally involved in cross-sectoral action with more traditionally health-friendly sectors, but have little involvement in environmental issues including green spaces, without any inter-sectoral bodies, with the exception the cities belonging to the D1 class. The advanced HiAP profile includes fewer cities , but these are distinguished by a strong commitment to environmental issues including green spaces which can be illustrated by health-environment bodies and specific working groups. There is no one established tool kit on how to implement health approach in all policies, nor is there a consensus on how to evaluate them. It is a complex and evolving concept that differs according to the socio-political contexts of the countries that implement them. We propose here a way of looking at the concrete implementation of HiAP based on the level of collaboration between health services and green spaces services in France.the education sector, for example the greening of schoolyards, good playground facilities, the urban development sector, ie, the greening of bicycle paths or the creation of new green spaces, or transportation, to ensure equitable access to green spaces via public transport or sustainable mobility modes .  Our study focuses on urban green spaces cooperation with the health sector policies but a similar approach could be taken to assess other policy collaboration. Finally, for the qualitative interpretation of the results for each variable to define the HiAP profiles, we did not only use variables linked to green spaces interventions. The most discriminating variables were related to the following 4 dimensions:The consideration of social inequalities in health and/or health issues in the policies/actions of the sector studied, Occasional intersectoral collaboration, ie, one-off initiatives between the health sector and others, The existence of joint projects, ie, common projects between two or more sectors, The existence of intersectoral bodies, in this case on the theme of urban green spaces including an intersectoral committee and/or working groups. The latter could also concern other sectors.  These 4 dimensions could be used as an assessment tool as they indicate the gradual progression of HiAP maturity see .23 call for the production of indicators that can be used by municipalities to measure their success in implementing HiAP. We provide four simple dimensions produced from real-life conditions which reflect actual practices. For example, these dimensions could be used as binary indicators (presence/absence) and be proposed as a simple dashboard to decision-to monitor the HiAP maturity degree in their city. In comparison with the current methodologies for assessing the HiAP approach, our methodology adds a practical tool of simple indicators that are easy to used by decision-makers. The first step would be for cities to take both health and also social inequalities of health dimensions into account . The second step, would be to actively promote intersectoral collaboration , the third step would be to implemented joint projects . The last step would be run an intersectoral body or permanent working groups . The creation of these dimensions was not planned but it contributes to Van Vliet-Brown et al24 The results of the context assessment through our in-depth qualitative study will validate the accuracy of our methodology for evaluating the HiAP approach. To test the internal validityof our methodology, we plan to compare the HiAP profiles based on the questionnaire analysis with a qualitative analysis based on in-depth interviews with elected officials and officers of a small sample of municipalities. To be discriminating by social inequalities in health, we selected cities that have a level of socio-economic inequality higher than the average for the sample of respondents. The results of the in-depth analysis are on-going and will be published soon. However, the first results ofon-site interviews seem to confirm the profiles described in this paper. This in-depth study will also help tocontextualize the results observed and provide understanding useful to better predict the transferability of the results such as the local organizational, social, historical, economic, population or cultural contexts of the municipalities as highlighted by Guglielmin et al. Concerning the external validity of our methodology, firstly, we did not use the specific questions related to green spaces in order to establish the cities profiles. The 4 dimensions could be applicable to all domains and policy sectors of the municipality who would like to assess their HiAP achievement. Another way to test this external validity would be to monitor the potential effect on population health and well-being among municipalities presenting different levels of HiAP maturity. Since HiAP impacts a large number of health determinants and health equity through different policies, we could track a series of indicators related to different key health determinants such as air quality, walkability of the municipality, cycling infrastructure, etc. Finally, this study could also be extended internationally, allowing us to test the transferability of our evaluation criteria to other contexts.41 as this issue should be a major concern of the respondents as they are part of the healthy cities movement.28 Many of the questions in our study were concerned with social inequalities in health, the answers to which are likely to include a social desirability bias, In comparison with Storm et al, the response rate was particularly high compared to the returns that FHCN normally records (84% of respondents our study vs 32% in Storm’s study). It mitigated the respondent selection bias where, generally, the most concerned individuals respond. Finally, asking health services about other services may have led to recall bias. Also very different cities responded to the questionnaire. The cities respondents had populations ranging from less than 30 000 to more than 200 000. Overall, the advanced HiAP group contained more large cities than the other groups. It can be assumed that small cities with few resources cannot set up intersectoral collaborations in the same way than a large city with several departments. The criterion of the size of the city, and also that of the way services are organised, should be explored further. In addition, our results may be less consistent for the less advanced HiAP profiles because for these profiles, some responses were missing. We assumed that they responded partially because they were not fully engaged in the HiAP approach. However, the questionnaire may have taken too long to complete. For these cities, it would be interesting to propose a short set of indicators to see if they fit the profile determined by our questionnaire. Finally, we had initially asked the green spaces departments to complete a questionnaire to establish their point of view of collaboration with the health departments and their level of awareness of social inequalities in health. Unfortunately as lack of replies meant, it could not be analysed. This article proposes a method for analyzing the degree of HiAP maturity at the local level is illustrated through considering specifically the cooperation between health and green spaces policies. It is based on a statistical classification method using MCA, enriched by a qualitative analysis. The results of this analysis allowed us to identify three main profiles among the cities studied according to their level of maturity in HiAP. Storm et al proposed a framework containing fourteen indicators aimed at taking into account social inequalities in health. Inspired by this framework, our results also enabled us to propose four dimensions which measure the degree of HiAP progress, (1) awareness of social inequalities in health and/or health issues by the sector under consideration, (2) level of collaboration between sectors, (3) existence of joint projects and finally (4) creation of effective intersectoral bodies and their outputs. Our study gives a realistic vision of the HiAP approach implemented among municipalities who are members of the French WHO Healthy city network.42 It identified three main types of intersectoral actions carried out in cities, namely intersectoral governance, intersectoral action and intersectoral policies. The indicators we propose cover similar fields. The findings are in line with the conclusions of the evaluation of phase V of the WHO European Healthy Cities Network using a realistic review. The authors acknowledge Dr. Jeanine Pommier and Linda Cambon for their support at the beginning of the study and the Réseau français des villes-santé which helped to disseminate the questionnaire and produce the results, especially Clément Bader. The GREENH-City project received prior ethics approval from Université Paris Descartes . The project has been carried out with full respect of current relevant legislation  and international conventions . Authors declare that they have no competing interests. MP wrote the article, designed the questionnaire and analysed the qualitative data, made the interpretation and drawn the results. FQ made the statistical anlaysis and drafted the paper. ARLG and MFT revised the paper. JS, NC, ZV, EF, ZH, and SR revised the paper. All the authors approved the draft. The GREENH-City projet is funded by the National institute for Cancer, France (RI 2017-03).Supplementary file 1. Questionnaire HiAP French.Click here for additional data file.Supplementary file 2. Variables Classification for Analysis.Click here for additional data file.


Alcohol overconsumption is a major cause of preventable mental disorders and death in the United States and around the world. The pathogenesis of alcohol dependence, abuse, and toxicity to the central nervous system remains incompletely understood. In vitro and cell culture-based models have been highly valuable in studying the molecular and cellular mechanisms underlying the contribution of individual CNS cell types to ethanol’s effects on the brain. However, conventional cell culture model systems carry the inherent disadvantage of rapid loss of ethanol due to evaporation following a bolus addition at the start of the treatment. We have established a multi-well cell culture plate-based ethanol evaporation compensation model that utilizes the inter-well space as a reservoir to compensate for the evaporative loss of ethanol in the cell treatment wells. Following a single bolus addition at the start of the treatment, ethanol concentration rapidly decreased over time. Through compensation using the multi-well plate platform, maintenance of ethanol concentrations ranging from 10–100 mM was achieved for up to 72 hours in a cell-free system. Greater effects in ethanol-induced decrease in neuronal cell viability were observed with than without compensation. Our method effectively compensates for the evaporative loss of ethanol typically observed in the traditional method. This method provides an economic, simple and effective in vitro model system for ethanol treatment over an extended timeframe where maintenance of a relatively constant concentration of ethanol is desired. At lower concentrations, one time and bolus addition of ethanol at the beginning of the treatment has been shown to alter a variety of cellular functions including intercellular redox state, membrane fluidity, lipid raft composition, transcellular and paracellular permeability –11. At hFor volatile, though water-soluble compounds such as ethanol, a considerable portion of the bulk of the initially added compound could be lost to evaporation, especially over an extended treatment time period to model a more chronic exposure scenario in vitro, leading to a lower than the intended amount available to the cells during treatment. Similarly, evaporative loss of ethanol during in vitro treatment might also have hindered the interrogation of effects of ethanol on cellular functions at concentrations close to the legal limit of 80 mg/dL, or 17.3 mM that are known to impair motor and/or executive functions in humans and animal studies .In this study, we characterized the evaporation of ethanol in a multi-well cell culture format after a bolus addition at the beginning of the treatment. We also determined the amounts and timing of ethanol needed to be added to the non-treatment wells to compensate for the evaporative loss for a time period up to 72 hrs. Finally, we compared the effects on cell viability of treatment of immortalized human neuronal cells with ethanol with and without compensation.2.2.12PO4 and 44 mM NaHCO3), heat-inactivated fetal bovine serum , penicillin and streptomycin (Corning 30–002-CI), trypsin/EDTA solution , ethanol , MTT  which is well below the equivalent legal limit of blood alcohol concentration (BAC) of 80 mg/dL of blood  in humans; the highest concentration of 100 mM  was 5.7 times of the equivalent legal limit of BAC. This range of ethanol concentrations is relevant to ethanol concentrations that have been used in various in vitro studies usually with lower concentrations affecting cellular functions and higher concentrations impacting cell survival, especially that of vulnerable neurons , 20–22. in vitro cell culture treatment to compensate for the evaporative loss has been highly desired and could, at the same time, be challenging [In many instances, maintaining a steady level of ethanol during llenging . Variousllenging . Rodrigullenging . Borgs ellenging . Kim et llenging . When inllenging .in vitro treatment of cells of interest. In the current study, the maintenance of 10–100 mM of ethanol at a relatively consistent level throughout the treatment periods of up to 72 hours was achieved. It is conceivable that the approach used in the current study can be readily adapted when concentrations of ethanol outside of this range is desired. Third, although the current method is based on 6-well culture plates, it does limit the growth of cells to be treated in the exact same format. For example, if the availability of cells is a limiting factor or if applications beyond cell harvest is desired, a much-reduced number of cells could be plated onto small-diameter cell culture coverslips or trans-wells that can be kept in the wells of the 6-well plates. Fourth, the utilization of the inter-well space within the confine of the same plate as the compensation reservoir makes it possible to house multiple plates of cells for treatments with different concentrations of ethanol in the same incubator. This would eliminate the need for an entire incubator to be dedicated to a single concentration of ethanol where an ethanol-containing vessel is placed inside the incubator as a compensatory reservoir. For starting and target concentrations of ethanol up to 100 mM characterized in this study, the presence of detectable amounts of ethanol was not detected in the media-containing culture wells of “blank” plates placed adjacent to the ethanol-containing plates. Furthermore, there was not a detectable difference in cell viability between the control cells (no ethanol treatment applied) in the plates kept in the same incubator as the ethanol-treated cells and the mock-treated control cells kept in a separate incubator  and vapors (humidified water and ethanol). Although no significant changes in media pH between the uncompensated and compensated plates were observed implying uncompromised gas exchanges, it would be ideal to document the local partial pressures for CO2, O2 in or above the wells of the multi-well plates during compensation to determine if the ethanol vapor from the compensation reservoir  had any impact on gaseous exchanges. However, in-line and uninterrupted (in situ) measurements of the local gaseous partial pressure, as well as pH, in and above the wells of the culture plates would require highly sophisticated instrumentation which is currently unavailable to the investigation team.The potential effect of the ethanol vapor from the inter-well space on the gas exchange over the neighboring cell culture wells in the current evaporation compensation model remains uncharacterized. Several studies reported in encased ethanol delivery systems such as tape-sealed culture plates and sealed plastic boxes, prolonged maintenance of cell cultures  has been shown to affect cell proliferation and viability , 41. TheThe SH-SY5Y neuroblastoma cells are dopaminergic neuronal cells and have been widely used cell culture model to study dopaminergic neurodegeneration , 42. Sevin vitro studies of the effects of ethanol on various cells types.In conclusion, we have devised and characterized a simple and versatile ethanol evaporation compensation model that is capable of maintaining a relative constant level of ethanol during the entire treatment period. We validated the evaporation compensation enhanced effects of ethanol on cell viability. This model system can be readily adapted for


Patients with type 1 diabetes must continually decide how much insulin to inject before each meal to maintain blood glucose levels within a healthy range. Recent research has worked on a solution for this burden, showing the potential of reinforcement learning as an emerging approach for the task of controlling blood glucose levels. In this paper, we test and evaluate several deep Q-learning algorithms for automated and personalized blood glucose regulation in an in silico type 1 diabetes patient with the goal of estimating and delivering proper insulin doses. The proposed algorithms are model-free approaches with no prior information about the patient. We used the Hovorka model with meal variation and carbohydrate counting errors to simulate the patient included in this work. Our experiments compare different deep Q-learning extensions showing promising results controlling blood glucose levels, with some of the proposed algorithms outperforming standard baseline treatment. Diagnoses of blood sugar conditions are determined by the insulin secretion from the pancreas . In thisInsulin is subcutaneously administered by the patients either through multiple daily injections or an insulin pump providing a continuous infusion . PatientThe artificial pancreas (AP) is the combination of an insulin pump, a CGM, and a control algorithm to automatically regulate BG concentrations ,10. The The requirement for an adaptive algorithm that personalizes the system for each patient is one of the major limitations of the AP . The staRL algorithms have been used before to regulate the BG levels in in silico T1D patients, showing that RL algorithms can improve BG control. Concretely, in Sun et al. , RL is uIn this work, we implement and evaluate several state-of-the-art improvements to the deep Q-learning (DQL) algorithms in the hybrid closed-loop AP to automatically regulate the BG levels in a T1D patient. We perform in silico experiments using the Hovorka model , demonstWe begin introducing RL and the T1D simulation environment in In this section, we introduce the RL framework, the deep Q-learning algorithm and its extensions, and the T1D simulator used in this work.agent and its unknown environment. This framework is shown in state of the environment and takes an action based on that state. As a consequence of this action, the environment moves to a new state and generates a positive or negative reward for the agent. The goal of the agent is to maximize the reward in the long run by taking actions that result in preferable states [RL is characterized by the interactions between a decision-making e states .s to a. policy represents the mapping from state to action and the goal of the agent is to learn an optimal policy s and thereafter following the policy A RL problem can be formulated as a Markov decision process represented by the tuple s, taking action a, and thereafter following the policy Similarly, the total amount of reward expected by the agent starting from the state The goal of RL is to find an optimal policy that is better than or equal to all other policies based on the values of the states. Actions are then taken such that the agent spends as much time as possible in valuable states. The policy is often simply a greedy search over each action in the given state, where the action that gives the highest value is chosen . In the One of the most popular methods to estimate the Q-values is the Q-learning algorithm , where t updates . Assuminxtracted .In this section, we briefly introduce the Q-learning extensions used in this work. References to the original papers are included for a full description of the algorithms.Deep Q-Learning (DQN): In DQN, the Q-value function is approximated using a neural network (NN). The input of the NN is the current state of the environment, whereas the output is the Q-value of all the possible actions the agent can take .Double Q-Learning (DDQN): This approach proposes two Q-value approximators represented by two NNs, one to estimate the target Q-values and the other one to estimate the predicted Q-values. DDQN reduces the overestimation of the Q-values, usually leading to a better performance compared to DQN .a over any other action. This method increases the stability of the optimization [Dueling DQN and DDQN: This extension changes the standard DQN and DDQN architectures presenting two separate estimators, one for the state-value function and one for the advantage function. The advantage function is defined as the difference between the Q-value function and the state-value function, mization .Prioritized Experience Replay (PR): DQN training is not efficient since transitions are randomly sampled to train the network parameters. PR gives priority measures to the transitions, sampling important transitions more frequently .Noisy DQN: Standard DQN uses a random policy resulting in an inefficient exploration, whereas noisy DQN adds parametric noise to the weights and biases for exploration .Categorical DQN: This algorithm learns a distribution of the Q-values instead of an estimation of the Q-value function, leading to a more stable and faster learning and usually outperforming standard DQN .Rainbow DQN: This algorithm integrates all the previous extensions introduced in this section, improving data efficiency and overall performance .There exist three main physiological models in in silico T1D research: the Bergman minimal model , the HovThree experiments were included in this work. The first experiment compares all the DQN algorithms introduced in State-of-the-art AP designs utilize commercially available insulin pumps and CGMs, operating in the subcutaneous tissue and introducing serious delays into the control task . The hybt is the time index, where one time step is 30 min. The time step limit is at 72 because we simulate the patient using episodes of The states t steps, i.e., every 30 min. We define The agent performs an action R is defined as a Gaussian function:We obtain a new state To measure the performance of our simulations, we use the time-in-range (TIR), which is the percentage of time the patient spends with its BG levels within the target range, defined as the healthy BG range between 70 and 180 mg/dL . This peAn individual who weighs 70 kg was used during the experiments. For each training episode, the virtual patient was given meals from a random meal generator. Fixed seed was used to ensure each agent trained on the same dataset.Breakfast:  because the agent tends to choose higher basal rate actions. As a consequence of the higher basal rates, the TBR increases (5.28%) compared to baseline (0.0%), whereas the TIR and the mean BG per episode decrease, presenting the worst results in terms of the TIR (92.82%) and the closest Regarding dueling DQN, we observe a TIR estimate approximately 2% lower than the baseline TIR, whereas the TAR has also decreased (0.45%). The TBR estimate is higher, scoring the highest TBR with a percentage of 6.25%. Similar to the DDQN algorithm, the low performance achieved by dueling DQN is the result of a set of actions with a high basal rate.The dueling DDQN algorithm shows better performance than the baseline, presenting a higher TIR (96.71%) and lower TAR (3.29%) while keeping the 0.0% TBR. The overall BG is higher per episode, as we can see from the The PR DQN agent presents the highest TAR (5.65%), scoring the worst among the different algorithms. In addition, the TIR for this agent is 94.35%, not improving the results obtained from the baseline (95.41%).The noisy DQN algorithm obtains the best results in terms of the TIR (97.04%) and BG standard deviation per episode Similar to DQN, the categorical DQN extension follows a strategy close to the baseline, obtaining results very similar to the baseline and DQN methods. This agent presents a slightly lower TIR and higher TAR, mean, and standard deviation BG than the baseline.Rainbow DQN drops BG levels due to higher basal rates, leading to the lowest episodic BG with mean BG per episode The goal of this experiment is to see what influence an increase in the action space has on the agent. By increasing the action space, one would assume that learning the right action at a certain state could be more difficult. If learned right, more actions could prove to be more efficient for the agent and the BG regulation since there are more choices in insulin amounts. Similar to experiment 1, we compare all the DQN algorithms using the same procedure and metrics. We used the same state space as in experiment 1, but the action space now has five actions in it, as described in Comparable to the results from experiment 1, the DQN algorithm performs similarly to the baseline, with no hypoglycemic events and almost no differences in terms of the TIR (95.23%) and TAR (4.77%). The mean BG per episode (124.46 mg/dL) is also close to the value obtained from baseline (124.00 mg/dL), whereas the standard deviation is slightly higher when using a DQN agent.The DDQN agent also works similarly to the experiment 1 version, in which actions with high basal rates lead to lower episodic BG levels, For the dueling DQN agent, we obtained the best TIR (The dueling DDQN method performs worse than the baseline, decreasing the TIR (The results in Noisy DQN presents a lower TIR (Similar to PR DQN and noisy DQN, the strategy learned by categorical DQN is defined by actions with a high basal rate leading to a low mean BG per episode (The results from rainbow DQN are even worse than in experiment 1, with the agent giving too high basal rates which lead to very high TBR (Experiment 2 has shown that increasing the number of actions does not necessarily improve the performance in the BG control task. The larger the action space, the more difficult the learning process, hindering the decision-making by the agent. DDQN, PR DQN, noisy DQN, categorical DQN, and rainbow DQN agents did not succeed when regulating BG concentrations better than baseline. Only the dueling DQN algorithm improved the results from the baseline, whereas DQN and dueling DDQN obtained very similar results.Experiment 3 investigates the ability of a trained agent to deal with skipped meal boluses. The agents are trained following the experimental setup from experiment 1. After training the agents, a similar comparison to that of experiments 1 and 2 was performed, in which the action variance per episode Starting with DQN, the TIR has dropped by roughly 3%, whereas the TAR (The DDQN agent’s performance is not affected by the skipped meal boluses, since the results obtained in experiment 3 are very similar to the results obtained in experiment 1. In this case, the agent presents a moderate variation in the insulin actions, with For the dueling DQN, the performance improves in terms of TBR (The dueling DDQN also shows very similar results in both experiments 1 and 3 and was the only algorithm able to reduce the mean BG per episode (For PR DQN, noisy DQN, and categorical DQN, the TIR scores have lowered by approximately 1% and the TAR have slightly increased. Similar to experiment 1, noisy DQN scored the highest TIR (Rainbow DQN seems to be unaffected by the meal disturbances at first glance. With the lowest TBR score, rainbow DQN nearly obtained the same results as experiment 1.In this work, the current state-of-the-art DQN algorithms have been tested and evaluated for the task of controlling the BG levels in a simulated T1D patient. These algorithms were compared to a baseline, where only the optimal basal rate was given to the patient. Concretely, three experiments were conducted. Goals with TIR vary from person to person and may depend on the type of medication they use, type of diabetes, diet, health, age, and risk of hypoglycemia . GeneralExperiment 2 evaluates how a larger action space affects the performance of the agents. Our experiments show that the DQN algorithms perform better with an action space with three actions, rather than an action space with five actions. The extended action space is hindering the training process of the agents, resulting in undesirable high insulin action tendencies. In this setting, DQN and dueling DDQN performed close to the baseline, whereas dueling DQN was the only algorithm able to outperform it. Compared to the smaller action space, the experiments showed more promising results, with DQN and categorical DQN performing similarly to the baseline and dueling DDQN and noisy DQN outperforming it.A final experiment was conducted, skipping meal boluses at random for an already trained agent. The goal of this test is to gain a deeper understanding of which algorithm performs best when the meal schedule is more unstable. When skipping meal boluses, we found that the overall TIR was slightly lowered whereas the TAR was slightly higher. The TBR was virtually unchanged, suggesting that this experiment led to more algorithms failing during the BG control task and so neither adapting nor generalizing to the lack of meal boluses. Only the dueling DDQN agent was able to obtain a similar TIR while reducing mean BG per episode during experiment 3, showing some robustness against skipped meal boluses.Given the current experimental setup, further experiments need to be carried out to fully validate DQN as a realistic algorithm for the AP, since the extent of these experiments is not sufficient to claim that the DQN algorithms are beneficial in a complete and general sense.In this work, we have shown that some state-of-the-art DQN algorithms outperform standard base bolus treatment in our experiments. The most competitive results were obtained by noisy DQN for experiments 1 and 3 and by dueling DQN for experiment 2. These DQN agents were able to cope with both carbohydrate counting errors and to a certain degree skipped boluses. Therefore, we consider this work a strong proof of concept for the use of DQN algorithms in the AP framework. However, most of the algorithms did not perform better than the baseline when controlling BG levels in experiment 2, indicating that there is room for improvement in both the algorithm’s implementation and the environment setup.Experimenting with different types of NN architectures might help to alleviate the training problems associated with the larger action space, leading to better learning for future work. One step further in this research direction would be to test policy gradient algorithms, allowing the use of a continuous action space instead of a discrete one. Moreover, it would also be worth it to test different state spaces; for example, use the last 24 h of BG and insulin data.Due to the fact that T1D is a well-studied disease and multiple treatment strategies already exist, there is a lot of domain knowledge that gets lost in our experiments. An obvious research direction is including domain knowledge into the RL framework for T1D; for example, through the reward function.


We contend that financial institutions were able to achieve progressively unsustainable positions due to the ‘enforced indebtedness’ of US households, which played a functional, albeit secondary, role in the development of the crisis.The causes of the 2007-8 subprime crisis continue to be the subject of much debate, with explanations ranging from de-regulation and fraudulent behavior to global imbalances and rising inequality. However, a comprehensive analysis of the endogenous forces that made the crisis inevitable has yet to be presented. This paper offers a ‘structural’ interpretation of the crisis by synthesising insights from conventional financial economics and the Minskyian and Schumpeterian literature. While highlighting the innovative character of US financial firms evolving from credit providers to  The assessment of the causes of crises in capitalist systems is one of the core topics in economic analysis. Debates about the major financial crisis from the XVIII century are still open today. The Global Financial Crisis (henceforth GFC) of 2007-8 makes no exception. In his review of twenty-one books on this crisis, Lo :151 argu1The general view seems to be that the crisis resulted from an interaction of various aspects related to regulation  and growing current account surpluses in commodity exporters countries (mainly China). The high liquidity in the global financial markets was fuelled by the substantial and lasting easing of US monetary policy and the anchoring to the US dollar of emerging markets’ exchange rate  played a central role; (ii) households’ indebtedness largely resulted from dynamic forces taking place within the financial system. The main contribution of this paper is to provide an interpretation of the Global Financial Crisis (GFC) which, albeit encompassing various elements put forward by the literature, is characterized by three fundamental aspects of novelty. First, we use a macro-finance perspective. We thus place financial dynamic forces at the centre of the stage and interpret them in relation to the macroeconomic structure of the economy, also assessing the role of different financial segments within the financial system  the fragility of the financial position of sectors within the financial system  of Minsky .8 Minskyhedge, speculative, and Ponzi. In the first case, firms expect that money inflows always exceed their financial commitments . In the second case, their revenues cover interests but are not sufficient to repay the principal and it becomes necessary to roll over the debt. Finally, Ponzi units need to borrow further money or sell assets, since expected revenues are lower than their interest payments alone. After a prolonged period of tranquillity firms’ and banks’ expectations tend to become more and more optimistic. The margins of safety narrow, and speculative and Ponzi finance become consuetudinary. The economic system endogenously moves towards an unstable financial position and then ‘something’ happens. This could be an increase in interest rate - due to tighter monetary policy or endogenous forces - or a change in expectations. At this point, speculative and Ponzi units cannot meet their financial commitments. The upward phase of the cycle comes to an end, an economic crisis is ultimately triggered, and the downward phase of the cycle starts as a Fisherian debt deflation takes place.Conversely, as steps have been taken in the application of neo-Schumpeterian insights on the financial system, Minsky’s financial interpretation of capitalism offers important intuitions for the analysis of the economy as a system in evolution Knell . NonetheIn this context, the innovative features of financial institutions are mostly reflected in their ability to circumvent regulations through liability management, finding “new ways to finance activities” Minsky, :220, thuAlthough Minsky was able to identify the emergence of a new stage of capitalism, i.e. money manager capitalism (MMC), he did not have the opportunity to observe its unfolding. The financial system has experienced a ‘technological revolution’ that characterizes a new regime of accumulation, with a ‘cluster of innovations’ that can be placed under the label ‘securitization’. The evolution experienced by financial institutions led them to transcend their traditional role as credit providers, making them ‘producers’ of financial commodities. This double function was noted by Schumpeter himself  to assess its evolution in terms of (i) sectoral composition Fig. , (iii) isubprime mortgage market.The GFC was systemic in nature, as it was linked to the evolution of the entire FS, and it transcended the domestic dimension due to the worldwide centrality of the US monetary system CGFS . HoweverFigure This metamorphosis was associated with a rapid expansion of the FS vis-à-vis the non-financial sector, as shown in Fig. insurance and pension funds guarantee schemes fell to the benefit of the rise of equity and funds shares. This is mirrored by the liability side, where more traditional liabilities, i.e. currency and deposit, and insurance and pension funds schemes, decreased their relative weight, while debt securities and equity and shares boomed. Two elements are worth stressing. First, the FS largely financed its operation by issuing securities and funds shares. Second, the similarity of the two sides of the balance sheet is striking (see the double role of equity and shares) showing how the FS was at the same time issuer and buyer of the very same kinds of assets.The transformation can be appreciated also in terms of the overall balance sheet composition Fig. . Concerninterest paid relative to the corresponding increase in interest received , the non-financial corporate sector (NFC), the household sector (HH), and the domestic financial sector (DFS). Equities are not included, and for DFS two debt measures are used. The first includes securities, loans, and repos, whilst the second adds the shares. This choice is motivated by the consideration that shares do not technically constitute a kind of liability to be serviced regularly and, as in Minsky’s FIH, we assess financial fragility by looking at income and debt services - cushion of safety . As the crisis determined a negative net income, from 2007 to 2009 debt to net income ratio falls in negative territory for all sectors.securitizing system. The traditional banking sector (private depository institutions) experienced the highest financial fragility in the 1980s, during the so-called savings and loan crisis and until the 1987 black Monday, after which its debt to GDP ratio dropped, to slowly recover only in the 2000s. While the banking sector played a pivotal role in the subprime crisis as the issuer of the loans used in the processes of securitization, other sectors within the securitizing systems carried a heavier weight in terms of financial fragility.Therefore, just like with the GDP, the ratio between debt and net income provides insight into the inner trends of the financial system. For sake of parsimony, besides private depository institutions, we show five of the twelve financial sectors other than pension funds, insurance companies, and monetary authority. The choice is based on the selection of the most important actors among those involved in the subprime mortgage crisis. The figures for mortgage pools (MP) and issuers of asset-backed securities (IABS) are extremely clear in this sense , display similar dynamics, with a steep rise in the financial fragility proxy and the debt-to-GDP ratio  and Exchange-traded funds (ETF) are shown in Figs. Finally, the trends for security brokers and dealers and real estate investment trusts).The (disaggregated) data presented above confirm a deep transformation in the financial sectors, with new actors increasing their size and influence both within the sector and in relation to the economy. We highlighted the rising indebtedness of the financial sector and its increased financial fragility in correspondence with crises. In particular, the data confirm how several actors within the financial system were involved in the securitization business. The resulting increased indebtedness in their balance sheet did not take place exclusively through an accumulation of securities and shares, but also repos and loans  by Gorton , the roorepo agreements as the main source of funds”  and funds (as money market funds).Innovation can take multiple forms Winter . In the We believe that it is revealing to look at this complex process as a ‘Fordist mass-production industrial model’ . The money spent entered the economic system and could ultimately be saved and allocated in the financial market, expanding the liability side of financial institutions’ balance sheets, while potentially feeding the demand for securitised loans. On the financial side, it created the credit relation embodying the financial raw material then transformed into a financial commodity to be used for financial production. The core sector for this stage was Private depository institutions  or ‘primary’ financial assets or liabilities such as mortgages are used to produce structured financial instruments. This type of production.“is essentially analogous to the manufacturing firm where one production department produces and supplies an output which is used directly as an input in another process. Eventually, the intermediate outputs culminate in the final economic output of the firm, i.e. earning assets. The output of the financial firm is, therefore, produced with capital, labour, material, and loanable fund inputs where loanable funds are ‘produced’ through other production operations of the financial firm.”  as inputs to produce CDOs, a structured financial product that pools and repackages cash-flow generating assets . Complex securities were made of tranches with different seniorities from various loans. Earning assets are the output of the financial production process that, as happens for traditional businesses, generates products that are more highly valued in the market than the inputs used to manufacture them. Securitization granted both liquidity and credit enhancement , on the one hand, and issuers of asset-backed securities and mortgage pools, on the other hand, had their unique type of liability, respectively shares and securities. Security brokers and dealers, just like real estate investment trusts and other financial businessessubprime crisis ought to be seen as a case of  productions increasingly financed through debt.This spectacular growth needed to be financed, and the types of liabilities issued to finance this new phase differed among sectors. While non-bank financial institutions embodied the novel ‘entrepreneur’, the traditional banking sector played the double role of credit and  commodity provider. Indeed, although most of the financing through repurchase agreements (repos) took place within the investment banking sector, commercial banks represented the main external source of funds  had a crucial role in this scheme, being both a safe source of finance and a very liquid form of short-term investment and a source of demand since MBS were used as collaterals. As mentioned, the whole financial system was involved: either in one of the steps of this financial supply chain, which started from the creation of the loans and ended with the manufacturing of complex financial products, or in the demand side  and financial producers (investment banks), together with the weakening of the limits posed by lenders’ risk, set the stage for the explosive dynamic we witnessed: “Securitization was one of the most brilliant financial innovations of the 20th century.”  the indebtedness of the former was largely due to the activities of the latter, and (ii) the financial position of the latter, usually overlooked by the literature, was equally important to the widely discussed household debt.dot.com crisis reduced the role of equities, the relative growth of the financial sector with respect to the real side of the economy did not stop until the subprime crisis  of safe assets Gorton,  implied The production of securities to match this rising demand needed an increasing amount of inputs: banks needed to issue credit. We saw how the new ‘originate to distribute’ facilitated supply, but what about the demand side? It is worth reminding that the main reason behind households’ indebtedness, especially in the US, is the purchase of a dwelling. The peculiar environment of the US market from 1996 to 2005 saw houses price rise by 45%, an increase not driven or explained by fundamental factors such as income or population growth rates Baker . Within 22In a scenario with 1) financial markets eager for securities and 2) an evolved financial system capable to manufacture securities starting from an input that it was itself generating , financial fragility grew on two parallel roads. On the one hand, households got indebted, obtaining easy credit, and fed the house price boom. On the other hand, financial firms, eager to participate in the profit-generating process of producing and selling securities, issued increasing levels of debt to finance their activities thus feeding the boom in the market for securitised loans.repo agreements to financial firms , which in turn used these funds to purchase ABS  previously assembled by the banking system itself . Financial firms obtained access to the repo using ABS held as collateral. This clarifies how banks have been able to stimulate the demand for these securities. Money entered the economy through mortgages, with limited benefit to the productive capacity of the system. The two assets created in the process were dwellings, whose construction may stimulate economic growth just in the short run, and financial assets, which are ultimately debt. What appears to be extraordinary in this period is not an exceptionally low level of real investment.The macroeconomic implications of the link between money, debt, and investment highlighted by Skidelsky , assumedpseudo-wealth, namely ‘wealth that individuals perceive they have, but which is to some extent divorced from the physical assets that exist in society’ . The second ‘run on repo’, with a 20% points increase in the haircut in just one month, occurred in September 2008, when Lehman Brothers was declared bankrupt: “In this second event, we see parallels to 19th-century banking crises, with a famine of liquidity leading to significant premia on even the safest of assets”  whose values collapsed, as in the first stages of the standard Fisherian debt-deflation story retrieved by the FIH. Metrick , which ik ibid., :425. TheWhile in the original FIH, financial fragility surged within the financing of real investment, in this updated dual version, the kind of investments involved were either dwellings or financial instruments , both having only a limited impact on the overall productive capacity. From a macro perspective, the essence of this dynamic was financially unsustainable. As someone’s financial assets are someone else’s liability, at the aggregate level, an economy investing in domestic financial assets is an economy investing in its own debt.financialized monetary theory of production. Our empirical evidence suggests a significant transformation in the US financial sector from the late 1970s, characterized by rising indebtedness and financial fragility. The evolution of US financial institutions led them to transcend their traditional role as credit providers and become ‘financial producers’ through securitization. The coincidence of this double role within the same entities made the dynamic in the credit provisioning system explosive. We use this to provide a novel explanation of the financial crisis. Our analysis aims at providing a ‘structural interpretation’ of the crisis, identifying and understanding the endogenous forces that progressively drove the US economy towards an unsustainable financial position, and made the crisis an inescapable event. The pathogens that led to the crisis were inherent in the specific innovative form that US capitalism assumed since the early 1980s. The crisis erupted in the subprime mortgage market precisely because of the specific forces driven by the new role that financial firms acquired through securitization.The debate about the causes and the mechanisms of the 2007-8 crisis is still ongoing. This paper explains the financial crisis by drawing on a financial reading of Schumpeter’s and Minsky’s contributions, in particular using an adapted version of Minsky’s financial instability hypothesis that is consistent with the key features of a money manager capitalism phase in the evolution of capitalism and was able to grasp the cornerstone role played by securitization. This innovation found its fuel within what was believed to be one of the most stable components of the economic system, namely the real estate market. We argued that the traditional ‘financialization’ and ‘money managers capitalism’ literature is insufficient to understand the unfolding of the crisis. These theories do not give a central role to financial production and see household and business indebtedness as a triggering, rather than functional, element in this story. Our interpretation of the crisis can be summarized as the description of a ‘financial demand-driven macro-dynamics’. The voracious demand for securities by the actors involved in the securitization process triggered an explosive production of structured financial products, for which the condition of chronic indebtedness of the household sector was the fundamental raw material. The unsustainable dynamic primarily sprang from the behaviour of financial firms. The trends in the household sector appear as a reverberation of more profound destabilizing dynamics that were inherent to the financial sector itself. As banks are the endogenous creator of money, financial firms became endogenous manufacturers of financial assets, with their balance-sheet management activities exemplifying a process of financial production.Minsky captured the Our empirical analysis confirms that the financial sector is characterized by continuous evolution. The relevance of these transformations for the understanding of future tendencies is embodied in the post-2010 volatility. In particular, the extent to which the phenomenal rise in the financial fragility of Exchange Traded Funds  will have an effect at the macroeconomic level remains to be seen. Our analysis should be considered as a starting point, which calls for a more detailed and robust empirical analysis.in primis, but also their reactions to the COVID-19 pandemic and the war in Ukraine, go hand in hand with the evolution of financial entities, which always try to overcome their regulatory and productive limits. This paper calls for future research to gain a deeper understanding of the active and innovative role of financial firms within developed capitalist systems.The major interventions of the Central banks after the crisis,