Moreover, the outcomes of the theory can contribute to understanding and limiting the detrimental effects of graphene grain boundaries, while also providing a benchmark for more elaborate techniques.The implementation of graphene-based electronics requires fabrication processes that are able to cover large device areas, since the exfoliation method is not compatible with industrial applications. The chemical vapor deposition of large-area graphene represents a suitable solution; however, it has an important drawback of producing polycrystalline graphene with the formation of grain boundaries, which are responsible for the limitation of the device’s performance. With these motivations, we formulate a theoretical model of a single-layer graphene grain boundary by generalizing the graphene Dirac Hamiltonian model. The model only includes the long-wavelength regime of the charge carrier transport, which provides the main contribution to the device conductance. Using symmetry-based arguments deduced from the current conservation law, we derive unconventional boundary conditions characterizing the grain boundary physics and analyze their implications on the transport properties of the system. Angle resolved quantities, such as the transmission probability, are studied within the scattering matrix approach. The conditions for the existence of preferential transmission directions are studied in relation with the grain boundary properties. The proposed theory provides a phenomenological model to study grain boundary physics within the scattering approach, and represents  Graphene (G) is a two-dimensional honeycomb lattice that is constituted by carbon atoms. Its reciprocal lattice determines a hexagonal Brillouin zone that has six corners (K/K’ points) where the low-energy part of the band’s structure is well described by a linear energy–momentum dispersion relation, defining the so-called Dirac cone. The existence of Dirac cones in the graphene band’s structure can be understood by using a tight-binding model. Consequently, the electron motion in the graphene lattice follows the Dirac equation , with thsp2 carbon structure accommodate localized states, acting as trapping centers for the charge carriers. The effects of charged line defects in graphene have been studied in the literature and are recognized as an important factor for mobility degradation in chemically vapor-deposited graphene [Moreover, the implementation of graphene-based electronics requires the chemical vapor deposition technique, which is able to cover large device areas and is compatible with industrial fabrication processes. However, an important drawback of this technique is represented by the polycrystalline nature of the samples. Polycrystalline graphene presents grain boundaries ,18,19, wgraphene ,26.From a theoretical point of view, linear defects in graphene have been studied by J.N.B. Rodrigues and coworkers in Refs. ,28,29,30In this work, we provide an alternative method and develop a continuous model of polycrystalline graphene in which general boundary conditions are derived without specifying the microscopic structure of the linear defect.With these motivations, we formulate a theoretical model of the graphene grain boundary by generalizing the graphene Dirac Hamiltonian model. A continuous approach is used that is justified within the long wavelength limit assumed throughout this work. Using symmetry-based arguments, we derive unconventional boundary conditions characterizing the grain boundary physics, and analyze their implications on the transport properties of the system. The scattering matrix approach  is used The work is organized as follows. In valleytronics [spintronics. However, valleytronics manipulations require engineered scattering centers, which are quite challenging to implement using current technologies. In view of this circumstance, we can argue that valley-mixing scattering events are quite rare in spontaneously formed grain boundaries. Moreover, to a good approximation, electrons that have different valley degrees of freedom can be described as two independent quantum fluids. The latter argument justifies the single-valley treatment that is adopted. Generalization of the ideas exposed hereafter to spinful particles having two-valley degrees of freedom is in principle immediate.In this section, we provide a first approach to the grain boundary problem in monolayer graphene, while studying the grain boundaries in multilayer graphene is beyond the purposes of this work. We deal with a theory of spinless particles described by the continuous Dirac Hamiltonian, and consider a single valley degree of freedom. Neglecting the electron spin is appropriate in the absence of magnetic fields or magnetized regions, which is the case considered in this work. On the other hand, describing graphene electrons adopting a single-valley perspective requires an appropriate justification. Indeed, in principle, the grain boundary region can be the source of valley-flipping scattering events. The existence of such kind of phenomena has inspired the emerging field of ytronics , aiming In this subsection, we describe the continuous Dirac Hamiltonian of a single graphene sheet within a rotated reference frame. The continuous approach considered in this work is appropriate for describing the system conduction properties  when the electron wavelength, which is associated with the particle momentum, is much longer than the interatomic distance of the honeycomb lattice. While the latter assumption is clearly invalidated in graphene nanoribbons devices, it is appropriate for a large variety of graphene-based devices where micrometric graphene sheets are employed as conduction channels.Let us describe the Dirac Hamiltonian of the graphene sheet GS2 in Using Equation (3) in evaluating the quantity The line Before treating the grain boundary problem, let us consider the current density conservation law originated by the rotated Hamiltonian (4). A continuity equation in the form We are now ready to treat the problem of a grain boundary. This situation is schematized using the Hamiltonian model given in Equation (6). Accordingly, the current density operator does not admit a global definition, while the following piecewise definition:An alternative and interesting approach is completely based on the algebraic properties of the matching matrix In this subsection, we study the general problem of a grain boundary formed between two graphene sheets whose reticular axes are rotated by the angles ℳ and disregarding the unknown properties of the grain boundary potential. First of all, we do observe that current density conservation takes the general form Moreover, we assume that an unknown grain boundary potential i) matching matrices belonging to the special unitary group SU(2) constituted by ii) matching matrices belonging to the special linear group SL constituted by In order to follow our program, we consider two relevant classes of matching matrices:  leads to the formal solution A direct inspection of Equation (17) allows us to identify the matching matrix Case 1:andThis case corresponds to consider an interface potential of the form:Following the procedure described before, we obtain the matching matrix:Case 2:,and(mass term)This case corresponds to consider a mass term potential of the form:The matching matrix corresponding to this interface potential takes the form:Case 3:andThis case corresponds to consider an interface potential of the form:The physical meaning of this kind of interface potential is explained in Before characterizing the scattering properties of a grain boundary, we would like to understand if a phase gradient alone can generate a non-vanishing scattering potential at the grain boundary junction. To answer this question, we need a convenient model in which the phase profile at the interface is a smooth function of the position. Thus, a regularization of the Hamiltonian model presented in Equation (11) is needed. A regularized Hamiltonian model is obtained by taking the rotation angle In particular, as anticipated in When the Hamiltonian presented in Equation (26) includes a generic interface potential of the form:The purpose of this section is to provide simple examples of the use of unconventional matching matrices, and create a link between the transmittance properties of the problem and the specific matching matrix that is adopted. A complete analysis of all of the conduction regimes of the junction  is beyond the scopes of this section. Thus, we study the scattering problem of a n/n’ grain boundary junction described by the Hamiltonian Equation (6). The peculiar properties of the transmittance are derived using matching matrices with a simplified structure compared to those described in The scattering states in the left and right part of the junction are expanded in terms of local eigenstates, taking into account the translational invariance of the scattering region along the y-direction. According to this procedure, the scattering states, which are normalized by the absolute value of the group velocity Here, Case 1: SU(2) matching matrix (Equation (13))We have solved the scattering problem using the SU(2) matching matrix given in Equation (13) by fixing The main property of Equation (32) is that er λ see a. This bCase 2: Matching matrix belonging toexample 1; We have solved the scattering problem using the Case 3: Matching matrix belonging toexample 2; We have solved the scattering problem using the diagonal The transmittance of the problem has been derived, and takes the following (energy-independent) form:A reduction of the transmittance see b is obseCase 4: Matching matrix belonging toexample 3; We have solved the scattering problem using the following A reduction of the transmittance is observed when the scattering strength parameter ined see c.The analysis of cases 1–4 shows that the grain boundary transmittance can be independent of ture see  is not aTo clarify this point, in the subsequent section, we analyze a simple model of the n/n’ grain boundary junction based on the Hamiltonian treatment proposed in We now formulate a theory of the n/n’ grain boundary junction based on the Hamiltonian model presented in In order to present the results of the model, we focus our attention on the n/n’ junction described in Since the quantity Once the scattering problem has been solved using the matching matrix given in Equation (41), the angle-resolved transmittance Before treating the general case of A direct inspection of Equation (45) evidences a dependence on d θR≅60° . In FiguThe general expression of the transmittance pertaining to the In In The general conclusion associated with the analysis performed in We have proposed a continuous model to study the grain boundary effects in graphene. The model provides a description of the grain boundary based on a Dirac Hamiltonian written in a rotated side-dependent reference frame describing crystallographic axes mismatching at a grain boundary junction. We have shown that the scattering problem related to the transmission properties of a grain boundary junction requires modified boundary conditions, which can be implemented by using the matching matrix method in the scattering problem. We have characterized the algebraic properties of all of the possible matching matrices showing that the SU(2) and Moreover, using a space-dependent unitary transformation, we have formulated a grain boundary model where usual boundary conditions can be used. Conditions to observe scattering properties related to the crystallographic axes mismatching at the grain boundary interface are studied. Reduction of the junction conductance has been demonstrated as the effect of the misorientation angle between the two junction sides.Compared to alternative theoretical methods ,28,29,30per se an important enrichment of the scattering theory of graphene, having the potential to stimulate new experiments and further theoretical investigations. The proposed theory provides a phenomenological model to study grain boundary physics within the scattering approach, and represents


Heart failure with preserved ejection fraction (HFpEF) accounts for approximately half of the current burden of HF, and the prevalence is continuing to rise. In contrast to HF with reduced ejection fraction (HFrEF) there are no clinically effective evidence based therapies for HFpEF. The principal pathophysiologic disorder is an elevation of left atrial pressure, most notable during physical activity, which results from impaired left ventricular diastolic reserve, and increased left atrial stiffness. This review outlines the clinical development of a potential device based therapy for HFpEF, the interatrial shunt device (IASD). Heart failure (HF) is one of the most prevalent cardiovascular disorders and recent data highlight a continuing rise in the number of people living with HF, most notably in the context of an aging population . In partAs outlined in Various mechanisms contribute to the disproportionate, exercise mediated rise in LA pressure in HFpEF including increased left ventricular and LA stiffness together with impaired active relaxation , 29, 30.In the absence of effective drug therapies for HFpEF, a device-based approach to reduce LA pressure, especially in the context of the exertional rise, has been developed. This principle was based upon the observation that patients the combination of mitral stenosis and a congenital atrial septal defect (Lutembacher's syndrome) are less symptomatic than patients with isolated mitral stenosis of similar severity due to LA pressure offloading. Based upon invasive hemodynamic data obtained from patients with symptomatic HF and an LVEF ≥ 40%, computer modeling studies were conducted to determine the optimal size of an interatrial shunt that would result in a lower peak LA pressure, whilst avoiding excessive left to right shunting . As showBased on these computations an interatrial shunt device  was developed to be positioned within the atrial septum and to provide continuous left to right shunting through an 8 mm central lumen . The impFollowing an initial pilot study of the IASD which confirmed safety together with evidence of improved hemodynamics  a multi-These encouraging data have prompted the commencement of a pivotal multi-center randomized clinical trial  of 608 patients, with hemodynamically confirmed HFpEF, randomized to IASD implant, or sham procedure and followed for 5 years. Cross over will be allowed at 24 months. The primary endpoint of the study is a composite of the incidence of cardiovascular mortality or ischemic stroke, the rate of heart failure admissions or healthcare facility visits for IV diuresis, and a change in the KCCQ total summary score. In this context, it is relevant to consider the nature of overall and cardiovascular mortality in patients with HFpEF. Epidemiologic studies highlight the complex, multimorbid nature of HFpEF patients in which mortality relates predominantly to cardiovascular events, however with a significant contribution from non-cardiovascular events .An alternate device, the V-Wave System  has also been trialed in HF. The device is an hourglass shaped self-expanding prosthesis on a nitinol frame, with a polytetrafluoroethylene (PTFE) skirt extending across both sides of the septum. The central lumen is smaller than the Corvia device (5.1 mm), and includes a porcine trileaflet valve on the right atrial side designed to prevent right-to-left flow. All patients were anticoagulated for 3 months post implantation. Thirty-eight patients were enrolled in a single arm feasibility study, with 100% initial implant success. All forms of heart failure were included, with 8 (21%) having an ejection fraction over 40%. Half of the devices were either occluded or stenotic by 12 months, with some evidence of improved hemodynamics in those with patent devices. Further randomized data in the HFrEF and HFpEF population are required following further device modification to improve shunt patency.post-hoc analysis in this study also demonstrated that patients who had a particularly favorable hemodynamic response to IASD implantation had lower rates of HF hospitalization. The potential for IASD implantation to impact on survival is consistent with the study by Dorfs et al. who showed that a higher workload correct exercise PCWP was associated with poorer survival (The precise nature and type of cardiovascular mortality in HFpEF is not well-characterized, given the limitations of available datasets, however it appears to result from a combination of heart failure, MI, sudden death, and stroke. The potential effect of IASD implantation on HF outcomes has been investigated recently by modeling the predicted outcome of the REDUCE LAP-HF cohort in the absence of the IASD using the MAGGIC prognostic model , 42, andsurvival .Given the complex nature of HFpEF and the invasive nature of IASD implantation, as compared to pharmacotherapy, it is appropriate to consider those patients who might benefit most from the procedure. Given that the IASD implant functions to reduce LA pressure, attempts have been made to better understand the factors that contribute most to LA pressure particularly during exercise. Wessler et al. demonstrated that the magnitude of the reduction in exercise PCWP is influenced by the baseline pressure gradient between left and right atria . As suchTaken together, the management of HFpEF patients continues to be a major, unresolved clinical challenge. Despite the complexity of the pathophysiological pathways leading to the development of HFpEF, the fundamental final consequence is an elevation in left atrial pressure. The IASD was developed to specifically address this physiologic target and early studies have shown the device to be safe and with promising evidence of efficacy. A pivotal randomized controlled study, REDUCE LAP-HF II is currently underway to definitely determine the clinical utility of this procedure.All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Claustrophobia is an anxiety disorder characterized by the fear of enclosed spaces. Although medication treatment can effectively control symptoms, the effects quickly disappear once medication is discontinued. Many studies have shown that combining psychotherapy and medication is more efficacious than solely using medication. However, the weaknesses of the traditional psychotherapy are that it is time-consuming and expensive. Alternatively, vivo exposure therapy is proposed in which anxiety is gradually triggered with stimuli. Targeting claustrophobia is diagnosed using the traditional method, and this study established virtual reality (VR) and augmented reality (AR) environments consistent with claustrophobic characteristics, comparing the two using an experimental process to examine whether VR and AR environments are equally capable of triggering anxiety in participants. This study further analysed the efficacies of VR and AR by measuring changes in participant's heart rates variability (HRV) and examining data from survey questionnaires. HRV results indicated that the proposed VR system and AR system were both able to trigger anxiety. Furthermore, the AR environment produced a stronger experience for the participants and caused physiological reactions more evident than those caused by the VR environment. Regarding the anxiety questionnaire, the participants suggested that their anxiety was significantly higher in the VR environment than in the AR environment. Claustrophobia is an anxiety disorder characterized by the fear of enclosed spaces . Under cCognitive therapy is a commonly accepted treatment for anxiety disorders . The goaVirtual reality (VR) originated from Sutherland's (1965) concept of “The Ultimate Display” , which uVR applications enjoy the advantages of simulating environments that are difficult or impossible to find in everyday life at a low cost, and these environments can be customized based on requirements to meet specific demands. It was difficult for early VR technologies to enable immersion because their graphics were unrealistic. However, years of development have led to constant innovation and improvement in both software and hardware, providing stable and reliable environments for a variety of entertainment, simulation, and training purposes. Many studies of phobias now involve experiments performed in VR , 20. InsAugmented reality (AR) is an approach that integrates virtual objects with the scene of real world that enables the user to perceive an augmented world, as defined by Azuma  and MilgThe purpose of this study was to compare established VR and AR environments to determine whether the environments are equally capable of triggering anxiety. The triggering characteristics in VR and AR environments considered claustrophobia and some other anxiety disorder-related diseases, such as agoraphobia. Outcome measurements included heart rate variability and state/trait survey results. Yeh et al.  had prelThis experiment compared the degree of anxiety stimulated by VR and AR environments and the degree of realism experienced by participants. An enclosed elevator was used as the simulated environment because of its relevance to everyday life and relatively high likelihood of triggering claustrophobia. In addition, an interactive virtual skyscraper elevator system was constructed using VR and AR technologies, integrating visual and auditory stimuli to trigger anxiety in the participants. The 3D virtual environment created for the proposed system was developed using the Unity 3D game engine with Windows 7 Enterprise Edition. To increase the degree of interaction between the participants and the virtual environment, the system used a Z800 3D Visor Head-Mounted Display (HMD) for the output, and an embedded posture recognition device was used in conjunction with the official software development kit (SDK) to convert users' head movements into mouse motions. Furthermore, an MSI MyECG E3-80 First Professional-Grade Portable Electrocardiogram (ECG) was employed to measure and extract the changes in the heart rates of participants during the experiment for later analysis.For the VR system, all visuals were composed of virtual scenery. The 3D modules used therein were developed using 3DMAX software in conjunction with texture mapping. The modules were incorporated into Unity 3D for use after development. The system framework is shown in To enable adjustments in the stimulus level based on the viewer's circumstances, the following events were designed: (1) elevator door closing: when the participants enter the virtual environment, they are located in an open opaque elevator; the elevator doors suddenly close after a certain amount of time. Visually, the entire environment switches from an open to an enclosed environment; the participants are unable to see the environment outside of the elevator, generating a sense of constriction and the sound of the elevator door closing heightens the presence of the scenario. (2) Brightness level: after the elevator doors have been closed for some time, the lights inside the elevator begin to flash and then eventually turn off. Consequently, the elevator enters a state of blackout. Visually, the lights switch from light to dark, preventing the participants from seeing the objects around them and inducing a level of psychological stress. (3) Alarm sound: from an auditory perspective, fire alarms, evacuation alerts, and impact noises begin to sound, when the elevator lights begin to flash and when the elevator enters blackout, causing the participants to begin doubting the circumstances outside the elevator and eliciting fear. (4) Heartbeats sound: after the elevator enters blackout, participants cannot see any visuals for a certain period of time. A series of faint heartbeats start sounding in the darkness and, coupled with the quiet surroundings, make it difficult for participants to distinguish whether the heartbeats are sound effects or the sound of their own heartbeating. (5) Flames and heavy smoke: after participants gradually become acclimated to the surrounding darkness, flames suddenly burst out . VisuallThis experiment focused on examining whether VR and AR environments elicit fear of enclosed spaces and comparing the effectiveness of these two methods as a model of claustrophobia treatment. We used heart rate variability (HRV) as an objective measure of participants' physiological status and survey questionnaires to examine the participants' experiences.We recruited 30 participants ranging in age from 18 to 35. These participants had no medical history in regard to claustrophobia or other types of fears. The order of environment conditions was counterbalanced to reduce order effect errors. The experiment took approximately two hours to complete both conditions. The participants were first fitted with the HRV physiological data measuring the instrument to collect their normal HRV for 1 h, after which the system timestamps were synchronized for the MyECG instrument and the computer was used for the experiment. The VR/AR environmental conditions lasted 5 min each. The physiological data regarding the HRV of the participants were collected continuously. The participants were given a 10 min rest between conditions. After both conditions were completed, they were asked to complete a survey in regard with technology acceptance . The SD In this experiment, both the VR and AR environments were comprised of an enclosed elevator. In the VR experiment, the participants were unable to move their body after entering the environment but were allowed to rotate their heads to change their viewing angles. In the AR environment condition, to be consistent with the circumstances of the VR environment, the participants were asked to stand in the center of the elevator and to avoid making movements besides rotating their heads to view their surroundings. Baseline time was recorded as the time from the start of the experiment to the start of the first anxiety-inducing event. An equivalent baseline time was in both conditions. Heart rate variability (HRV) was analysed using ECGs. The analyses were divided into time domain and frequency domain. Time-domain analysis  uses ECGTo assess whether anxiety was triggered for the participants in the VR and AR environments, a questionnaire was used to measure their degree of anxiety. This questionnaire was based on the State-Trait Anxiety Inventory and modified to utilize a 5-point scale for measuring the degree of the anxiety experienced. This scale is typically used to measure anxiety in adults. The questionnaires were divided into the VR and AR environment sections, and the participants were asked about the degree of anxiety they experienced in the VR and AR experiments. More specifically, items in questionnaires were in regard to each stimulus event  associatIn addition, the technology acceptance model (TAM)  was usedt-test and performed a paired sample analysis and an independent analysis. The statistical analysis was conducted by the tool of SPSS™.We measured the correlation of the collected survey questionnaires using a For HRV, we divided the measured data into three main parts: data from 1 h before the experiment and data obtained during the VR and AR experiments. To compare the effectiveness of the VR and AR environments in triggering fear, we compared and analysed the normal HRV physiological data with those obtained from the VR and AR experiments. The results are shown in Tables A comparison of the participants' HRV under normal circumstances versus HRV in the two conditions can be seen in In a comparison of the means and standard deviations of the various values , the AR t-test and performed a paired sample analysis and an independent analysis . In contrast, the AR conditions required integration with their surroundings, requiring consideration of equipment for sound and light effects, as well as an emphasis on maneuverability. In addition, regarding AR conditions, participants must be physically present in the environment, leading to differences in presence when compared to VR conditions. Although the two conditions utilized the same equipment and stimuli, other factors may influence the experience of users, such as changes in temperature, standing posture, and external noises beyond the control of the researchers. Assessing the presence of AR environments may require a different approach than that for VR environments to allow participants to adequately evaluate their experiences.In summary, the researchers found that, in this experiment, the AR environment produced a statistically significantly stronger experience for the participants and caused statistically significant physiological reactions than those caused by the VR environment. However, in clinical therapy for claustrophobia, AR environment experiments are more difficult to construct than are VR environments. Furthermore, patients have a lower degree of acceptance during exposure therapy. Therefore, in conjunction with developing AR-based therapy, improving VR display equipment to provide greater presence may help induce the anxiety associated with enclosed spaces and therefore an opportunity to provide an intervention for claustrophobia.This study successfully developed virtual reality (VR) and augmented reality (AR) environments for claustrophobia treatments. A test was conducted to validate these two systems and examine the effect between these two systems using HRV and anxiety questionnaires. HRV results indicated that the proposed VR system and AR system were both able to trigger anxiety. Furthermore, the AR environment produced a stronger experience for the participants and caused statistically significant physiological reactions than those caused by the VR environment. Regarding the anxiety questionnaire, the participants suggested that their anxiety was significantly higher in the VR environment than those in the AR environment. In the future, a large-scale clinical test is planned to further verify the therapeutic effect of the proposed systems.


Advances in single-cell RNA-sequencing techniques reveal the existence of distinct cell subpopulations. Identification of transcription factors (TFs) that define the identity of these subpopulations poses a challenge. Here, we postulate that identity depends on background subpopulations, and is determined by a synergistic core combination of TFs mainly uniquely expressed in each subpopulation, but also TFs more broadly expressed across background subpopulations. Building on this view, we develop a new computational method for determining such synergistic identity cores of subpopulations within a given cell population. Our method utilizes an information-theoretic measure for quantifying transcriptional synergy, and implements a novel algorithm for searching for optimal synergistic cores. It requires only single-cell RNA-seq data as input, and does not rely on any prior knowledge of candidate genes or gene regulatory networks. Hence, it can be directly applied to any cellular systems, including those containing novel subpopulations. The method is capable of recapitulating known experimentally validated identity TFs in eight published single-cell RNA-seq datasets. Furthermore, some of these identity TFs are known to trigger cell conversions between subpopulations. Thus, this methodology can help design strategies for cell conversion within a cell population, guiding experimentalists in the field of stem cell research and regenerative medicine. Advances in high-throughput single-cell RNA-sequencing (RNA-seq) technologies make it possible to profile RNA expression in thousands of individual cells and provide unprecedented resolution with which biological systems can be investigated. In particular, single-cell RNA-seq data for populations of cells reveal the existence, and enable the characterization of, distinct subpopulations of cells. In this way, single-cell RNA-seq data provide a more fine-grained, detailed information on populations of cells compared to the bulk transcriptomics data available for the same populations. Moreover, it has been reported that the expression of only a handful of specific TFs is enough to maintain the identity of a cell subpopulation . KnowledExisting methods presented in the literature for the identification of TFs for conversion of cells from one type to another  depend eIt is worth noticing that cell subpopulation identity TFs depend on the level of cell specialization when comparing different subpopulations within the population . For exasynergistic identity core. Experimental evidence in various systems shows that several TFs are necessary to operate together in order to maintain a cellular phenotype  present in a considered cell population. The context (i.e. background subpopulations) is important to consider, as depending on, for example, whether the identity core TFs for a cell type or for a cell subtype are sought, the result may be different. Moreover, the background subpopulations enable us to define which TFs are specifically expressed in the target subpopulation and which are more broadly expressed in the entire population. Our method uses only single-cell RNA-seq data, and does not require additional prior knowledge or inference of gene regulatory networks. In particular, the latter constitutes a desirable feature, as inferred gene regulatory networks are incomplete and often fail to capture multiple direct and indirect gene interactions.The term ‘By applying our method to a collection of datasets available in literature, we show that our computational approach can correctly recapitulate experimentally validated known identity TFs among those forming the predicted synergistic identity cores. Moreover, by considering examples of TF-mediated cell conversion experiments reported in literature, we show that our method can identify core TFs whose perturbations have been shown to convert one cell type into another, including mouse embryonic fibroblasts into oligodendrocytes, embryonic and postnatal mouse fibroblasts into astrocytes, and mouse embryonic fibroblasts into haematopoietic progenitors. Importantly, we show that the detected synergistic identity cores cannot be obtained by simply considering pairwise mutual information between the constituent TFs. This result indicates that indeed the identity TFs need to be considered in combination as a whole in order to capture the emergent property of synergy arising from multiple interactions among them.In conclusion, we argue that the knowledge of the synergistic identity core that is responsible for maintaining the identity of a given target subpopulation with respect to background subpopulations has important implications for cell conversion and regenerative medicine. In particular, due to the synergistic character of identity cores, perturbation of only a few key TFs should be sufficient to trigger a cell conversion from a target subpopulation to any of the background subpopulations. In this way, our computational method can potentially be used to improve efficiency and fidelity of cell conversion.Comparison of gene expression profiles of cell subpopulations enables the identification of target subpopulation transcriptional cores formed by a specific combination of TFs. We postulate that the target subpopulation transcriptional core is primarily composed of a synergistic combination of TFs that are uniquely expressed in the target subpopulation but not in background subpopulations. In addition, other TFs, which are also expressed in background subpopulations and contribute to the overall synergy, can be part of the core. Thus, the target subpopulation transcriptional core is characterized by a unique synergistic combination of those TFs.Our computational method for the identification of target subpopulation transcriptional cores proceeds as follows. In the first stage (Stage I), candidate TFs for identity TFs are prefiltered: TFs that both have the highest expression ratio in the target subpopulation and are specifically expressed in the target subpopulation but not in the background are selected. In the second stage (Stage II), all subsets of the target subpopulation-specific TFs of sizes ranging from three to five are considered and the most synergistic subsets are identified. Finally, in the third stage (Stage III), in order to account for the possibility that the identity core can consist also of TFs that are non-specifically expressed in the target subpopulations, the set of TFs found in Stage II is extended by all possible combinations of size three among such TFs. The most synergistic extended set is reported as the synergistic identity core for the target subpopulation.The particular choice of the sizes of subsets considered by our method is made to facilitate exhaustive computations for all subsets and by the fact that the transcriptional identity core is formed by a relatively limited number of TFs . Moreoveexpression ratio is computed . We denote this value quantile function of package stats version 3.5.1 of R with type = 3 and probs = 0.8 applied to the expression ratios of all TFs).In order to determine the set of TFs that have the highest expression ratio in the target subpopulation, the following procedure is employed. We denote the set of all TFs present in the considered single-cell RNA-seq dataset as The set of TFs that are specific to the target subpopulation is determined in the following way. Let Once the JSD values are calculated for normalized expression vectors for all Since this JSD value measures the closeness of the observed gene expression profile to the ideal gene expression profile, the lower the value, the higher the specificity of the expression of roach by , we rankIn addition, TFs that are non-specifically expressed in the target subpopulation and have the top 50 highest expression ratio values are selected from The top 50 is set, as above this value, the exhaustive computation of MMI for all subsets of three TFs (see Stage III) becomes too long or crashes due to the out-of-memory error.The aim with the JSD operation is to identify TFs that both have the highest expression ratio in the target subpopulation and are target subpopulation-specific. For the latter, as described above, we first consider the JSD values of normalized expression vectors. This idea is based on the approach proposed in  for idenroach in , we do nGiven the set of candidate TFs specifically expressed in the target subpopulation, rmation)  to a setariables ,12.10-transformed, where zero expression values of a TF are replaced with ‘1’ prior to the transformation. After the transformation, the expression values for each TF in the target subpopulation are discretized by binning the values into a number of bins determined by applying the Freedman-Diaconis rule implemented in the nclass.FD function of the version 3.5.1 R package to the log10-transformed expression values of the TF across the entire population. Note, due to the difference in the implementation of the nclass.FD function, R package version ≥3.5 is required. The bin size is set to the value returned by the nclass.FD function +1. Since the number of bins varies with TFs, to facilitate fair comparisons, Shannon's entropies are normalized by the theoretical maximum Shannon's entropy of the respective bin size asEach TF is considered as a random variable whose realizations are its expression values in the individual As MMI is a measure that quantifies synergy among The search algorithm consists of three steps. First, all subsets of The core returned by Stage II is obtained by considering TFs only in The statistical enrichment of identity TFs with experimental evidence (known identity TFs) is computed by the hypergeometric testSingle-cell RNA-seq data used in this study are obtained for the following biological systems; the mouse datasets for lung, striatum, cortex and hippocampus, intestine organoids, hair follicles and gastrulating embryo, and the human dataset for pancreas. The reference to each dataset is described in We postulate that the identity of a cell subpopulation is relative, and is determined by the background subpopulations, and therefore our computational method discerns a synergistic identity core of the given target subpopulation with respect to background subpopulations. Furthermore, we propose that a synergistic identity core of a subpopulation is mainly composed of TFs that are specifically expressed in a target subpopulation, but also of TFs that are not specifically but more broadly expressed in background subpopulations which contribute to the overall synergy of the identity core. Notably, by considering combinations of TFs rather than individual TFs, our method searches for an identity core composed of TFs that possess a synergistic co-expression pattern, which reflects direct and indirect tight regulatory interactions between the TFs. In this way, the approach aims to take into account synergistic transcriptional regulation, including protein-protein interactions between transcriptional co-factors and cooperative binding of TFs to promoter/enhancer regions of target genes .As input, the proposed method only requires single-cell RNA-seq data of the target subpopulation and a number of background subpopulations. By the fact that the method considers synergistic combinations of TFs rather than individual TFs, it is capable of capturing the core identity TFs of the target subpopulation even in the cases where background subpopulations are closely related to the target one without any additional prior knowledge or gene regulatory network inference. The overview of the method is presented in Figure To assess the performance of our method, we calculated the percentage of target subpopulations for which at least one of the predicted core TFs has been experimentally validated to define the identity of the target subpopulation. We applied our method to eight published mouse and human single-cell RNA-seq datasets, consisting of 88 subpopulations , for whiP-value equal to 2.617e–11, which is statistically significant and unlikely to become insignificant due to small variations in the composition of the synergistic identity cores. Hence, we conclude that subpopulation identity tends to be maintained by TFs that exhibit high transcriptional synergy and the computation of synergistic cores can selectively further reduce the candidate TF space from the We also evaluated the effectiveness of transcriptional synergy in selecting known identity TFs from target subpopulation-specifc TFs . For each maximum core size ranging from four to seven, synergistic identity cores were identified and the statistical enrichment for known identity TFs was computed, as described above. The Next, we evaluated whether pairwise MMI  was capable of revealing all the interactions between the TFs constituting the synergistic identity cores reported by our method. The result showed that the majority of pairs of core TFs did not appear among the top pairs with the highest mutual information values . In othecell-type-specificity score, the value of JSD applied to an observed expression vector consisting of normalized bulk expression measurements of a TF in the target and background subpopulations and the ideal vector having 1 for the target subpopulation and 0’s everywhere else. As candidate identity TFs, the method simply reports the top ten TFs ranked by the cell-type-specificity score. Since in the original study the method is applied to bulk microarray gene expression data, we averaged single-cell RNA-seq data over each cell subpopulation prior to applying this method. It predicted at least one experimentally validated identity TF in 35% of the target subpopulations considered in this study . Thus, non-specific TFs had a high tendency to be involved in interactions with known identity TFs, possibly giving phenotype specificity to subpopulations.Next, we specifically investigated the contributions of non-specific TFs to synergistic identity cores. Results revealed that in 36 subpopulations non-specific TFs contained at least one known identity TF, showing that these non-specific TFs also contribute to the synergistic identity core. Furthermore, extracting from MetaCore  literatuFor example, the synergistic identity core of the striatal oligodendrocyte contained Sox10 (specific) and Nfia (non-specific) and the cross-inhibition between these TFs has been reported to be important for oligodendrycyte specification . Mafb s. In addiIn summary, the above results indicate that our method discerns synergistic identity cores that contain TFs that are known to maintain cell type/subpopulation identities and they capture known functional, potentially synergistic interactions between identity TFs.There is evidence in literature for TFs that can convert one type of cells into another. These cell conversions usually do not occur during organism development, rather these TFs establish new cell identity upon artificial overexpression in the starting cells. For example, in the case of reprogramming mouse fibroblasts into astrocytes a synergistic effect between three TFs, Sox9, Nfia and Nfib, was observed . ThereinThe three TFs identified in the synergistic core of the blood progenitor cells, Gata2, Tal1 and Fli1, have been shown to be able to reprogramme mouse embryonic fibroblasts into haematopoietic colonies . Indeed,in vivo . Commonly used network-based computational methods for determination of cell type-specific transcriptional cores rely on the construction of networks from pairwise correlated genes . Other eOur method takes as input only single-cell RNA-seq data, and is a network-free approach, which is a desired property considering the limitations of network reconstruction, such as incomplete prior knowledge and failure in distinguishing between direct and indirect gene interactions. We compiled a dataset comprised of different publicly available examples of cell populations, including in total 88 target subpopulations. By applying our method to this dataset, we showed that our approach was able to recapitulate experimentally validated identity TFs in majority of the subpopulations. These synergistic identity cores were enriched with not only known identity TFs, but also with known interactions that involve known identity TFs. Interestingly, pairwise mutual information between known identity TFs in synergistic identity cores were not always high, indicating that these TFs should be considered in combination to capture the emergent property of synergy, which arises from multiple interactions among them.Further analysis showed that known cell conversion factors were present in synergistic identity cores identified by our method. Hence, we argue that identification of synergistic identity cores not only provides characterization of the target cell subpopulation identity, but can also facilitate the design of novel strategies for cell conversion between cell subpopulations. In particular, this methodology can be useful in identifying TFs capable of triggering cell conversions between novel subpopulations, including closely related ones that exhibit subtle differences in gene expression profiles. One limitation of the current implementation of the method is that the maximum size of the target subpopulation-specific part of synergistic identity cores is fixed to five. However, we observed that for five of the 88 subpopulations, there were more than five known identity TFs and not all of them can be identified in the synergistic identity cores if the maximum core size is set to five. Therefore, although we showed that this size is the optimal in terms of the enrichment of known identity TFs, the development of an algorithm for dynamically adjusting the maximum core size could improve the recall of the method.Future extension of the current approach can be devised to select optimal combinations of core TFs, whose upregulation could increase cell conversion efficiency and fidelity. It would also be interesting to experimentally validate synergistic interactions among TFs in identified synergistic identity cores. Thus, our method can become a useful computational framework to guide experimental work in stem cell research and regenerative medicine.In this study, we postulated that identity of a cell subpopulation is relative, and is determined by the background subpopulations. Moreover, identity is determined by a unique combination of TFs that are mainly specifically expressed in a target subpopulation, but also TFs that are not specifically but more broadly expressed in background subpopulations, which together form a synergistic core at the transcriptional level. Based on this notion, we developed a novel computational method for the determination of synergistic identity cores of target subpopulations in a considered cell population. Application of the method to a large number of cell subpopulations demonstrated that it was able to recapitulate experimentally validated identity TFs in majority of the subpopulations. Importantly, our method is a network-free approach and does not rely on prior knowledge, and therefore it can be directly applied to any cellular system, including novel cell subpopulations for which single-cell RNA-seq data is available. Hence, this methodology can be useful for designing novel strategies for cell conversion, and in turn be a guidance to experimentalists in the field of stem cell research and regenerative medicine.https://git-r3lab.uni.lu/satoshi.okawa/synergisticcore.The R and C++ scripts used to perform the computation of the synergistic identity cores are available at Supplementary DataClick here for additional data file.


This paper presents an adaptation of the flying ant colony optimization (FACO) algorithm to solve the traveling salesman problem (TSP). This new modification is called dynamic flying ant colony optimization (DFACO). FACO was originally proposed to solve the quality of service (QoS)-aware web service selection problem. Many researchers have addressed the TSP, but most solutions could not avoid the stagnation problem. In FACO, a flying ant deposits a pheromone by injecting it from a distance; therefore, not only the nodes on the path but also the neighboring nodes receive the pheromone. The amount of pheromone a neighboring node receives is inversely proportional to the distance between it and the node on the path. In this work, we modified the FACO algorithm to make it suitable for TSP in several ways. For example, the number of neighboring nodes that received pheromones varied depending on the quality of the solution compared to the rest of the solutions. This helped to balance the exploration and exploitation strategies. We also embedded the 3-Opt algorithm to improve the solution by mitigating the effect of the stagnation problem. Moreover, the colony contained a combination of regular and flying ants. These modifications aim to help the DFACO algorithm obtain better solutions in less processing time and avoid getting stuck in local minima. This work compared DFACO with (1) ACO and five different methods using 24 TSP datasets and (2) parallel ACO (PACO)-3Opt using 22 TSP datasets. The empirical results showed that DFACO achieved the best results compared with ACO and the five different methods for most of the datasets (23 out of 24) in terms of the quality of the solutions. Further, it achieved better results compared with PACO-3Opt for most of the datasets (20 out of 21) in terms of solution quality and execution time. The traveling salesman problem (TSP)  involvesThe TSP problem can be used for modeling several wireless sensor network (WSN) problems . In a WSMany methods, including heuristic or hybrid, have been proposed for solving the TSP, but most of them were unable to avoid the stagnation problem, or they may have obtained good solutions but took a long execution time to do so . In thisThe main contributions of this work on the flying ACO (FACO) algorithm are as follows: (1) proposing a dynamic neighboring selection mechanism to balance between exploration and exploitation, (2) reducing the execution time of FACO by making flying ants equal to half the ants, and (3) adapting the flying process to work with the TSP problem.The main contributions of this work in general are as follows: (1) obtaining better-quality solutions, (2) significantly reducing the execution time, and (3) avoiding getting stuck at a local minimum.The paper is organized as follows: in This section reviews some important and more recent works in this area.The TSP has been widely used as a benchmark problem to evaluate many metaheuristic and nature-inspired algorithms. Chen and Chien  presenteThe 3-Opt algorithm is widely used to enhance the local search of metaheuristic algorithms. Mahi et al.  presenteOuaarab et al.  proposedMany works in the literature present improvements on existing algorithms by suggesting methods to control the tradeoff between exploitation and exploration, which are the two main steps that form the basis of many metaheuristic approaches and nature-inspired algorithms. In exploitation, the accumulated knowledge about the search space is used to guide the search, while in exploration, risk is taken to explore the unfamiliar region of the search space, in the hope that this region may contain a solution better than the known solutions .Researchers usually use hybrid methods to merge different algorithms’ capabilities; however, new methods can become too complex and sometimes even incomprehensible. By contrast, we aimed in this work to enhance the ACO algorithm by adding extra procedures while keeping the method understandable and easy to use.The 3-Opt algorithm was introduced to solve the TSP. It exchanges three edges from the old tour by another three edges to produce a new tour  and retaThere are n cities . For insn cities .The experiments reported in ,28 show ACO was inspired by the way real ants forage for food. Initially, real ants forage for food randomly, depositing a chemical substances called pheromones on their paths. The path between the colony and the nearest food source tends to receive more pheromones, which attracts more ants to follow the same path. Once the food source is exhausted, the ants abandon the path and the pheromones evaporate, forcing the ants to start searching for another food source randomly.The ACO algorithm ,29,30 sii to α and β are coefficient parameters that determine the importance of the pheromone value . This update is called the local pheromone update:ACO selects the best solution greedily. The best ant updates the pheromone trails on its path using Equation (3). This process is called the global pheromone update:gbL(t) is the tour length for the best ant.These ACO algorithm processes are illustrated in In , a flyinThe algorithm assumes that, in addition to walking normal ants, there are also flying ants. Flying ants inject their pheromones from a distance, so that not only the nodes on the path receive some pheromones but also their neighboring nodes. The amount of pheromone a neighboring node receives is inversely proportional to the distance between it and the node on the path. This makes the ants more likely to explore these nodes during future iterations, which encourages exploration.C, RT, A, and R are the cost, response time, availability, and reliability, respectively; x is the best web service obtained by the best ant in task i+t1; y is one of the neighboring web services to x in task i+t1; and x ≠ y.Since determining the nearest nodes might be an expensive iteration in terms of execution time, we only considered the ant that finds the best solution as a flying ant in each iteration. The rest of the ants are dealt with in the usual way. The flying ant then determines the nearest neighboring nodes (web services) by using Equation (5) to calculate the distance between two web services NS is the number of neighbors. i + 1 and its neighbor web service l in the same task.The nearest neighbors receive an amount of pheromone that is inversely proportional to their distance from the best node. Equation (6) describes how much pheromone a node receives:The distance  Many methods, including heuristic or hybrid, have been proposed for solving the TSP, but most of them cannot avoid the stagnation problem, or they may obtain solutions but take a long execution time . In thisIn this section, we present a modification of the FACO algorithm to make it suitable for the TSP problem in the following ways.First, the number of neighboring nodes in FACO were static based on the experiments. However, the number of neighbors in DFACO that may be injected with pheromones was dynamic. The number of neighboring nodes varied in each iteration depending on the quality of the best solution reached so far compared to the other solutions. The number of neighbors was determined based on two cases: (1) If the best solution is slightly better than the other solutions, then the number of neighbors should be large to obtain more neighbors. This encourages exploration in future iterations. (2) If the best solution is considerably better than the other solutions, then the number of neighbors should be small to encourage exploitation in future iterations.NS, was determined according to the following formula:gbL(t) is the tour length of the global best tour at time t, klL(t) is the tour length of ant k at time t, S is the number of ants, and N is the number of cities.The number of nearest neighbors, The second modification aimed to reduce the execution time. If all ants were flying ants (such as in FACO), we would have to determine many neighbors for each node on the best path, which may substantially increase the execution time. Therefore, here, only 50% of the ants were flying ants and the rest were normal walking ants.The third modification aimed to encourage exploration at early stages and exploitation at later stages. This modification is similar to the FACO algorithm, but we modified the process to adapt it to the TSP problem. The intuition behind this is that at early stages, we have no idea about the location of the best solution in the search space; and therefore, ants should be encouraged to explore the search space. On the other hand, at later iterations, the region that contains the best solution is more likely to have been located, and therefore, exploitation should be encouraged. This was achieved by injecting pheromones at farther neighbors during early iterations, while at later iterations, we injected the pheromones at only the nearest neighbors. Equation (6) was used to determine the amount of pheromone for each neighbor.As a final modification, we embedded the 3-Opt algorithm in FACO to reduce the chances of getting stuck at a local minimum.We conducted empirical experiments using TSP datasets from TSPLIB  to test http://adibaba.github.io/ACOTSPJava/) implementation of ACO.We implemented DFACO in Java and used the ACOTSPJava , the standard deviation (SD), and the best solution for each run (Best). We also compared DFACO, PACO-3Opt, and ACO with respect to execution time in seconds.http://comopt.ifi.uni-heidelberg.de/software/TSPLIB95/STSP.html). Bold font indicates the best results.Both ACO and DFACO were run for 100 iterations (Z) and each dataset was used in 30 independent experiments. N = 8 and p ≤ 0.05. A t-test was used to see if the results were statistically significant in the 30 independent runs for each one of the eight datasets. The results indicate that the proposed method’s results were statistically significant for four out of eight datasets, namely, for the datasets rat575, rat783, rl1323, and d1655. We did not perform a t-test for the remaining 16 datasets because both algorithms achieved the BKS.For the remaining eight datasets, DFACO found the best Mean solution for seven datasets, while ACO found the best Mean solution for one dataset (fl1400). These results were found to be statistically significant according to the Wilcoxon signed-rank test, with For the second set of comparisons, we followed the comparison method of PACO-3Opt . In the The table reveals that the proposed algorithm achieved the best results for all datasets except one (rat783), for which Deng’s method  achievedPDav) and the percentage deviation of the best solution to the BKS value (PDbest) in the experimental results. PDav and PDbest were calculated using Equations (9) and (10), respectively:PDav and PDbest for DFACO were better than those for the other methods on all datasets except one (rat783), for which Deng’s method [The results revealed that the values of s method  was bettThis paper proposed a modified FACO algorithm for the TSP. We modified FACO in several ways to reduce the execution time and to better balance exploration and exploitation. For example, the number of neighboring nodes receiving pheromones varied depending on the quality of the solution compared to the other solutions. This helped to balance the exploration and exploitation strategies. We also embedded the 3-Opt algorithm to improve the solution by mitigating the effect of the stagnation problem. Moreover, the colony contained a combination of normal and flying ants. These modifications aimed to achieve better solutions with less processing time and to avoid getting stuck in local minima.DFACO was compared with (1) ACO and five recent methods for the TSP ,18,20,21


High performance materials, from natural bone over ancient damascene steel to modern superalloys, typically possess a complex structure at the microscale. Their properties exceed those of the individual components and their knowledge-based improvement therefore requires understanding beyond that of the components’ individual behaviour. Electron microscopy has been instrumental in unravelling the most important mechanisms of co-deformation and in-situ deformation experiments have emerged as a popular and accessible technique. However, a challenge remains: to achieve high spatial resolution and statistical relevance in combination. Here, we overcome this limitation by using panoramic imaging and machine learning to study damage in a dual-phase steel. This high-throughput approach now gives us strain and microstructure dependent insights into the prevalent damage mechanisms and allows the separation of inclusions and deformation–induced damage across a large area of this heterogeneous material. Aiming for the first time at automated classification of the majority of damage sites rather than only the most distinct, the new method also encourages us to expand current research past interpretation of exemplary cases of distinct damage sites towards the less clear-cut reality. The damage encountered in forming of metallic components poses both a great challenge in manufacturing, as well as enormous potential for improved lightweight design. Current efforts to lighten load-bearing structures, e.g. car bodies, focus on an increased use of low density materials, such as aluminium and magnesium alloys, enhancement of the (cold) formability of high strength steels and minimisation of the material used in structural elements.In the context of increasing formability and improving the (remaining) local strength after processing, the key challenge is to comprehend the microstructural damage induced during deformation in metal forming. This would allow engineers to minimise damage by adapting the forming processes and accurately assess the remaining material performance in service after processing to save excess material. However, the physical mechanisms underlying damage in the most prominent high performance materials, such as dual-phase steels, are incompletely understood in spite of an ever-increasing research effort over the last decade .The reason why the fundamental mechanisms of damage formation remain elusive lies in the complex microstructure of dual-phase steels. They are categorized and marketed based only on attained yield strength and the presence of a dual phase (ferrite and martensite) microstructure, but their properties are determined by a zoo of often interdependent parameters. These include volume fraction, morphology and carbon content of the hard martensite phase, mechanical contrast between the phases, homogeneity of texture, grain size and phase distribution as well as local and mesoscopic segregation of alloying elements –4.As in any other multiphase material that combines two components with different mechanical properties, the mechanical contrast between the constituents can induce high local strains in the microstructure. These typically exceed the macroscopic strain because the soft phase  has to compensate for the harder phase (martensite) to maintain strain compatibility . Hence, In the case of dual-phase steels, the pressure to achieve this, is clearly evident from the challenges still faced in unravelling damage. A vast number of studies exists from the macro- to the microscale, including a multitude of variations of the critical parameters, which have been identified –14. HoweHere, we propose a promising new framework  to achieIn the following, we first describe the employed computational and experimental methods in detail before reporting our findings from bulk and in-situ deformation experiments. The results section is divided into the three main research questions concerning this topic, namely, the distinction between inclusions and deformation-induced damage, the definition of appropriate classes for deformation-induced damage, and, subsequently, the observed dependency of damage mechanisms on tensile strain in the dual-phase steel. The results are discussed and compared within both the fields of artificial intelligence and materials science and engineering.oj is the output of a specific neuron, σ a transfer function and wij a weight matrix describing the strength of the connection between input variables xi and the output.Machine learning is a broad term which originally encompasses the “programming of a digital computer to behave in a way which, if done by humans or animals, would be described as involving the process of learning” . In pracUsing these neurons as building blocks, artificial neural networks are then built—in a simple picture—by combining many of these neurons as building blocks into layers. The first layer is called the “input layer” and receives the input from the outside world, in our case the micrograph images containing damage sites. The input is then passed through one or more intermediate or hidden layers to the output layer, which outputs the results to the outside world, in our case the researcher that receives the classification of the individual damage sites by the neural networks. As more and more powerful computer hardware has become available, neural networks can now consist of many such layers and are hence called “deep neural networks” (DNN). The exact shape of the weight-matrix allows to construct flexible and powerful DNN architectures which are ideally suited for different specialised tasks. In the case of images, so-called convolutional neural networks (CNN) have proven to be particularly apt in image recognition and analysis . The namIt can be shown mathematically that this is equivalent to the convolution of two distributions. Hence, CNNs can obtain the ability to act like filters in conventional image processing . The main idea behind using CNNs is then that the various convolutional layers of the network can automatically learn the relevant features in the images associated with the content of the image, in our case a particular class of microstructural damage. The weight matrix of the final layer of the CNN network is fully populated and the individual weights are free to choose any value, i.e.This so-called “fully connected” layer is used to translate the output of the CNN analysis into a probability associated with the identified damage class via a softmax function. The fundamental setup of the neural networks used in this analysis is then: input layer, several convolutional layers, a fully connected layer followed by softmax function.The neural networks are trained using supervised training. At each iteration of the training process, a set of images is presented to the network for which the “true” output (label of damage sites) is known. Using the current state of network weights, the network calculates an output based on each image. Using an appropriate cost function, the difference between the calculated output and the desired (true) output is transformed into a penalty. The network weights are then iteratively adjusted via back-propagation, such that the discrepancy between the computed and desired output is minimal. One of the principal difficulties in training neural networks in general, and for this study in particular is access to sufficient high quality training data. Since no prior labelled training data was available, a dataset of—in total 4944 labelled damage sites—was obtained in the following way : PotentiThe analysis setup is illustrated in It was decided to split the total classification into two separate networks, for the sake of its use in application. As for many purposes like measurements of void area fractions of deformation-induced damage, only a separation into inclusion voids and deformation-induced voids is required, a first step should include this distinction exclusively. Then, in a second step, only the voids classified as deformation-induced should be processed further into classifying individual damage mechanisms. For the first network, a standard network (InceptionV3 ) was fouBoth networks were implemented in TensorFlow (v1.2.1)  using thm,n are the elements of the confusion matrix. The uncertainty of the accuracy was determined in the following way: The CNNs are initialised using pseudo-random numbers with variable seeds. This implies that in each training round the final network weights will be slightly different due to the numerical optimisations during the training. In this regard, the true positive rate  was taken as the ratio of samples correctly identified as belonging to class m and the number of all samples belonging to this class, i.e.The performance accuracy of the network was evaluated on 20% of the available data, which were retained as an independent test sample and not used in the training process. The accuracy is defined as the ratio of correctly identified images vs all available images in the sample:The CNNs were run on a Linux-based system, equipped with AMD FX-8340 CPU and 32 GB RAM. For the training a GeForce GTX 1070 GPU was employed. Since GPUs are non-deterministic in the implementation of their algorithms, slight variations are expected even if all other parameters are initialised to the same values. To estimate the size of the effect, each network was trained 10 times and the mean of the resulting spread in accuracy was taken as the point estimator, and the standard deviation as the uncertainty measure.1 = 0.9, β2 = 0.999, ε = 10−8, and a learning rate decay of 0.0. See Figures A and B in the The training of a network took typically a few hours depending on the network architecture and more importantly on the number of training samples. Training on CPU would be possible in principle but training times are expected to be significantly higher by some orders of magnitude and have thus not been attempted. Training was executed for 50 epochs for the first network and 90 epochs for the second network. Both networks were trained using the Adam optimizer with a learning rate of 0.001 for the first network and a learning rate of 0.0005 for the second network, the remaining parameters were left at their default values, i.e. βThe two dual phase steels used in this work are of commercial origin  and are marketed under the label “DP800”, which warrants both a microstructure consisting of ferrite and martensite  and a tensile strength of at least 800 MPa. The microstructures differ in grain size and martensite morphology due to changes in processing and heat treatment. All tensile samples were cut out of the 1.5mm thick sheet metal by wire erosion, with the tensile axis being parallel to the rolling direction.Metallographic preparation of all samples included mechanical grinding up to 4000 grit paper and mechanical polishing in diamond suspension in three polishing steps down to 1µm, followed by an additional final polishing step in a 0.25µm oxide suspension. All samples were lightly etched to ensure clear visibility of the ferrite and martensite constituents, while avoiding the build-up of a strong topography creating shadowing on the SEM images that might be detected as black areas by the void recognition system. For etching, a 1% Nital solution was used and the etching time was chosen to be 5 s to maintain the above-mentioned conditions.The tensile samples were deformed by an electromechanical testing machine  with an accuracy of ±0.17 MPa in the stress measurement. Strains in post-mortem experiments were measured as global strains over the deformed sample area based on the elongation between two markers in the undeformed parts of the sample. For in-situ tensile tests, a smaller deformation stage  was useThe panoramic images were acquired using scanning electron microscopes . All images were obtained using secondary electrons with a resolution of 32.5 nm/px . Image stitching for panoramic imaging was realised using MATLAB based on the VLFeat Matlab toolbox . Damage In the performed in-situ experiments, all samples were pre-strained to 14% global strain with reference to the initial gauge length of 3.6 mm. Additional strains were measured by averaging over a field of view in the panoramic imaging using a correlation between the individual panoramic images. The strains were averaged from the DIC measurements performed with GOM Correlate image correlation software . The so-obtained values therefore have to be regarded as average values, not over the complete sample, but rather limited to the investigated area by panoramic imaging.In understanding damage and deriving mechanism-based models of deformation in the dual phase microstructure of DP800 steel, one has to first distinguish between voids induced by the co-deformation of the ferrite and martensite phases and those induced around inclusions of foreign phases. Although the latter do not commonly cause a large amount of damage , they caThe first step of our damage analysis is an automated acquisition of secondary electron micrographs with a resolution of 33 nm/px and a total field width after stitching to a single panoramic image of the order of 1 mm. In order to avoid a significant contribution of shadowing to the images yet be able to distinguish between ferrite and martensite phases, the commercial-grade DP 800 steel was subjected to light chemical etching beforehand. All voids within the panoramic micrograph were recognized via a thresholded image followed by a clustering algorithm  any transition from brittle to ductile damage and (ii) the attainable resolution to resolve this transition. Furthermore, the categories need to be simple enough to allow reliable attribution to the individual categories independent of user experience or knowledge. This is important as deep learning involves training on labelled images and the success of the approach is therefore based on our ability to reproducibly assign distinct damage classification labels. A small study on 19 materials scientists ranging from undergraduate students to experienced researchers revealed that the application of the plethora of common damage classifications including cracking and decohesion of different boundaries is not an exact science. It is prone to great deviations between individuals where voids are regularly attached to both phases in a fine microstructure or damage has evolved significantly at advanced strain levels .We therefore propose the following damage classes for a statistical and automated analysis of damage across high-resolution micrographs: (i) brittle martensite cracking, (ii) martensite-ferrite interface decohesion, and, (iii) a category labelled “notch” for short in which we collect those damage sites, which are too small to be reliably assigned to a distinct damage initiation mechanism. Mostly, these include round voids at thin martensite bridges, where one suspects the rupture of the martensite bridge due to the stress concentration to be the source of damage without the typical fracture faces that are usually clearly visible. Examples of each class are shown in This leaves us with 4 clear-cut categories of damage (including the inclusion-induced voids discussed above) and the less distinct category of “evolved” damage. In order to achieve a clean training dataset for the neural networks, we therefore used only the former four categories for training of the neural networks. Clearly, the choice of training data and categories is therefore a critical point in which the main aim of the analysis is set for all following analyses using the same network. We chose to focus on the stage of nucleation of damage to achieve a clearer picture of the dominant damage nucleation mechanisms at different strain levels, leaving the analysis of evolving damage to the more appropriate method of in-situ experiments.In order to identify the dominant damage mechanism(s) associated with a certain strain path, in this case uniaxial tension, the relevance of both the quality and quantity of results has to be ensured. In most DP steels, the martensite fraction is distributed heterogeneously across the material, mainly Each of these methods has its strengths and weaknesses and we therefore employed both ex-situ and in-situ experiments in this study using damage quantification and classification via artificial intelligence to enable the analysis of suitably large areas. The results showed that the in-situ experiment yields significantly different damage characteristics due to evolution of damage at the free surface, as opposed to ex-situ experiments, where deformation occurs in the bulk and is revealed post-mortem. A direct application of damage classes defined using the morphologies observed after deformation in the bulk to in-situ experiments is therefore fraught with peril. In the following, we first present the ex-situ statistical analysis of damage at different strain levels followed by the results of the use of automated identification, classification and tracking of individual damage sites during their evolution in in-situ experiments.To achieve the required size of analysed area capturing the variability in the microstructure we imaged an area of 900.000 µm² at a resolution of 32nm/px for each strain step reported below. Note that these areas were recorded in the sheet plane, but the dimensions would be equally applicable to cross-sections in rolling or transverse direction to capture the gradients in grain elongation, phase morphology and banding across a typical sheet thickness of 1 mm.We present here the gathered data on detected damage sites from tensile tests to different strains with two purposes in mind: first, to present a statistical meaningful dataset, which covers strains from 12–24% with smaller strain increments analysed to much greater statistical relevance than available elsewhere and, second, to illustrate the success of automating this laborious analysis by artificial intelligence. The latter step is crucial to make this kind of analysis possible on a regular basis in laboratories around the world. We can thereby finally overcome the challenge of comparison and transfer of data and derived models in this (and other) technologically important, yet immensely complex, class(es) of alloy(s).The statistics reveal an exponential increase in the total number of damage sites with increasing tensile strain in the sample . This isThe damage mechanisms classified as outlined above paint an interesting picture. The relative contribution of each damage class to the overall damage  shows a At this point, it is important to distinguish between the classification by a human investigator and the neural networks. As outlined above, the neural networks were trained using the categories inclusion, martensite cracking, phase boundary decohesion and notch. Strongly evolved damage sites, e.g. martensite cracks, which have already induced flow in the surrounding ferrite, were excluded in order to remove the bias of the training data, where different human investigators might not be confident that they can identify the underlying nucleation mechanism. As this distinction results in a training dataset with clearer dominance of each mechanism but cannot be based on distinct thresholds, the category of evolved damage cannot be implemented in a straightforward manner into an automated analysis by artificial intelligence. The classifications by a human investigator and the analytical algorithm consequently differ in some respects: first of all, not all sites are attributed to a mechanism by the human investigator, as a significant percentage is labelled as evolved damage sites. Most of these sites are, however, classified by the neural network as phase boundary decohesion sites due to their curved outline resulting from ductile flow, rather than brittle fracture across a straight path. While this might be a wrong classification in respect to the nucleating mechanism, it is consistent with ductile processes dominating evolution of voids at intermediate strains  regardleTo quantify the considerable differences in damage behaviour in different grades of commercial DP800 steel, a second microstructure containing a different martensite morphology was investigated. This material showed not only a higher ductility, but interestingly also a significantly lower number of nucleating damage sites during tensile deformation . The obsIn dual phase steels, investigation of a large area at high resolution is essential to provide comparability and transferability of results within this class of materials due to the scale of the microstructure and damage initiation at the sub-µm scale coupled with material heterogeneity up to the mm-scale. The results from our comparison of damage evolution in two different DP800 microstructures underlines this. The stark difference in damage densities with strain will make it necessary to statistically quantify and classify the damage mechanisms and their occurrence in every grade of DP-steel commercially available by combining high-resolution imaging at the scale of the damage mechanisms and large-scale observations to obtain sufficient statistics. To achieve an accurate and transferable results in characterising and, ultimately, predicting damage nucleation and evolution in any specific dual-phase microstructure, these time-consuming statistical analyses need to be conducted for each distinct material grade. Such an endeavour is in fact feasible via our automated approach as it is able to handle a high volume of damage sites in a drastically reduced amount of time compared to a human investigator.Using a neural network to classify images into different damage categories leads us beyond the common threshold analysis of greyscale images. It allows us to distinguish not only the constituent ferrite and martensite phases and related pores from the surrounding matrix, but also distinguish other types of pores. In particular, image classification allows us to treat inclusions and the pores they cause where they may have fallen out during metallographic surface preparation separately from deformation-induced damage. This is of particular importance at small strains and where damage is quantified by means of an area fraction.Owing to the large number of damage sites observed at high resolution, we could show for example that martensite cracking precedes what is often classified as phase boundary decohesion in later stages of damage evolution. Over a strain interval of 12–24% plastic strain, we further demonstrated a pronounced change in governing damage mechanisms from brittle martensite fracture at the onset of damage to ductile damage initiation and evolution at higher strains. This is consistent with the observations from the literature, where martensite cracking is frequently reported as the dominant mechanism of damage initiation  and intWhile statistical evaluation of deformed samples provides the important quantification of the number or area of damage sites and the dominance of each mechanism for a given strain level, it cannot deliver local information about the specific evolution of individual damage sites. To achieve this, additional in-situ tensile tests within an SEM were performed, completing the picture of damage statistics with time-resolved information at high resolution. The experiments were performed in such a way that a pre-strain of 14% was imposed prior to surface preparation for the in-situ observation, providing a pre-damaged state at the start of the in-situ test. This was necessary because the overall strain increment is usually limited by the formation of artefacts due to surface roughening from plastic flow at the open surface. The induced damage sites were then identified from the previously acquired panoramic image and classified in this state, which is equivalent to the post-mortem analysis presented above. Each site of interest can then be automatically tracked over subsequent in-situ deformation steps and pre-processed to achieve efficient analysis and interpretation as shown in Our in-situ tests show that the prevalence of nucleation of different types of damage sites is affected by the chosen method of investigation. For example, phase boundary decohesion, seen frequently in the post-mortem analysis of bulk samples, was barely encountered in the in-situ tests. Instead, plastic flow at the free surface caused “sink-in” of the ferrite matrix that led to the formation of deformation artefacts in the form of trenches . These aIn contrast, the brittle mechanisms of martensite fracture appeared largely unaffected by the free surface in terms of their morphology. Further analysis of the nucleated martensite cracks during in-situ testing is therefore much more likely to shed a reliable light on the evolution of these damage sites with increasing strain level. This allowed us to not only track pre-existing damage site caused by martensite cracking in their evolution during the in-situ experiment, but also identify newly nucleated sites in subsequent deformation steps. These were then back-tracked to the pristine site , giving Across all sites of martensite cracking, we found that while some sites exhibited rapid growth as a result of plasticity in the ductile ferrite phase , other sThe comparison of the results obtained in an in-situ test with our results on statistical damage analysis of material deformed in bulk (ex-situ) highlights the importance of large-area datasets and the observation of bulk deformation as the essential reference case, even where the experimental facilities for elegant in-situ tests are readily available. An ex-situ evaluation is unable to reveal the evolution of individual voids, however, it is essential to consider that the in-situ observation of void nucleation and evolution at the surface causes a substantial variation in void nucleation and evolution. This is widely acknowledged, but without the possibility of statistical analyses to reveal which damage mechanisms constitute the typical bulk case, crucial differences, like the altered void growth behaviour of martensite cracks at the free surface, are hard to quantify rigorously as an in-situ artefact caused by the free surface. Of course, the out-of-plane component of flow in in-situ experiments may still be considered in conjunction with crystal plasticity modelling, but so far, a truly quantitative evaluation is impeded by the experimental difficulties again encountered when combining high resolution and investigation of large areas , 43.Our comparison of the statistical analysis of damage in bulk and that at the free surface reveals that in-situ tests are, however, representative in the context of brittle fracture in martensite. In this case, each individual martensite island present in the bulk or being part of the free surface may of course experience an altered stress state, leading to the absence of further growth of damage induced in the bulk during subsequent in-situ deformation. Still, the general morphology of martensite cracks remains the same and in-situ tests are therefore ideally suited to reveal the mechanisms of brittle damage nucleation. In-situ tracking of martensite cracks also included the ductile growth of a fraction of these originally brittle martensite cracks. This process portrays the transition to ductile mechanisms after brittle cracking. It was, in a similar way, observed in the material after bulk deformation as an increasing dominance of ductile mechanisms in void evolution . While tThe neural networks developed in this study reduced the time required for micrograph analysis in terms of damage classification from days to minutes. While identifying and classifying a single damage site takes a human investigator about 20 s, leading to 5.5 hours per panoramic image with ~ 1000 damage sites, the use of trained artificial intelligence for this purpose reduces the time required to less than 1 minute . DependiThe use of automated recognition and analysis of damage sites by artificial intelligence in combination with in-situ testing is not an end in itself. Beyond the ability to evaluate representative datasets and save human supervision time on repeated analyses of similar microstructures, the approach enables studies that were previously inaccessible due to the constraints on human time; for example, the quantification of damage sequences in a statistical manner and coupling with high-throughput simulation to overcome the challenge of the missing 3D microstructure information where no suitable experimental methods , 45 are In this study, we successfully implemented a high-resolution large-scale analysis with high throughput to provide damage quantification and classification in ex-situ and in-situ experiments. The following conclusions can be drawn:For complex, heterogeneous microstructures, such the ones encountered in dual-phase steels, the use of trained convolutional neural networks to classify a large number of damage sites at high resolution has proven extremely useful in gathering statistical information of representative damage mechanisms.For engineering purposes concerned with quantification of damage by means of an area fraction, it is prudent to split the automated classification work into two separate networks. The first network is to discriminate voids formed due to foreign inclusions from those induced by plastic deformation, and the second is to classify deformation-induced damage in conjunction with responsible damage mechanisms.A separate treatment of inclusion-induced and deformation-induced damage (as previously mentioned) provides a valuable improvement in accuracy for quantitative evaluation of damage statistics, especially at low deformation strains, where in the present case over half of the measured damage sites were affiliated with inclusion damage.The statistical viewpoint made possible by our approach unravels a more complete picture of the dominant deformation-induced damage mechanisms depending on the strain level. At incipient stages of deformation and damage initiation, brittle martensite cracking seemed to be the prevailing mechanism. An increasing amount of plastic deformation dominates at larger strains and led to rapid growth of voids and expansions of martensite cracks into the adjacent plastic ferrite phase.The damage behaviour of martensite cracking was reproduced in in-situ experiments since it was not affected by the surface artefacts generated upon straining. However, the morphology of the ductile damage sites were found to differ considerably from the bulk case. This highlights the need not to rely too heavily on in-situ experiments and interpret respective findings and their implications for bulk damage behaviour with great care.The present approach can, in principle, be transferred to similar image-based challenges in other complex microstructures at all scales. In the context of dual phase steels, a meaningful comparison of the manifold of microstructures subsumed under each industrial grade would be an exceedingly fruitful next step that now appears within reach. If successful, it would truly bring together the insights into the materials physics of deformation-induced damage, currently scattered across laboratories worldwide to enable more powerful knowledge-driven microstructure and process design for this important material class.Finally, any image recognition algorithm would have its limitation in real microstructures due to the complex interplay of individual damage formation mechanisms, rendering a major proportion of damage sites not clearly attributable to a single mechanism. Although we chose to focus on the mechanisms initiating damage in this work, we do note that in future work it should be feasible to further sub-categorize evolved damage based on sequential evolution stages associated with the onset of additional mechanisms.S1 Supporting informationThis supporting information contains a more detailed description of the network and relevant terms in (PDF)Click here for additional data file.