Additive manufacturing technologies have numerous advantages over conventional technologies; nevertheless, their production process can lead to high residual stresses and distortions in the produced parts. The use of numerical simulation models is presented as a solution to predict the deformations and residual stresses resulting from the printing process. This study aimed to predict the tensions and distortions imposed in the gear repair process by directed energy deposition (DED). First, the case study proposed by National Institute of Standards and Technology (NIST) was analyzed to validate the model and the numerically obtained results. Subsequently, a parametric study of the influence of some of the parameters of DED technology was carried out. The results obtained for the validation of the NIST benchmark bridge model were in agreement with the results obtained experimentally. In turn, the results obtained from the parametric study were almost always in line with what is theoretically expected; however, some results were not very clear and consistent. The results obtained help to clarify the influence of certain printing parameters. The proposed model allowed accounting for the effect of residual stresses in calculating the stresses resulting from gear loading, which are essential data for fatigue analysis. Modeling and simulating a deposition process can be challenging due to several factors, including calibrating the model, managing the computational cost, accounting for boundary conditions, and accurately representing material properties. This paper aimed to carefully address these parameters in two case studies, towards reliable simulations. Additive manufacturing has gained increased relevance in recent years. This unconventional manufacturing process is an important complement to the existing traditional processes . Its appWithin this new universe that is additive manufacturing, metal processing is currently at the forefront of the development field ,4. Due tFor cost-effective fabrication of end-use metallic AM parts, three processes are commonly used: selective powder sintering, selective powder melting, and DED (with powder or wire feeding). The primary heat sources for those processes include lasers, electron beams, electric arcs, and plasma .Powder bed fusion (PBF) processes were among the first commercialized AM processes. All PBF processes share a basic set of characteristics. These include one or more thermal source for inducing fusion between the powder particles, a method for limiting the powder fusion to a prescribed region of each layer, and mechanisms for adding and smoothing the powder layers ,10.In the powder bed fusion process, thin layers of powder are applied to a build plate, and an energy source (a laser or electron beam) is used to fuse the powder at the locations specified by the model of the desired geometry. When one layer is completed, a new layer of powder is applied, and the process is repeated until a three-dimensional part is produced. The PBF process is alternatively known as selective laser sintering (SLS), selective laser melting (SLM), directed metal laser sintering (DMLS), directed metal laser melting (DMLM), and electron beam melting (EBM) .Unlike powder bed fusion techniques, DED processes are not used to melt a material that is prelaid in a powder bed but are used to melt materials as they are being deposited . One of A number of organizations have developed DED machines using lasers and powder feeders. These machines are referred to as laser engineered net shaping (LENS), directed light fabrication (DLF), directed metal deposition (DMD), 3D laser cladding, laser generation, laser-based metal deposition (LBMD), laser freeform fabrication (LFF), laser directed casting, LaserCast, laser consolidation, LasForm, and others . AlthougThe DED and PBF processes are the predominant technologies used worldwide; therefore, several articles have been published to verify the applicability of DED for repairing damaged parts in various industries ,14,15, oThe use of numerical models is presented as a solution that allows accurately and efficiently predicting the behavior of parts ,18,19,20Since it was not possible to validate the gear numerical model with experimental data, the efficiency of the numerical model was validated using the results obtained from the NIST AMB2018-01 Benchmark study . A compaThis parametric study was motivated by the GEAR3D project, which consists of the 3D reconstruction of large gear teeth for the wind power sector, via direct laser deposition, ensuring mechanical properties equal to or greater than those of the new components . The parSome works have already focused on this problem, and an advanced automated damage detection and damage reconstruction algorithm for damaged gear tooth repair has already been proposed .This work focuses on the parametric investigation of the DED technology used to reconstruct mechanical components, namely gears, aiming to demonstrate the possibility of predicting the residual stresses resulting from an additive manufacturing process.The objective of this paper was the numerical simulation of the reconstruction of mechanical components using a directed energy deposition additive manufacturing process. Before simulating the repair of the gear through the DED process, a preliminary simulation was carried out to validate the numerical procedures. The AMB2018-01 Benchmark bridge simulation from NIST  was analMesh: Convergence studies to choose a mesh that allows obtaining accurate results with the lowest possible computational cost under user-defined parameters;Step Time: The time step has a direct influence on the number of elements that are activated, and therefore it is also an important parameter to be studied;Preheat temperature: Study of the influence of different preheat temperatures;Scanning strategies: Comparison of different laser scanning strategies, to study the impact of this parameter on the residual stresses;Material: The use of different materials has a direct influence on the residual stresses produced, hence the importance of their study.It was intended in this work to conduct a thermostructural study of gear repair with DED, using the finite element method via Abaqus, to understand the origins of residual stresses and the impact of different process parameters. For this parametric study, the following simulations were performed:The AMB2018-01 tests consist of laser powder bed fusion (LPBF) 3D metal alloy builds of a bridge structure geometry that has 12 legs of varying sizes. The 12 legs consist of four replications of the section described in the green box in As shown in The bridge is positioned on a build plate. The build plate structure is 90 mm long, 15 mm wide, and 12.7 mm tall.The DED process can be used to repair various mechanical components, including gears. Gears are a very useful transmission mechanism that is used to transmit rotation from one axis to another. The gear system can be used to decrease the speed . Gears are commonly used in high-load situations, because the teeth of the gear allow finer, more discreet control of the movement of a shaft. Due to the high load situations to which gears are subjected, it is important to ensure that repairs are capable of withstanding these forces.To simulate the repair of a damaged gear, two main elements were considered, the tooth and the gear with a cut tooth, as shown in The gear in The parameters used in the DED process to repair the gear in The models for numerical simulation were created from the technical drawing of the pinion, The material for the benchmark bridge simulation was nickel-based superalloy IN625. In the parametric study of the DED process, IN625 was always used. In addition to this material, 42CrMo4 and 18Ni300 Maraging are used only in numerical simulation in order to study the influence of different materials.IN625 is a nickel-based superalloy known for its high level of strength, temperature resistance, and corrosion resistance. The strength of IN625 is derived from the stiffening effect of molybdenum and niobium in its nickel–chromium matrix, thus precipitation–hardening treatments are not required .While conventional manufacturing of components using these high-performance alloys has been difficult with machining, due to excessive tool wear and low material removal rates, powder-based AM technologies can remove these constraints, while improving lead times and reducing manufacturing costs .Latent heat effects can be significant and must be included in heat transfer problems involving a phase change. The latent heat is assumed to be released over a range of temperatures, from a lower (solidus) temperature to an upper (liquidus) temperature, The latent heat is the extra energy required to achieve the phase change in the material, in the form of thermal energy absorption (melting) or release (solidification). Latent heat can be combined with any other material behavior in Abaqus, but it should not be included in the material definition unless necessary; it adds an extreme non-linearity to the thermal solution, which may inhibit convergence.The temperature dependence properties can be seen in Young’s modulus is a mechanical property of solids, which gives the stiffness of the material. It can also be defined as the ability of the material to withstand changes in its shape under expansion or compression. The Young’s modulus of a material is defined as the ratio of longitudinal stress to the linear strain; as the temperature rises, Young’s modulus decreases, The term “maraging” is a portmanteau of martensite and aging and represents the age hardening of a low-carbon, iron–nickel martensitic matrix. Maraging steel is a class of high-alloy, low-carbon steel developed for structural applications requiring high strength. This family of alloys have good weldability, due to the practically complete absence of interstitial alloying components. As a result, they are conducive to metal additive manufacturing procedures such as laser metal deposition (LMD) and selective laser melting (SLM) .The qualities of this material have not been well established and documented; as a result, the thermal property values were obtained experimentally  and are A is the static yield strength, B is the strain hardening coefficient, n is the strain hardening exponent, m is the thermal sensitivity parameter, and The Johnson–Cook constitutive model is not based on traditional plasticity theory and reproduces several important material responses observed in the impact and penetration of metals. Strain hardening, strain-rate effects, and thermal softening are the three main material responses. The Johnson–Cook constitutive model multiplicatively combines these three effects .(1)σ=A+In Equation , the fir42CrMo4 (1.7225) steel is a low-alloy steel that contains chromium, molybdenum, and manganese. The presence of chromium and molybdenum increases the corrosion resistance. Molybdenum is especially useful in resisting the corrosion caused by chlorides. This steel has different grades: the European designation is 42CrMo4 (1.7225), and the USA grade is 4140.This steel is widely used in a variety of industries such as aerospace, oil, and gas, as well as many other industries. The thermophysical properties of the material 42CrMo4 are given in To characterize the plastic regime of this material, the Johnson–Cook constitutive model was used—Equations  and 2)..2). In TThe latent heat properties are shown in In a powder-bed-type additive manufacturing method, a recoater or a roller blade deposits a single layer of raw material. The part is then scanned in a single cross-section with a high-powered laser across the raw material layer to fuse it with the previously deposited layer beneath. Layer-upon-layer raw material deposition is represented by a progressive element activation in structural or thermal analysis, while laser-induced heating is represented by a moving heat flux .During the laser-directed energy deposition (LDED) method, the material is deposited by a nozzle positioned on a multi-axis arm, while being melted by a laser beam. In structural analysis, a beginning temperature denoting a relaxation temperature above which thermal straining causes low thermal stress is typically assigned, to accurately capture the melting effect .In the additive manufacturing benchmark test simulation, two parts were used— Several simulations were performed, until the model was fine-tuned, to give consistent results. IN625 was used in both parts, and the results obtained from this simulation were later compared with the experimental results presented by NIST.In the heat transfer analysis, an initial temperature of 80 °C was applied to the build plate. The initial temperature of the part was 40 °C, which was the temperature at which the powder material was dispensed from the powder bed reservoir.In structural analysis, the initial temperature of an assembly represents the relaxation temperature above which thermal strain results in negligible thermal stress. In material activation, it represents the temperature at which the initial thermal shrinkage occurs .In this analysis, the initial temperature of the bridge was set to 750 °C. The initial temperature of the build plate was 80 °C.A 3D 8-node linear brick element was employed in both the thermal and mechanical simulations. The structural element had the designation C3D8 and the heat transfer element was DC3D8. Along the printed area, the spatial distribution of the elements was uniform or “mapped.” The printing simulation results were influenced by the element sizes.The first letter of the element’s name indicates the family to which the element belongs, in this case, the C in C3D8 indicates this is a continuum element. The number 3 represents the degrees of freedom.The mesh of the bridge structure consisted of 8-node linear diffusive heat transfer DC3D8 elements. The elements varied in size but had a common characteristic height of 0.2 mm, so that there were approximately ten layers of material in each element. The build plate was more coarsely meshed with DC3D8 elements. In the static analysis, C3D8 solid elements were used for both the bridge and the build plate, maintaining the same mesh topology used in the heat transfer analysis.A laser beam scans at a controlled speed the selected locations of the powder bed and fuses the powder, then the powder bed is lowered by a defined layer thickness and a new layer of powder is added. Progressive element activation can be used in Abaqus to represent layer-upon-layer raw material deposition in a thermal-stress analysis.A sequential thermal stress analysis of an additive manufacturing process consists of a transient heat transfer analysis of the thermal loads introduced by the process, followed by a static structural analysis that is driven by the temperature field from the thermal analysis .During the analysis, it is possible to control the activation and the volume fraction of the material for each element in each increment. Any element that is defined as progressively activated is filled with material or remains empty (inactive).Heating source: The toolpath–mesh intersection module takes the time–location history and automatically computes all of the information required to activate elements and apply the proper thermal energy to the model.Recoater motion: The infinite line representation is useful for describing this process of layer-by-layer material deposition. All path segments when the tool is in the “on” state must be perpendicular to the global z-direction.Table collections: Table collections that encapsulate parameter tables or property tables can be used to define additional process parameters needed for the simulation.To define the material deposition process, it is necessary to consider the following steps :Heating The scanning strategy is the spatial motion pattern of the energy beam . For a single-layer scan, the scanning strategy varies with different scanning directions, scanning sequences, scanning vector rotation angles, scanning vector lengths, scanning times, and hatch spaces .The impact of the scanning strategy during the processing of different materials has been the subject of numerous research works. The laser speed, the solidification direction imposed by the moving heat source, the hatch space, and the cooling rate are some of the parameters that greatly influence the results obtained. These parameters influence the mechanical properties obtained, as well as the residual stress, porosity levels, and crystallographic texture of the material .For this simulation, a scan strategy was used in which for odd-numbered layers, the infill scans were horizontal lines  that were separated by 0.1 mm (hatch spacing), and for even-numbered layers, the infill scans were vertical lines  that were also separated by 0.1 mm. The process starts with the laser going through a full layer of the bridge, then the laser is disabled, and the roller is activated. The roller extends a thin layer of metallic powder, and then the laser is activated again to go through another layer. This process is repeated until the final structure of the bridge is obtained. Each powder layer is 0.02 mm thick, and a total of 625 layers are needed to build the bridge structure. In The two heat-transfer mechanisms of cooling to the environment are convection and radiation. Both were subject to a sink temperature of 25 °C. Except for the bottom surface of the build plate, all of the model’s free surfaces were set to a typical film coefficient for an inert gas atmosphere of 0.018 mW/ to remove a part from a build plate or to remove support structures, heat treatment, or other subsequent machining processes, are frequently required to be simulated .After the build was completed, the parts were cut using wire electrical discharge machining, so that only the ends of the parts remained attached to the plate. In the simulation, the cutting process with wire EDM was modeled using progressive removal of specified elements in the cutting region. This was modeled in a separate step using a model modification to remove a layer of elements near the bottom of the legs. One of the ends of the part was still attached to a portion of the build plate. The cut part of the bridge bent upward, due to the relaxation of residual stress, The GEAR3D project  was the The main objective was to perform a parametric study evaluating the influence of the various parameters of the simulation. Thus, different materials, mesh sizes, scanning strategies, preheat temperatures, and step times were the five parameters studied.During the process of modeling the gear, several attempts were made to obtain a reliable model. One of the first possibilities was to model two pieces separately. One of the pieces would be the substrate—the gear without the tooth that would be rebuilt, and the other piece would be the shape of the tooth to be rebuilt by DED. Both parts are represented in Another possibility was the design of a single piece, The mesh discontinuity problem becomes evident through the detachment of the two pieces when the obtained results are analyzed. Thus, only one of the mentioned possibilities could be considered for the simulations.For the study of this parameter, three different meshes were used. The most discretized mesh had an approximate size of 0.25 mm and will be named Mesh-1, the second mesh had an approximate size of 0.5 mm and will be referred to as Mesh-2, finally, the coarsest mesh had an approximate size of 1.5 mm and will be referred to as Mesh-3. In The material used in the mesh study was IN625, the same for the entire gear. The strategy chosen was alternate hatches—single pass.The mesh sizes presented above, and the number of elements related to each mesh are organized in The literature describes various methods for mitigating the residual stress effects of residual stress on a printed part ,42,43. OFor this parametric study, the strategies given in The scanning strategy used here was zig-zag, and the final piece was obtained by deposition of layers with different angles of deposition. Before the filling material is deposited, a layer is deposited on the edge.Preheating involves raising the temperature of the base metal above the ambient temperature, before the DED process begins. By preheating the base plate, the thermal gradients are lowered and stresses can be reduced, avoiding the cracks and delamination resulting from residual stresses .During the initial depositions of the AM process, reduction of thermal stresses is critical, due to the presence of the substrate base, which acts as a heat sink and thus causes the formation of even higher thermal gradients and cooling rates .Assume that two analyses are performed to simulate two different time increments: a small time increment activating one element per increment, and a large time increment activating two elements per increment. In the two stress analyses, the initial configuration of every second element differs, resulting in different residual stresses and distortions .The objective of this work was to make a parametric study of some of the important variables in the repair of gears. Therefore, it was important to consider the use of different materials when repairing gears, Thus, to study the influence of the material, it was considered that the base plate (the gear) was built from 42CrMo4 steel and the tooth printed by DED was made of IN625.For simulation of the AM process, Abaqus offers two options for additive process simulation, which allow for the definition of appropriate boundary conditions, loads, interactions, constraints, and temperature-dependent material properties.In the simulations performed, an uncoupled thermomechanical simulation was used, which allowed exact specification in time and space of the processing conditions.The results of the simulation are shown in The x strain components were mostly tensile, as shown in When the simulation results were compared to the NIST results, a few key features show significant agreement between the simulation and experimental results. A comparison of the simulation and experimental contour plots, In a similar manner, the comparison of contour plots in the z-direction revealed a very high level of convergence. As both neutron diffraction—ND and X-ray diffraction—XRD techniques measured lattice spacings, it was preferable to compare residual strains rather than calculated stresses. This was especially true because the synchrotron X-ray method could not measure all three orthogonal components required to calculate stresses reliably. Only strains in the X and Z directions could be directly compared using the neutron and X-ray results .It should be noted that the XRD measurements and simulation results were taken from X = 0 mm to X = 75 mm, and the ND measurements were taken from X = 0 mm to X = 60 mm.However, the results near the edges of the sample once again reflect ND’s inability to capture large strain gradients.On the other hand, the results obtained from the numerical simulation (blue line) were almost always slightly superior to those obtained from measurements.After being built, the parts remained on the build plate. The tops of the 11 ridges were skim cut, to remove the rough as-built surface. The vertical height, relative to the base plate, of the top of both side edges of each ridge was measured. The upward deflection results are presented in In The experimental values of deflection achieved by NIST, and measured with sensors ,48, are In this section, the results obtained from the parametric study performed are reported. The residual stresses presented in this section correspond to the Von Mises stresses.This section investigates the effects of mesh size on the residual stress. For the three different sizes shown in The results obtained for the three meshes studied are grouped in Analyzing Computational cost is a parameter that must be taken into account, because one of the objectives of this type of numerical simulation is to obtain results in the shortest possible time. This compromise between computational cost and accurate results must be balanced, in order to have credible results in an acceptable time frame.In this way, After analyzing the results it becomes clear that Mesh 2 is the best option. This mesh presented results very similar to mesh 1; however, the computational cost was much lower, due to the lower refinement of this mesh. Although mesh 3 had the lowest computational cost, the results obtained do not suggest that it presented an adequate description of the process.Therefore, mesh 2 was used in the subsequent numerical simulations.The scanning strategy is a key factor in determining the thermal history and melt channel combination within a part in a laser powder bed fusion (LPBF) process, which can significantly affect the microstructural evolution, defect formation, and mechanical properties of the part .There are numerous ways to scan a single layer, and various scan strategies have been investigated in terms of residual stresses. In this section, the influence of the three types of scanning  already presented in In Analyzing the results, it was concluded that both strategies 1 and 2 presented very similar results, with a negligible variation. In turn, the third scanning strategy presented values lower than those obtained by the previous strategies.Again, residual stresses were concentrated at the root of the tooth; however, with scanning strategies 1 and 2, they propagated along the tooth, as shown in The preheating temperature has a lot of influence on residual stresses and distortion. It was expected that by increasing the preheat temperature in the base plate (gear), the residual stresses and warping of the printed part (tooth) would decrease.The numerical results obtained for the preheat temperature variation are summarized in The results obtained are shown in Analyzing the obtained spatial displacement results, a large increase is visible, as the preheat temperature also increased. Comparing the results obtained, an increase of 527% from the minimum preheating temperature to the maximum temperature was verified.The images taken from Abaqus and grouped in The results obtained from the step time variation are presented in The time step is a parameter that influences the activation of the elements, so if the chosen step time is higher, more elements will be active at once, which reduces the quality of the process simulation and consequently decreases the quality of the results obtained. Therefore, it was expected that the simulations with high time step values would present less accurate results; conversely, simulations with lower time steps would present more accurate results.Five simulations were performed, varying only the initial increment and maximum increment, and the total step time was 60. Analyzing the results obtained, see Note that the graphs in In turn, in In this section, two different materials were used for tooth reconstruction; IN625 and Maraging steel. To obtain reference values to analyze the evolution of residual stresses, it was first assumed that the gear and the reconstructed tooth were made of the same material. Subsequently, it was considered that the gear was made of 42CrMo4 and the tooth was made of IN625 or Maraging steel, as shown in Analyzing the values in The results obtained for spatial displacement, One of the causes of this difference is that Maraging steel is more sensitive to temperature variation than 42CrMo4 steel, as can be seen in the properties in In In On other hand, the spatial displacement decreased, as can be seen in The results obtained with maraging and 42crmo4 steels, In this section, the influence of fatigue on the gear obtained through the DED process will be studied. The DED process, used to reconstruct the gear, induces residual stresses in the part. To assess the impact of these stresses when conducting a fatigue test, a comparison was made with a gear without residual stresses.Fatigue has become increasingly crucial for technological advancement in automobiles, aircraft, compressors, turbines, and other systems subjected to repeated loading and vibration. Today, it is commonly stated that fatigue accounts for at least 90% of all mechanical service failures . FatigueThe fatigue problems of gears, one of the most widely used mechanical elements in transmission systems, limit the reliability and longevity of machines, especially the typical high-cycle-fatigue machines, such as wind turbines and high-speed trains. Gear fatigue represents a gradual deterioration process of material properties and the continuous accumulation of damage. Cracks induced by damage accumulation eventually lead to final failures, such as pitting, spalling, or tooth failure .Martukanitz and Simpson  reportedb is the fatigue strength exponent, and Fatigue loading with constant amplitude (or constant amplitude loading) is where all load cycles are identical. The Basquin relation can describe the stress–life relationship under high-cycle (or stress-controlled) fatigue conditions, where the significant controlling parameter is the elastic strain or stress level : (3)σa=Δb, for Inconel 625 were obtained for the S–N curve and are shown in the following relation [The fatigue strength coefficient relation : (6)σa=(To perform stress simulation of the gear, a force was applied, as shown in The values corresponding to the range, mean, amplitude of stress, and the number of cycles to failure for the Mises and principal stresses are grouped in A comparison of the maximum stress must always consider the same node. However, the node where the stress is maximum at the end of a simulation does not correspond to the node where the stress is maximum at the beginning. Therefore, two distinct nodes were considered: one corresponding to the maximum stress at the beginning, and the other to the maximum stress at the end of the simulation. A fatigue simulation of a gear without previous residual stresses was also carried out; the results are also shown in The DED and PBF processes are the predominant technologies used worldwide, for that reason it is important to study the mechanical behavior and structural integrity of parts obtained using these two processes.Regarding the NIST bridge benchmark, the objective initially set was achieved, since the results obtained in the simulations were similar to the results measured experimentally. The case of the benchmark bridge was also a way of verifying and validating the work developed.This parametric study of the use of the DED process in gear repair was motivated by the GEAR3D project. Despite the many parameters existing in the DED process, only five were analyzed, thus representing a limitation of this work. The choice of mesh for the remaining simulations required a compromise between the computational cost and the accuracy of the results obtained. Although the scanning strategies chosen only differed in the deposition track angle, it was possible to conclude that the choice of an appropriate scanning strategy allowed reducing the residual stresses and therefore optimized the part manufacturing process.This parametric study also concluded that the preheating temperature is a parameter that strongly influences the values of residual stresses induced in a manufactured part, with variations that can reach 500%. Regarding the study of the time step influence, there were no significant differences in the results obtained, mainly in the residual stresses. These results did not correspond well to what was expected, so a more detailed analysis could be carried out to understand the progressive activation of the material and its influence on the results.Finally, taking into account the results obtained for the influence of the use of different materials, it was possible to verify that the residual stresses actually changed with the use of different materials. On the other hand, the spatial displacement decreased substantially. One of the reasons for this may have been related to the different expansion coefficients of the materials.The fatigue analysis concluded that when there are residual stresses from the AM process, the stresses have low amplitudes and high mean values, which can result in a longer life than without residual stresses, where there are higher ranges and lower average stresses. Thus, high residual stresses can lead to a redistribution of elastoplastic stresses, which turns out to be beneficial in terms of fatigue.However, there is still much work to be done in this field. Future research should analyze other parameters that influence the metal additive manufacturing process. Future work should also focus on validating the simulations performed using on-gear reparation.In conclusion, AM processes are complex, due to the number of parameters that can be changed and their impact on the obtained parts. This work significantly contributes to the knowledge of the influence of some of the parameters of the DED process. In addition, studying the fatigue of gears produced is an essential outcome in the perception of the influence of residual stresses induced during the gear repair process.


The Chemical Master Equation (CME) is a set of linear differential equations that describes the evolution of the probability distribution on all possible configurations of a (bio-)chemical reaction system. Since the number of configurations and therefore the dimension of the CME rapidly increases with the number of molecules, its applicability is restricted to small systems. A widely applied remedy for this challenge is moment-based approaches which consider the evolution of the first few moments of the distribution as summary statistics for the complete distribution. Here, we investigate the performance of two moment-estimation methods for reaction systems whose equilibrium distributions encounter fat-tailedness and do not possess statistical moments.We show that estimation via stochastic simulation algorithm (SSA) trajectories lose consistency over time and estimated moment values span a wide range of values even for large sample sizes. In comparison, the method of moments returns smooth moment estimates but is not able to indicate the non-existence of the allegedly predicted moments. We furthermore analyze the negative effect of a CME solution’s fat-tailedness on SSA run times and explain inherent difficulties. While moment-estimation techniques are a commonly applied tool in the simulation of (bio-)chemical reaction networks, we conclude that they should be used with care, as neither the system definition nor the moment-estimation techniques themselves reliably indicate the potential fat-tailedness of the CME’s solution. Randomness and uncertainty govern nearly every aspect of our life. This especially also holds for the evolution of (bio-)chemical reaction networks (CRNs). As the quality and quantity of biochemical data have increased dramatically over the last decades, modeling a process’s noise has simultaneously become a valuable part of research . AccordiHowever, recent results indicate that biochemical stochasticity is often significantly less well-behaved than previously assumed. In biological settings, but also in nearly every other stochastic context, a very popular coping strategy for randomness is based on the formulation of expectations. Generally, an expectation is a deterministic way of describing a stochastic process by calculating the average of several of its outcomes. Mathematically, the notion of expectation manifests in the form of statistical moments like the expectation value and the variance. These quantities, as well as their estimation, are well-understood. Furthermore, their use in describing stochastic processes is provably founded by the central limit theorem: For processes of finite expectation and variance, the theorem states that the average over several independent realizations converges to a normally distributed random variable with a shrinking variance for a growing number of averaged samples. While the central limit theorem is remarkable in every aspect, its implications are still limited to processes that are expectable in the notion of statistical moments.Wildly stochastic behavior has been described, i.e. in financial sciences, where 63% of financial returns of half a century can be associated with only 10 trading days . So far,Before constructing a wildly stochastic CRN, we first need a modeling framework to describe its general behavior. The Chemical Master Equation (CME) is a standard approach for this purpose see e.g. . Its solWe will nevertheless focus on the two fundamental approaches since most other approaches pose additional conditions to the CRN and are therefore not necessarily better suited to answer a scientific question at hand. In this work, SSA sampling and the MoM are applied to a CRN that converges to a power-law distribution without existing moments like the one presented in N molecule types that participate in J reactions. The configuration space of such a system is a set of system configurations described by a vector i. A reaction j is assigned a configuration change vector j fires once, and a propensity function j fires in an infinitesimal time instance We consider a CRN with We are particularly interested in the time-dependent first two moments of the probability distribution, the expectation value, and the variance, which are defined viaThe somewhat lengthy limit formulation stresses that this indeed is a series for which convergence is not trivially given. Throughout this article, we refer to distributions without converging moment series as fat-tail distributions. This term is ambiguous and our definition is one of the more restrictive ones. We especially would like to distinguish this definition from heavy-tailed distributions, as many researchers use this term also for the log-normal case that we associated with slow randomness in the introduction. Two complementary approaches approximate k moments from the CME. These equations are not closed for systems with reactions of order two or higher, since lower-order moments depend on higher-order moments in these cases. We use truncation closure (TC) to solve this problem, a closure method that neglects the influences of higher-order moments on the considered set of moment equations. Unsurprisingly, it is, therefore, one of the most simplistic approaches, and we refer the interested reader to more sophisticated closure techniques that have been proposed to break this infinite hierarchy -based approximation of the underlying quantity due to TC.In order to study the behavior of both moment-estimation methods for systems that exhibit wildly stochastic behavior, we inversely design such a system. To this end, we consider the steady-state condition of the CME, which is obtained by setting the time derivative in Reformulating our original goal, we would like to define a CRN whose equilibrium distribution m are defined by divergent sums,However, all moments of natural order This holds regardless of whether one is interested in centralized moments or non-centralized counterparts.The distribution defined in Here, x.By replacing all three instances of In summary, we defined a CRN whose equilibrium CME solution is a discrete fat-tail distribution: it neither possesses an expectation value nor any higher-order moment.A to the system. The resulting configuration transition graph of this modified system, which we refer to as System Our system is unphysical in two aspects. First, defining the propensities for second-order reactions according to the law of mass action, one counts the number of educt combinations, which is given by An obvious question at this point is whether System To put our findings into context, all simulations are similarly performed for a reference System i denotes one of the n independent SSA trajectories over which we average. n in most scenarios. While this expectation is met for System From the sampled trajectories of the CRN, one can of course still attempt to estimate the distributions’ statistical moments using the bias-free estimatorsfor the expectation values System P , the sambehavior : Since sSummarizing, A reviewer brought another interesting scenario to our attention. By choosing For From a conceptional viewpoint, the CRNs resulting from the two choices of reaction rate parameters differ substantially: Setting As indicated earlier, we are interested to test how the MoM behaves when applied to the two chemical Systems Similarly, the MoM ODEs for System The just-presented equations are complemented with the deterministic initial conditions A qualitatively very different picture is painted for System W . InitialThe second observation in Similar to In summary, results show that the application of the MoM to CRNs that show wildly stochastic behavior provides smooth moment estimates that are similar to moment courses of systems with well-defined moments. Hence, MoM moment trajectories are not indicative of fat-tailedness. In contrast, the consistency of moment estimates from SSA trajectories is lost over time, and single SSA trajectories with large molecule numbers dominate the estimates of both moments. A visual indicator of this is the increasing ruggedness of the estimated moment trajectories over time.Our results outlined the difficulties of identifying and predicting wildly stochastic CRNs. The two challenges clearly are related, as the identification of fat-tailed CME solutions is the basis for a subsequent adequate choice of approximation methods. In this chapter, we would like to present a diagnosis approach while simultaneously discussing another interesting aspect of wildly stochastic behavior in the context of CRNs.j of the next reaction and a waiting time To this end, we revisit the SSA used to draw sample trajectories from the CME solution. Starting from the initial configuration The number of needed SSA steps to reach 5 s of simulated time is displayed as a histogram over all ems in P , the hisThe second aspect worth discussing is the long histogram tail in The number of performed SSA steps is directly correlated with the simulation run-time. A scientist attempting to draw samples from a fat-tailed CME solution will therefore experience many very fast simulations at first. However, the overall run-time is completely dominated by single extreme events. The simulation of trajectory The CME is widely recognized for accurately describing the behavior of CRNs of various different types and characteristics. Due to this flexibility, it is in principle not surprising that the CME solution of particular systems does not possess any statistical moments. However, it still is counter-intuitive that those systems can be derived from simple mass action kinetics of second-order reactions.In this work, we first derived a CRN that provably converges to a fat-tail power-law distribution. After slightly altering the system to increase interpretability, we numerically validated our theoretical claims. On this basis, we presented evidence that expectation-based CME approximation methods are prone to fail when applied to systems that converge to fat-tail distributions. Moreover, the MoM does not indicate its failure by any means. In our eyes, it is, therefore, crucial to examine each CRN thoroughly before choosing an adequate approximation technique for the CME. To this end, we propose to employ a sample of SSA simulations. The corresponding implementation is simple, yet, SSA trajectories represent a true sample of the underlying CME that can be used in different ways for a moment diagnostics. As demonstrated, applying moment estimators to the sampled trajectories and observing their convergence can indicate whether or not statistical moments of the CME solution exist and how easily they can be estimated. Another interesting aspect of the SSA is that its run-time scales linearly with the number of recorded reactions. In the context of our simulations, extreme events that effectively prevent the CME solution from having statistical moments correspond to simulations where the system escapes the slow dynamics of a few molecules, thereby further increasing the reaction propensities quadratically. As indicated earlier, this vast amount of reactions manifests itself through surprisingly large simulation run-times. Effectively, the fat tail of the CME solution is therefore transferred to the distribution of simulation run-times, which is a quantity most simulation researchers attentively monitor and which therefore serves as another criterion to assess the system’s expectability.Beyond the methods that we have used in the article, other diagnostic methods to characterize the CME solution’s moments are conceivable. For example, traditional outlier testing like Chauvenet’s or Peirce’s criterion and Grubbs’s test could be applied to SSA simulations to indicate extreme results.


Brood X is the largest of the 15 broods of periodical cicadas, and individuals from this brood emerged across the Eastern United States in spring 2021. Using distributed acoustic sensing (DAS) technology, the activity of Brood X cicadas was monitored in their natural environment in Princeton, NJ. Critical information regarding their acoustic signatures and activity level is collected and analyzed using standard outdoor-grade telecommunication fiber cables. We believe these results have the potential to be a quantitative baseline for regional Brood X activity and pave the way for more detailed monitoring of insect populations to combat global insect decline. We also show that it is possible to transform readily available fiber optic networks into environmental sensors with no additional installation costs. To our knowledge, this is the first reported use case of a distributed fiber optic sensing system for entomological sciences and environmental studies. Magicicada  is a new technology that transforms fiber optic cables into sensing elements . HistoriMagicicada cicadas in situ along a suburban stretch of Central New Jersey over a 16-day period during their 2021 adult cycle. The author’s sampling near this location in both Princeton Junction and Princeton recovered all 3 species, so it is likely that all 3 species was recorded in our study area. Using the DAS data collected from the field, the spectral patterns of cicada sounds were evaluated and a direct relationship between these calls and ambient temperature was observed. We propose the use of DAS as a potential tool for remotely monitoring insects in situ wherever fiber cables are available.In this work, a DAS system was used to interrogate standard telecommunication grade outdoor fiber cables installed in our testbed to monitor Brood X When a highly coherent optical pulse travels along a fiber cable, a small fraction of it backscatters toward the source due to the inherent nonuniformity centers randomly distributed along the fiber cable . By moniMoreover, if there is a strong enough external perturbation, such as an acoustic event, at or near any point along the fiber cable, the generated acoustic pressure wave will cause a time-varying small strain on that point of the fiber cable. These changes in the strain will modulate the local refractive index of the fiber at that point, which in turn will modulate the optical phase of the light backscattered from that point . Since tA simple operation principle of the DAS system is demonstrated in To monitor Brood X cicadas in their natural environment, we used a standard 36-strand single-mode outdoor-grade figure-8 telecom fiber cable with a 0.25-inch messenger installed in an outdoor testbed. The testbed consists of 3 Class-II-type 35-feet tall utility poles in a linear arrangement and 90 feet apart from each other. The telecom fiber is connected to a DAS system located inside the lab; the first 340 m of the fiber cable is underground, then it rises along the first pole and becomes aerial, and at the third pole it goes underground again. The overall schematics of the testbed with the fiber route and a picture are shown in The DAS system used to monitor the fiber route is a commercially available NEC LS3200 model distributed acoustic sensor. During the data collection, the optical pulse width was chosen as 8 ns, and the spatial resolution was set at 1.2234 m. The optical pulse repetition rate was chosen as 20 kHz for short-time duration acoustic recordings, and 2 kHz for long-time duration data recordings to manage the data size. The DAS data were collected from 9 June to 24 June 2021 intermittently. Using the DAS system, the acoustic signal of the cicada group  was recorded and analyzed in situ.As discussed above, the DAS system can detect the acoustic signals along the whole fiber route provided that they are loud enough or close enough to cause minuscule modulations on the optical fiber. For the acoustic analysis of the cicada sounds, a single point along the fiber was selected based on the detected signal quality and power. The selected location is at the 345-m point along the fiber, which corresponds to a fiber coil at the first utility pole; hence, it is an aerial point and collects the cicada data accordingly. As expected, fiber coils are more sensitive and are preferred to detect external vibrations. This fiber coil was highlighted by red circles in Cicada signals were recorded successfully over the duration of the experiment. Lastly, Due to the large coverage of the fiber, the system can record multiple locations simultaneously as discussed earlier. However, depending on the cable installation and location, each point of the fiber may not have the same sensitivity to external disturbances; hence, fiber coils are more sensitive than linear cables. The aerial part of the fiber detected and recorded the collective sound of the nearby cicada group, i.e., the cacophony of the cicada deme/population, instead of the sound of an individual cicada. In contrast, the underground cable is acoustically insulated, and the cacophony sound of the cicada population cannot reach the underground fiber to generate a detectable signal.As noted above, the DAS system monitored the testbed for a temporal duration of 16 days, intermittently from 9 June to 24 June 2021. For this long-term data collection, we used a sampling frequency of 2 kHz to reduce the data size. Even though the corresponding Nyquist frequency is lower than the cicada peak frequencies and causes aliasing, the energy of the cicada group is still present in the recorded signal and its frequencies can simply be recovered by the aliasing frequency formula. The day markers correspond to the noontime of each day, and as seen in this figure, the peak frequency of the cicada group varies from a maximum of 1,500 Hz to a minimum of 1,200 Hz in that time frame. When comparing Male cicadas produce sound using a tymbal organ, as shown in Using the DAS system as a long-term monitoring tool, not just the frequency but also the total sound energy of the cicada population was calculated as a function of time as shown in Indeed, the active calling period for the periodical cicadas is relatively short, compared to their 17-yr juvenile stage, lasting typically 2–3 wk in duration. Cicadas emerge after the juvenile period underground and usually take between 4 and 5 days to begin chorusing depending on the ambient temperature. The chorusing period ranges from approximately 1 month. Here, the waning sound signal power in their chorusing toward the end of the reproductive period was recorded. The females lay eggs by making slits in the branches of trees using an egg-laying apparatus called an ovipositor. After a period of egg laying, the females die. The eggs hatch 6-8 wk later, and nymphs drop to the soil to begin their 17-yr juvenile period . We hopeOur results show the feasibility of in situ monitoring cicadas via DFOS technology. We want to stress that all the results presented in the paper are obtained by the use of standard outdoor telecommunication fibers, and not some specialty fibers. Moreover, even the fiber cables in our testbed are installed in the same way as in actual field fibers. Hence, our proposed method purely relies on readily existing fiber cable networks and does not require any special fibers nor a specific way of fiber installation in the field. We hope that it may be beneficial for long-term monitoring of cicadas in large areas and initiate a baseline for a more quantitative analysis of the insect decline. This novel use case also allows one to map cicada activity in real-time along an interrogated fiber route. Future studies should compare cicada activity using these fibers across their geographic range and should test whether the activity of single individuals can be recorded over time via underground fibers.


The purpose of this study was to introduce the application of a monoportal scope and bipolar coagulator used in full-endoscopic spine surgery (FESS) for unilateral biportal endoscopy-unilateral laminectomy bilateral decompression (UBE-ULBD) in those with central stenosis. A 68-year-old man who presented with cauda equina symptoms underwent UBE-ULBD to improve his central stenosis at the L2/3 level. In this technique, a FESS scope was attached to a camera portal in place of a common arthroscope. A decompression tool was subsequently inserted through the working portal, and the lower border of the vertebral lamina and the lower border of the contralateral lamina were resected. Additionally, the superior border of the L3 level was thinned using a high-speed drill, and the ligament flavum was excised. The operation time was 70 minutes, and his symptoms improved. The patient was discharged from the hospital four days postoperatively. We found three advantages of using a FESS scope and bipolar coagulator, including the ability to 1) stabilize the camera via placement of the sleeve against the bone, 2) minimize the wounded area by irrigating saline on the side of the scope, and 3) provide bipolar tissue hemostasis in an isolated area around the nerves. Therefore, among the UBE techniques, we believe that assisted full-endoscopic spine surgery (AFESS) is a viable option to offer a more minimally invasive surgery for patients with stenosis. With the rapid development of arthroscopic surgical techniques for patients with stenosis, technical advances have offered a higher degree of freedom due to the use of two independent portals for viewing and working. The improved surgical field and unrestricted working space provide a useful means for decompression surgery.Unilateral biportal endoscopy (UBE) has been shown to decompress a greater area in the same amount of time compared to conventional micro-endoscopic and full-endoscopic spine surgery (FESS) decompression techniques ,2. On thA 68-year-old man complained of intermittent claudication. MRI revealed central stenosis with thickening of the ligamentum flavum and protrusion of osteophytes at the L2/3 level. Figures -1C BecauA 2002 report by Yeung suggested that a bi-portal approach may be used instead of a single-portal if biportal access is required . After pField of view and camera stabilization via placement of the sleeve against the boneCurrent commonly used UBE scopes have originated from arthroscopes and have been designed to utilize a fixed sleeve that is the same length as the scope. There is no fluid irrigation area from the side of the scope. Under normal intraarticular usage of arthroscopes, the surroundings of joints are composed of cartilage; therefore, to prevent cartilage damage, arthroscopes are not usually designed to come into contact with surrounding tissues. When used in the spinal region, the need arises to hold and suspend the scope to secure the field of view. As a result, it becomes necessary to first treat the muscle layer, and there is concern that more damage could occur to the soft tissue layer. In addition, visualization will be unstable due to the shaking of the camera. On the other hand, the FESS scope can be used within the confines of a sleeve, and it is possible to stably hold the camera by applying the beveled part of the sleeve to the bone. Moreover, tissue damage may potentially be reduced as the sleeve can be directly applied to the bone to ablate the soft tissues. From the above, we believe that the use of the FESS scopes is a useful option for spine surgery to secure a stable field of view. In addition, as shown in Figure Fluid irrigationContinuous fluid irrigation is essential to maintain a clear surgical field. The UBE technique using an arthroscope often requires enlarging the working portals to ensure irrigation. The FESS scope can be combined with a sleeve that allows the insertion of surgical instruments while providing continuous fluid irrigation. Therefore, there is no concern regarding fluids being discharged from the working portal. In addition, the skin incision is also small.HemostasisPlasma-based radiofrequency (PRF), commonly used in UBE, conducts current from the tip of the coagulator to the dorsal side . Plasma Compared to scopes used in arthroscopy, the diameter of the 6.9 mm scope used in this study is larger. Therefore, when performing treatment on the contralateral side via the subspinous process, decompression is required to secure the visual field. Considering the possibility that more decompression may be required than is necessary for treatment, a narrower scope may be more useful in cases that necessitate a low-profile scope.In this technical report, we described the use of FESS scope and HFB in a patient with central stenosis. We were able to obtain a stable visual field, provide bipolar tissue hemostasis in an isolated area around the nerves, and minimize the skin incision by improving fluid irrigation. The UBE technique using an arthroscopic device is considered to be an extension of FESS surgery, and assisted full-endoscopic spine surgery (AFESS) will further develop in combination with FESS technology in the spinal region.


Object detection algorithms require compact structures, reasonable probability interpretability, and strong detection ability for small targets. However, mainstream second-order object detectors lack reasonable probability interpretability, have structural redundancy, and cannot fully utilize information from each branch of the first stage. Non-local attention can improve sensitivity to small targets, but most of them are limited to a single scale. To address these issues, we propose PNANet, a two-stage object detector with a probability interpretable framework. We propose a robust proposal generator as the first stage of the network and use cascade RCNN as the second stage. We also propose a pyramid non-local attention module that breaks the scale constraint and improves overall performance, especially in small target detection. Our algorithm can be used for instance segmentation after adding a simple segmentation head. Testing on COCO and Pascal VOC datasets as well as practical applications demonstrated good results in both object detection and instance segmentation tasks. Object detection plays a crucial role in robotics. For instance, in the context of household serving robots, achieving an accurate and reliable grasp of objects requires the robot to be able to acquire the precise locations of objects . Object Object detectors generally fall under two categories, namely, two-stage object detectors and one-stage object detectors. Standard two-stage object detectors locate all possible object positions by maximizing the recall rate in the first stage but identify objects within these positions based on their likelihood scores. The optimization objectives in the two stages are distinct, which results in a lack of probabilistic interpretation and structural redundancy in standard second-stage object detectors. One-stage detectors maximize the likelihood of annotated ground-truth objects during the training stage and rely on the likelihood scores as the basis for inference. They are a probabilistically sound framework but the problem of insufficient accuracy may arise due to the impact of imbalanced positive and negative samples. CenterNet2  modifiedThe precise detection of small targets is another important issue in the field of object detection. There have been numerous works aiming to solve these problems. Feature pyramid network (FPN)  is the pIn this paper, we proposed a probabilistic two-stage detector that has a reasonable probability interpretation and a compact structure, enabling accurate object detection. Upon the integration of a simple segmentation header, our detector further achieves precision instance segmentation. Notably, our detector exhibits notable control over details, thereby demonstrating exceptional performance in detecting objects.Specifically, we first introduced a robust single-stage object detector as a replacement for the region proposal network (RPN) in standard two-stage detectors. We trained both stages simultaneously to maximize the likelihood of ground-truth objects, which is then used as the detection score during inference. Secondly, we enhanced the method of ground-truth matching and improved the first-stage proposal generator by coupling the classification branch with the box generation branch and incorporating a better prior for the box regression branch. This resulted in a more stable first stage and a more comprehensive probability interpretation. Thirdly, we proposed an effective pyramid non-local attention (PNA) module, we incorporate the non-local attention mechanism into FPN to capture non-local dependency across multiple levels and embed a pyramid sampling module into every non-local block, which significantly reduces computational overhead while preserving semantic features. Finally, we made minor modifications to BiFPN, resulting in improved accuracy. Our main contributions can be summarized as follows:1. We built a probabilistic two-stage detector that achieves higher accuracy with a more reasonable probability interpretation.2. We proposed a strong proposal generator by coupling different branches and providing a prior for box regression. This makes the first stage more stable and interpretable, thus improving the overall accuracy of the network with almost no cost.3. We proposed a pyramid non-local attention(PNA) module, which enhances the network’s ability to extract detailed features, ultimately significantly improving its detection capabilities for objects, especially for small objects.The rest of this paper is outlined as follows. In Object detectors:Two-stage detectors, such as regions with CNN feature (RCNN) series ,16,17, eOne-stage detectors, such as the you-only-look-once (YOLO) series ,28,29,30In recent years, there has been a high level of research interest in visual transformers. The visual transformers (ViT)  algorithFeature pyramid: The utilization of a feature pyramid can enhance the network’s resolution, improving the detection accuracy of small objects. One of the primary challenges is to efficiently encode and handle features across multiple scales. FPN  proposedAttention mechanism: Attention mechanism plays an important role in human visual perception. In 2017, Vaswani et al.  introducRecent studies have also focused on how to make sufficient use of long-range dependencies. Wang et al.  proposedThe architecture of our proposed object detector is shown in Based on these features, we then use a robust proposal generator to generate a series of proposals, which will also be detailed in later sections. The proposals generated by this module are then fed into the cascade heads, which consist of three heads that use different thresholds for bounding boxes regression and filtering, to obtain the final results.Our probabilistic interpretable framework draws inspiration from CenterNet2 . The aimith candidate object belongs to the cth classPCi=cEquation :(2)PCi=cWe used maximum likelihood estimation to train our detectors in our framework for annotated objects; our goal is to maximize the log-likelihood like Equation :(3)logPCThe two terms in the above formula correspond exactly to the first and second stages of the detector, respectively. For the background, the maximum-likelihood goal should be Equation :(4)logPCHowever, this objective involves both stages and it does not factorize. In practical applications, it can cause difficulties in back propagation of gradients. Using Jensen’s inequality as in Equation :(5)logαxEquation :(6)logP(It is a tight bound when Equation :(7)logPPOk=1With Equations  and 8),,8), our Our feature fusion section references EfficentDet  and makeSimilar to EfficentDet, our feature pyramid is composed of a single block repeated multiple times. The size of each feature map is half of the size of the previous feature map, and all feature maps have the same number of channels. In this paper, we use two forms of feature pyramid: three-layer and five-layer; the blocks that make up them are shown in In terms of feature fusion, we take a five-layer feature pyramid’s block for example. Equation ; there wAs shown in X and Y; then, we summed them up. We used convolution layers instead of fully connected layers to embed features, thus reducing the computational cost. After two rounds of convolution, we obtained the feature W, which represents the importance of each channel. For regularization, we adopted the method of dividing all elements in W by the maximum value of W instead of using sigmoid, which also aims to reduce computational complexity. To clarify, channel attention is not applied to every repeated FPN but only appears in specific FPN modules, intending to balance accuracy and time. For the five-layer feature pyramid, this module only appears in the second block. For the three-layer feature pyramid, it appears in the second and fourth blocks.We apply the input to a max pooling layer and an average pooling layer separately, with the pooling operation performed along both the width and height axes, resulting in the extraction of features The pyramid non-local attention (PNA) module is the core module of our method, which effectively utilizes the multi-scale and multi-level features generated by the feature pyramid, and establishes dependencies between different locations based on this.Firstly, let us revisit the definition of non-local attention block, as shown in Equation :(10)M=NoY as Equation Y=CoY before performing convolution and resize operations is denoted as X is as Equation yi=1However, the information that can be obtained only by scaling the image is limited. Inspired by this method, as shown in Our detector will use up to five layers of the feature pyramid at most due to the high computational cost of the non-local attention mechanism,; if we directly calculate each point in each feature map, it will cause great computational cost. Looking back at the process of the non-local attention mechanism, we can see that Equations  and 11)11) are tIt can be noticed that the red-highlighted parts do not affect the size of the output s in the s neighborhood has changed.In our method, we use spatial pyramid pooling (SPP)  module, The structure of the entire PNA module is shown in The proposal generator in this paper integrates the advantages of various excellent algorithms. The structure of our proposal generator is shown in Firstly, for the generation of prior anchor boxes, we conduct k-means clustering on the bounding boxes in the training set to automatically find good priors instead of choosing priors by hand, which is similar to the YOLO  series. k boxes whose centers are closest to the center of ground truth box Regarding the allocation of ground truth boxes, we use adaptive training sample selection . At eachIn complex scenes, the mutual occlusion of objects and blurriness of the main image can lead to uncertainty in the borders, as shown in We denote the distribution we predict as F(x), where F(x) satisfies Equation ; in pracDuring training, we want Equation :(18)LossFor the heatmap branch, we use soft one-hot encoding to label the ground truth, which is different from the traditional method where the value of positive sample points is all 1 and the value of negative sample points is all 0. We assign a value of 0 < y ≤ 1 to the positive sample points, where y is the IOU score of the point, and the larger the IOU value between the anchor and the ground truth at the point, the larger the value of y. The advantage of this approach is that it establishes a connection between the position and the IOU, making the consistency of the network better during training and prediction. At the same time, positive samples with a higher ground truth IOU can contribute more weight, thereby improving the performance of the network.In the follow-up process, we will encode the distribution of the border distribution branch and apply the result to the heatmap. The specific process is shown in In this paper, we adopt cascade heads as the second stage of our detector, which decompose the regression of categories and bounding boxes into multiple stages; each stage takes the bounding boxes from the previous stage along with the feature map as inputs, and outputs the classification and a new distribution of bounding boxes. The detailed structure of cascade heads is illustrated in Regarding the bounding box regression part, it relies on a cascade of specialized regressors, as depicted in Equation .(20)f(xx represents the input feature map, and T represents the total number of stages. In this paper, we set f multiple times. The cascaded regression is a resampling procedure that changes the distribution of hypotheses to be processed by the different stages. Likewise, each regressor f in the cascade is optimized based on the sample distribution In this formula, As the number of regressions increases, the quality of the bounding boxes improves; in other words, the cascade regression begins with a set of examples g is the ground truth, As for the classification part, each cascade head has an independent classification branch with different parameters, which outputs the probability of the target belonging to each class. Unlike the bounding box regression part, the classification results of each stage are not affected by the results of the previous stage. The cascade heads is learned by minimizing the loss in Equation . where bDuring the prediction phase, we also couple the two stages. Specifically, the score of the final bounding box is obtained by multiplying the score of the first stage with the score of each cascade. This is one of the essential differences between our method and traditional two-stage object detectors, as the two stages of our detector are not separate.To demonstrate the effectiveness of our algorithm, we conducted comparisons with baselines and ablation experiments on the COCO dataset, including object detection and instance segmentation tasks, and provided detailed explanations for the performance of each part of our algorithm. We also compared our algorithm with state-of-the-art algorithms on both the COCO  and PascThe architecture of our method is inspired by CenterNet2, so we used CenterNet2 as the baseline for comparison. As mentioned earlier, the core difference in the structure between our method and CenterNet2 as well as other two-stage object detectors lies in the generation and use of proposals in the first stage. Our algorithm can be seen as CenterNet2 with a replacement of the proposal generator, the addition of the PNA module, and modifying part of the FPN. This experiment was conducted on the COCO dataset, and all methods used DLA as the backbone and a three-layer FPN for feature fusion. All methods were trained for 60 epochs, using 0.02 as the base learning rate and 640*640 as the base training size. No data augmentation methods were used except for random cropping and random resizing. The training and evaluation were performed on Intel Xeon6130 processor and a single TITANxp GPU with PyTorch 1.10.0 and CUDA 10.2. The ablation experiment results for object detection tasks are shown in Comparing the results of object detection, it can be seen that our algorithm has a significant advantage in accuracy compared to CenterNet2. Our method can better detect small targets and capture details. Further analysis of the table shows that the PNA module contributes the most to the algorithm’s performance, followed by our robust proposal generator. Although the channel attention mechanism module we designed has the smallest contribution to the overall accuracy improvement, it hardly affects the efficiency of the algorithm.To further explore the principles of our various modules, we conducted the following work. As mentioned earlier, the reason why the PNA module can significantly improve the algorithm’s performance is that it can establish feature connections between long distances and different feature layers, allowing the network to better focus on important information and restore details. We separated the feature maps output by the feature pyramid during the prediction process, selected the k channels with the highest activation in the feature maps, generated a heatmap, and superimposed it on the original image, as shown in The detection performance of our algorithm and its comparison with CenterNet2 are shown in The advantage of our robust proposal generator compared to RPN and other proposal generators is that it can generate higher-quality proposals. Specifically, it generates fewer proposals, but with a higher IOU with the ground truth, as shown in In addition, thanks to the coupling of the bounding box distribution branch and the heatmap generation branch, we fully utilized the distribution information of the bounding box. It can be seen that when multiple targets overlap and the target bounding box is blurred, our false positive rate is significantly lower, and the bounding boxes we regress are more reasonable, as shown in After adding a simple segmentation head, the method we constructed can complete the instance segmentation task. Further comparison with CenterNet2 was conducted with the same segmentation head on the COCO dataset instance segmentation task under the same training environment. The experimental results are shown in In addition to conducting experiments on object detection tasks on the COCO dataset, we also conducted experiments on instance segmentation tasks. The training strategy and environment were the same as those for object detection tasks. Although we only added a simple segmentation head based on the original algorithm, our algorithm still performed better than current mainstream algorithms when using the same backbone. The comparison results are shown in Our first robot platform, as shown in We built our own dataset by combining the actual working scenarios of the robot with the target to be grabbed. The dataset consists of 2300 images, 3443 instances totally, with 1800 images for training and 500 images for testing. The performance of our method on the dataset is shown in The application of our algorithm in actual scenarios is shown in Our second robot platform, as shown in Similarly, we have also built a dataset of rebar intersection points, with 260 images used for training and 50 images used for testing, 9150 totalling instances. As shown in The performance of our algorithm in practical scenarios is shown in In this study, we proposed a probabilistic two-stage object detector. The detector has a relatively compact structure and a better probability interpretation, which leads to higher accuracy, stronger adaptability, and greater sensitivity to small objects. We proposed a strong proposal generator as the first stage of the detector. The generator uses a more reasonable ground-truth matching method and takes into account the case of blurred object boundaries. Its bounding box distribution branch is coupled with the heatmap branch, allowing the generator to make full use of various information. Our generator can generate proposals with scores that have higher IOU with ground truth. Furthermore, we proposed the PNA module, which combines the non-local attention mechanism with the feature pyramid. This module breaks the limitation of scale for non-local attention mechanisms and greatly enhances the detector’s ability to mine details and comprehend global semantic information. We also integrated the SPP module into the non-local attention mechanism to reduce computational costs.Subsequent experiments have demonstrated the superiority of our method. Our method achieved outstanding performance in both detection and segmentation tasks on the COCO dataset and outperformed most mainstream algorithms on the Pascal VOC dataset. Moreover, we applied our method to challenging scenarios in construction sites and demonstrated its excellent performance in completing various tasks. However, our algorithm still has certain limitations. Future research can explore how to compress the algorithm’s time to achieve more efficient object detection.


This research paper aimed to validate two methods for measuring loads during walking with instrumented crutches: one method to estimate partial weight-bearing on the lower limbs and another to estimate shoulder joint reactions. Currently, gait laboratories, instrumented with high-end measurement systems, are used to extract kinematic and kinetic data, but such facilities are expensive and not accessible to all patients. The proposed method uses instrumented crutches to measure ground reaction forces and does not require any motion capture devices or force platforms. The load on the lower limbs is estimated by subtracting the forces measured by the crutches from the subject’s total weight. Since the model does not consider inertia contribution in dynamic conditions, the estimation improves with low walking cadence when walking with the two-point contralateral and the three-point partial weight-bearing patterns considered for the validation tests. The shoulder joint reactions are estimated using linear regression, providing accurate values for the forces but less accurate torque estimates. The crutches data are acquired and processed in real-time, allowing for immediate feedback, and the system can be used outdoors in real-world walking conditions. The validation of this method could lead to better monitoring of partial weight-bearing and shoulder joint reactions, which could improve patient outcomes and reduce complications. Incremental weight-bearing is often required during post-surgery gait rehabilitation to reduce implant failures and improve soft tissue healing. The Partial Weight-Bearing (PWB) approach includes increasing weight loading on the limb progressively over time, which varies between patients depending on the extent of the injury and the judgment of the clinician ,2. WalkiForce platforms and motion capture systems are commonly used to extract kinematics and kinetics information of the gait also when performed with walking aids. Even if gait laboratories could perform highly accurate measurements , some liRecently, several wearable technologies have aimed to bring gait analysis outside laboratory conditions. Many instrumented insoles are used to collect data in daily life activities, and most of them allow real-time acquisition, but they usually require a preliminary calibration to measure forces accurately ,16,17. OIn this paper, we provide a preliminary methodological validation of an estimator of the partial weight-bearing of the lower limbs and the shoulder joint’s reaction forces during a walk with instrumented crutches. The estimate was provided by a pair of instrumented crutches to measure the ground reaction forces along their principal axis, and the forces were acquired and processed in real-time, which is suitable to collect data or provide feedback ,22,27,28The instrumentation used and described in the following chapter is summarized in p = 95%) [The instrumented crutches, shown in p = 95%) . The totBesides the instrumented cutches, the experimental set-up included two force plates from BTS Bioengineering to measure each foot’s ground reaction force and a marker-based motion capture system from Vicon Motion Systems with 8 cameras to measure the subject’s kinematics. The sampling frequencies were different for the three systems: the force plates acquired data at 1 kHz, the motion-capture system acquired data at 100 Hz, and the crutches acquired data at 60 Hz, so a resampling procedure was required before starting data processing.The Vicon Full Body Plug-In Gait model was used for the placement of the markers on the subject’s body . Two addA custom-made trigger box was used for data synchronization between the motion capture system, the force platforms, and the instrumented crutches . The intVicon’s software Nexus was used to start and stop the acquisition of all the devices involved in the experiment. The analog output channels of the Vicon Lock control box were connected to the BTS’s trigger box and the instrumented crutches’ trigger box. When the Vicon acquisition began, the control box generated a rising edge on the outputs. On the rising edge, the BTS’s trigger box enabled the acquisition from the force plates, and the instrumented crutches’ trigger box sent the start command and its timestamp through the ROS network. The instrumented crutches streamed the data once they received the start command. With a similar approach to the start, the control box generated a falling edge to stop all the acquisitions.Please note that this configuration was required for validation only; when the instrumented crutches are not used simultaneously with other devices, they can be controlled independently.The experimental protocol aimed to acquire data useful for validating the proposed PWB and shoulder reaction estimators. Since the effect of the gait dynamics was somehow neglected in the model that determined the PWB from crutches’ forces, the protocol included three different pacing conditions during walking: normal or subject’s preferred rhythm and slow and fast cadence rhythms. These different conditions enabled the verification of the dynamic effects of the proposed approach. A metronome helped the subjects follow a cadence of 50 or 90 steps/min, with the goal of covering reasonable boundaries of a walking cadence with crutches ,35. SubjFinally, the protocol required two PWB levels in each condition. To achieve weight-bearing values on the lower limbs comparable with those commonly prescribed by clinicians, the subjects must load 20% or 40% of their body weight on the crutch. In this way, during the 2-point contralateral pattern, the PWB should stay between 60–80%BW, and during the 3-point PWB pattern, it should stay between 20–60%BW. In order to verify subject repeatability, at least three trials were conducted for every condition. In addition, before starting the trial sequence, a calibration procedure was necessary to calibrate the motion capture model and properly adjust the force plates to zero. A training session of about 10 min was performed before these calibration procedures to let the subjects familiarize themselves with walking patterns, cadences, and crutch loads.The experimental protocol was applied to 6 subjects, 5 male and 1 female with a weight of 78 ± 12 kg, a height of 1.77 ± 0.03 m, and an age of 38 ± 12 years old (mean ± STD). Since at this stage, we were interested in the validation of the proposed approach and not in clinical validation, we performed the tests on healthy subjects only. The study was conducted in accordance with the Declaration of Helsinki, and informed consent was obtained from all subjects involved.Since during assisted walking, the external forces applied to the subject are the ground reaction forces on the feet and on the crutches, the authors believed that the load supported by the user’s lower limbs could be estimated by subtracting the left crutch force (LCF) and the right crutch force (RCF) from the total subject body’s weight (BW), which is the sum of the instrumented crutches’ weight and the user’s weight. The load is then normalized and expressed as a percentage of the total subject’s weight:The reference values are obtained from the sum of the left platform force (LPF) and the right platform force (RPF), normalized by the total subject’s weight:The difference between the two values is considered the error:The error was calculated when all the external forces were known, and the validation intervals were obtained from the gait events recorded by the motion capture. The valid interval started after the last toe-off event outside the force platforms, shown in Of course, the inertia contribution in dynamic conditions could worsen the results, so a specific testing protocol was proposed to verify this point, varying the user’s walking pace and the crutch loads. The root-mean-square error (RMSE) was used to measure the difference between the values predicted by the estimator and the values observed. The mean error (ME) was used to measure the systematic bias since the RMSE included both stochastic and systematic errors.Shoulder reaction reference values were obtained by the biomechanical model outcomes described in detail in . The modAs previously described in the introduction, the purpose of the shoulder loads and torque regression model is to obtain their estimates without measuring upper limb kinematics. In such a way, the biomechanical model is not required, and an online, rough indication of shoulder loads and torque can be directly given to the patient and to the physician. Since the biomechanical model does not need the ground reaction forces on the feet, any gait cycle is valid to create the regression dataset to compute the shoulder values.The wide dataset was then divided into identification and validation datasets. For this separation, we considered that, in accordance with the test protocol, each condition was repeated at least three times, so one trial was randomly assigned to the validation dataset, while the remaining trials were assigned to the identification dataset. With this procedure, we maintain a balanced representation of the population and conditions between the datasets with good numerosity: 2/3 for the identification and 1/3 for the validation. The model estimated the peaks and the root mean square (RMS) of the shoulder torques and forces, calculated by the biomechanical model in every stance phase of the crutches. Each subject performed at least two stance phases of the crutch on both sides in every trial.Since the purpose is to give an online indication to the patient, the time histories of the loads at the shoulders are not required, while peak and root mean square values are certainly more informative. The corresponding values were computed for each crutch cycle and the force is normalized by the subject’s weight while the torque is normalized by the subject’s weight times the subject’s height. In the first regression attempt, we considered forces in the sagittal plane  and the torque exerted along an axis perpendicular to this plane . In fact, these directions were the most significant in a previous study , as theyIn The PWB’s reference, estimate, and error are shown with respect to the time in According to the procedure described in Moreover, in order to maintain a clear physical correspondence of the model, we considered the same predictors set for both force and torque, so we assume that this parameter set is significant for the general shoulder load.The presented approach estimated the partial weight-bearing from the measurement of a pair of instrumented crutches. The ANOVA results of Moreover, if the gait phases are detected, both cadence and gait support are known, and the PWB estimation can be corrected by removing the systematic error. There are several devices that can be used to assess gait phases, but they may have limitations. IMUs or instrumented insoles require the patient to wear the device and sometimes require calibration. Fixed RGB or infrared cameras must be installed in some motion capture systems, which limits their use to specific locations. These devices also require a post-processing stage sometimes and cannot provide real-time feedback. While our instrumented crutches may provide gait phase detection, the process still needs to improve the reliability and elaborate data in real-time ,30.In the three-point PWB gait pattern, the PWB estimation during the single support of the unaffected limb was not useful for rehabilitation purposes, and the crutches were lifted from the ground most of the time with saturation at 100%BW of the prediction. As shown in 2 about 1), while it was more critical for torques (R2 about 0.6). The residuals for the RMS and peak force regression models followed a normal distribution, as shown in the normal probability plots in The shoulder regression model was based on RMS and the peak values for crutch force and their interaction with the subject’s height and squared height. The regression was good regarding shoulder force  but was less accurate for torque (R2 about 0.6). The less normal distribution of the torque residuals was possibly due to uncertainties in the anthropometric parameters used in the inverse dynamics equations, and a more complex regression model would improve the correlation.The study also developed a regression model for estimating shoulder force and torque during crutch-assisted gait. The model is based on RMS and peak values of crutch force and considers the subject’s height and squared height. The regression model provided good estimates for shoulder force (RThe main goal was to conduct a preliminary methodological study to validate the load estimator applied on the legs using instrumented crutches for force measurement. The inclusion of a larger sample size and a more balanced gender distribution could have provided more generalizable results. Nevertheless, this preliminary study provided promising results that demonstrate the validity and effectiveness of the load estimator system. In the future, studies should include a larger number of participants and adequate gender representation to confirm and strengthen the preliminary findings. Additionally, future studies should consider testing the system under different walking conditions, such as different ranges of speeds, inclines, and terrain types to further validate its accuracy and effectiveness. From the results obtained, it can be assumed that cadence has more effects on the PWB estimation, and it should be investigated more deeply, while the crutch load could be self-selected by the subject or indicated by the therapist since it has less influence. Furthermore, the estimator should also be checked with different impairments since the deviations in the gait could alter the predictions.