Tick-borne diseases are among the challenges associated with warming climate. Many studies predict, and already note, expansion of ticks’ habitats to the north, bringing previously non-endemic diseases, such as borreliosis and encephalitis, to the new areas. In addition, higher temperatures accelerate phases of ticks’ development in areas where ticks have established populations. Earlier works have shown that meteorological parameters, such as temperature and humidity influence ticks’ survival and define their areas of habitat. Here, we study the link between climatic parameters and tick-related hospital visits as well as borreliosis incidence rates focusing on European Russia. We have used yearly incidence rates of borreliosis spanning a period of 20 years (1997-2016) and weekly tick-related hospital visits spanning two years (2018-2019). We identify regions in Russia characterized by similar dynamics of incidence rates and dominating tick species. For each cluster, we find a set of climatic parameters that are significantly correlated with the incidence rates, though a linear regression approach using exclusively climatic parameters to incidence prediction was less than 50% effective. On a weekly timescale, we find correlations of different climatic parameters with hospital visits. Finally, we trained two long short-term memory neural network models to project the tick-related hospital visits until the end of the century, under the RCP8.5 climate scenario, and present our findings in the evolution of the tick season length for different regions in Russia. Our results show that the regions with an expected increase in both tick season length and borreliosis incidence rates are located in the southern forested areas of European Russia. Oppositely, our projections suggest no prolongation of the tick season length in the northern areas with already established tick population. Ixodes genus are the most important vectors of dangerous tick-borne diseases, such as encephalitis and borreliosis, in the Northern hemisphere [Ixodes ricinus and Ixodes persulcatus, commonly known as the castor bean tick and taiga tick, respectively. Both species have a similar four-stage life cycle, which consists of the egg, larva, nymph and adult stages, and a life span between three and six years [I. persulcatus [I.persulcatus was found to be limited by the minimum accumulation of 1410°C days, whereas the development of I. ricinus required a minimum of 1460–1550°C days [I. ricinus was not established in the areas where the snow cover period was longer than 150 days and the length of vegetation period was below 170 days [Ticks species of misphere . In Euroix years . The ticsulcatus , 4. Numesulcatus , 5–9. Ba0°C days . In Euro170 days .Ixodes tick activity and tick-borne diseases in the northern areas previously not inhabited by ticks. These trends have been reported for Sweden [As the probability of ticks’ survival is strongly dependent on environmental factors, climate change can drive tick-borne diseases to the north , 12. Recr Sweden , Norway r Sweden , 15, Icer Sweden , USA [17r Sweden  and Russr Sweden –21. In Sr Sweden . SimilarMany studies have attempted to link the northwards propagation of ticks to warming climate, and have identified a correclation between the increasing temperature trends and the incidence rates of tick bites. Tokarevich et al. 2011) reported a significant strong correlation between the mean annual air temperature and encephalitis incidence rates in Arkhangelsk region for the time period between 1980 and 2007 11 report. HoweverAlthough modelling approaches, which account for the ticks’ life cycle, host dynamics and land use change, and take into consideration the complexity of vector dynamics and vector-host relationships, have been previously used in a number of studies , 23, it This study aims to assess the link between climatic parameters and tick bites as well as borreliosis incidence rate for European Russia. In contrast to previous studies, we do not consider spatial propagation of ticks to new sites. Instead, we aim to predict the temporal evolution of tick-related visits and incidence rates in the territories already inhabited by ticks. We focus mainly on forested areas within the latitudes from 52°N to 65°N. Instead of considering separate regions, we show that the regions can be clustered based on the temporal dependencies of incidence rates pertaining to areas with a certain tick species’ abundance. Furthermore, we use multiple linear regression method to identify the most important climatic factors influencing borreliosis incidence rates for different clusters of regions. Finally, we use deep learning methods to simulate the length of the tick season and make prognosis for the RCP8.5 climatic scenario . We postulate that the tick season length is indicative of the risks associated with tick borne diseases in the changing climate until the end of the century.I. ricinus and I. persulcatus. In Western Europe, I. ricinus is the dominating species although the first evidence of establishing populations of I. persulcatus in Finland and Sweden has been recently reported [I. persulcatus.We chose European Russia because the regions are relatively small compared to those in the Asian part, which allows for higher spatial resolution. In addition, European Russia is a sympatric zone for both types of ticks, reported , 27. In The medical data used in this study originate from the Rospotrebnadzor’s  official statistical database on Ixodid tick-borne borreliosis (Lyme borreliosis) incidence rates per 100 000 population in European part of Russia for the 20-year period of 1997 to 2016. The incidence rates are presented for federal subjects of European Russia. Rospotrebnadzor is responsible for the investigation of outbreaks and cases of infectious diseases at all levels of medical care, the inspection of laboratory diagnostics and the routine monitoring of the epidemiological situation , and the monitoring and assessment of food, air and water quality. The procedure in which such cases are reported is that an initial emergency notification is sent to the epidemiological department of the regional Rospotrebnadzor by the medical practitioner when a patient is suspected to have an infectious disease at any level of medical care. This is then confirmed or excluded from the database based on the results of the specific laboratory tests and the diagnosis is logged by Rospotrebnadzor accordingly. Rospotrebnadzor provides such reports on infectious diseases on a regular basis, which are published and freely accessed through its regional and federal websites .For the purposes of this study, the medical data was retrieved from the Rospotrebnadzor database and provided to us by the Moscow State University, Department of Medical Geography. It includes annual borreliosis incidence rates per 100 000 population for the federal regions of European Russia for a 20-year period (1997–2016). In addition, we used the data set on the number of weekly visits to hospitals for each region or subject: weekly number of patients with tick bites during the tick season. The latter data set is available for 2 years, 2018 and 2019.The meteorological station data include daily measurements of temperature at 2m, precipitation and snow depth for 1–9 stations per federal region for the period between 1997 and 2016. The list of stations is provided in To capture the spatio-temporal variability of the weather in the European Russia, we obtained meteorological data through the ERA5 atmospheric reanalysis for the global climate dataset from the Copernicus Data Store (CDS) , using tFor training the neural networks and performing the future projections of tick-related hospital visits, an ensemble dataset comprised of six bias-adjusted and statistically downscaled NASA-NEX-GDDP climate datasets CMIP5 simulation models  (RCP8.5 scenario) was used , 32. TheThe day length (time between sunrise and sunset) was calculated using the Brock model . Brock dThe photoperiod was calculated using the method described above, for a 0.25° regular grid of the areas enclosed by the clusters used in this study. The centre of each grid box was used in the calculations and the average day length for each federal region was calculated by averaging the day length of all the grid boxes enclosed by it for each day. Finally, the weekly average photoperiod was calculated, which was used in the training and projection procedures by the deep learning models.As a first step in this study, we aimed to identify potential links between the borreliosis incidence rates and the meteorological parameters, which are presented in First, we made an effort to identify any linear correlations between incidence rates and meteorological parameters in different regions separately and for clusters of regions. Although this procedure revealed potentially important parameters for each cluster , none of these dependencies was statistically significant.We adapted our approach, as both incidence rates and meteorological parameters within the clusters were reasonably close to each other. To reduce the fluctuations in the data sets, we calculated the median values of incidence rates and meteorological parameters for each year and each cluster of data. Then we applied a multiple linear regression for each cluster using the standard Matlab package. For two clusters out of three, the results for a number of parameters were marginally significant, with p-value close to 0.1. In these data sets, we identified outliers based on the residuals and removed one outlier from each data set. In both cases the latter procedure improved the model and made the corresponding dependencies statistically significant.To establish the similarity between the borreliosis related weekly hospital visits and the environmental and the climatic features described above, we used a time lagged cross-correlation approach. The time series of the weekly hospital visits and each of the ERA5 and CMIP5 variables were considered as two signals, which were shifted in time relative to each other. The cross-correlation coefficient was calculated at each time shift (lag time), to compare the two signals and investigate any periodicities in the data. In the case that the data is periodic, the correlation coefficient with lag curve is expected to have a positive peak when the two signals are in phase and a negative peak when the two signals are out of phase. This type of analysis can be used to establish a leader-follower relationship between the two time-series’ and, in this case, to gauge whether the climatic/environmental variables used contained useful information to be included in a predictive model.A long short-term memory (LSTM), a special kind of recurrent neural network (RNN) was chosen to be trained to project the weekly tick related hospital visits. An LSTM network consists of a pre-defined number of memory cells, which are each comprised of an input gate, an output gate and a forget gate. This kind of configuration allows the neural network to store information over extended time intervals and, thus, regulate the flow of information in and out of the cell , 36. FurTwo independent neural networks were trained, one for each of the clusters in which a distinct species of tick is found (the Northern and Southern clusters). Both neural networks shared the same architecture, which consisted of 32 LSTM units and an output layer with shape . To minimise overfitting, a number of measures were taken, such as implementing a dropout and a recurrent dropout with a 0.25 rate, and applying L2 kernel, recurrent and bias regularisation and a batch normalisation layer. In addition, during training, an early stopping callback function was implemented, in which the training loss was monitored, and training was interrupted when the metric did not improve after 150 epochs.For training the neural network, the input features were scaled using the StandardScaler method, included in the scikit-learn module. The features used for training and projecting tick related hospital visits include the climate variables from the CMIP5 models  and photoperiod. Furthermore, the weekly hospital visits for each region were re-scaled into a 0–100 range and binned into 4 classes, as shown in To account for the delayed effect of climate variables, such as temperature and precipitation, to the life cycle of the ticks, the feature set of the previous two weeks were included for each week’s prediction (time step = 2). Furthermore, the time series for each federal region was treated independently in training. For each cluster of federal units (northern and southern cluster), the neural network’s weights were randomly initialised and trained on the time-series data of the federal regions belonging to the respective cluster sequentially. After model initialization, the training of the neural network commenced on the first federal region of the cluster, using the Adam optimizer, while the accuracy of the model was monitored in terms of the categorical cross-entropy accuracy. A schematic representation of the training and projection procedures for each cluster of federal regions is shown in Ixodes ricinus habits light mixed and deciduous forests and associated areas. However, the longitudinal span of the habitat of this species is rather wide: from Karelia in the north to Ukraine in the south [I. ricinus includes Europe.he south . The eashe south . In the Ixodes persulcatus include the middle and southern taiga and span from the east to the west over all the Russia [The areas of habitat of e Russia . In EuroI. persulcatus is a dominating species in the northern federal regions and I. ricinus is a dominating species of the two in the most southern federal regions within the habitat of these two species, as shown in The areas of habitat determine different tick distribution over the regions located in different vegetation zones. For example, I. persulcatus , demonstrate the following pattern: doubling incidence rates from the half mean value to the mean value during 1997–2003. The weekly hospital visits have one strong peak in May-June, which corresponds to one stage in the tick’s life cycle. Ticks can attack human in two stages of their development cycle, nymph and imago. In the northern latitudes, the ticks can pass only one stage of their development before they diapause or die.In I. ricinus , the normalised annual incidence rates have been increasing from almost 0 to 1.5 mean value during the period 1997–2016. In the weekly tick-related hospital visits time-series, there are two well-pronounced peaks observed, one in spring (May-June) and one in autumn (August-September). In warmer climates, ticks can go through two stages of development during one year, which can explain the two peaks observed during one season.In the Southern cluster of federal regions, where the tick population is dominated by Finally, in the mixed cluster of federal regions, where both ticks’ species are abundantly present , the annual normalised incidence rates are fluctuating around the mean value, with some tendency to decrease in the latter part of the time series (2013–2016). The weekly hospital visits time-series have a strong peak in spring (May-June) with a smaller peak in autumn (August-September).The results of the multiple linear regression analysis performed on the three identified clusters’ incidence rates and climatic/environmental parameters is summarised in The association between the snowmelt date and borreliosis incidence rates in the North is possibly related to human behaviour. In the weekly tick-related hospital visits time series dataset for the years 2018 and 2019, shown in I. ricinus, which is less contaminated by borrelia [In the Mixed cluster, the association with temperature is weaker than that with the hydrothermal index. In addition, there were no discernible trends observed in the incidence rates, as it fluctuates around the mean value. Our results predict some decrease of incidence rates with growing temperature. This may be considered as counter intuitive, since higher temperatures typically favor the development of ticks at all stages. Presumably during the warmer tick season, the prevailing species is borrelia . The linI. ricinus in other studies. Interestingly, adding winter parameters, such as snow cover and freezing degree-days, which also showed some potential importance in the preliminary analysis, did not improve the correlation. Freezing degree-days is likely not a fully independent parameter from the temperature sums during the active season.In the Southern cluster, the strongest link found was with the temperature sum parameter, which should not be surprising, given the observed strong secondary peak in autumn in hospital visits. A prolonged warm season facilitates tick activity, which has been also demonstrated for th century [Even though a number of climatic parameters were identified to have significant moderate correlations with annual borreliosis incidence rates in the three clusters, using exclusively climatic parameters in linear regression models resulted in predictive performance less than 50%. This is in line with previous studies of the relation of tick-borne encephalitis in the Baltics to climate change, which proposed that climate alone cannot explain the observed upsurge in the late 20 century . The driTo investigate any delayed effects of climate and environmental variables to the incidence of tick related hospital visits, we used a time lagged cross-correlation analysis approach. Here, the time-series of the weekly tick-related hospital visits and climate/environmental variables were considered as two signals, which were shifted in increments of 1 week between −20 and +20 weeks and their cross-correlation coefficient was calculated at each step.In The low and high vegetation indices as well as the precipitation TLCC analysis with respect to the weekly tick-related hospital visits are presented in I. persulcatus, whereas data from the Southern cluster was used to train the model for I. ricinus. The training accuracy and loss metrics for the training procedures of the two models are shown in As described in Section 2.7, two independent LSTM neural network models were trained, one for each of the two tick species investigated in this study. Data from the Northern cluster of federal regions were used to train the model for In I. persulcatus tick, of around 20 weeks, until the end of the century. In the Northern cluster of federal regions, the tick season time series follows a slightly decreasing trend, whereas in the mixed cluster there is no discernible upward or downward trend. This observation is possibly due to the high dependence of the tick’s life cycle to photoperiod [I. ricinus tick, the model projections suggest an increase in tick season length, by about one week, until the end of the century in both the Southern and mixed clusters of federal regions.The trained LSTM models were then used to project the length of the tick season for the federal regions investigated in this study. In I. persulcatus tick season, as both the start and the end of the suitable window are observed to have an increasing trend. For the I. ricinus tick, the LSTM model projects lengthening of the tick season, which is attributed to the extension of the end of the season in both the Southern cluster and mixed cluster.To further investigate the tick season in the three clusters of federal regions, we examined the onset and end of tick season projected by the two LSTM models, shown in We investigated the link between climatic parameters and two different variables related to Ixodid ticks and tick-borne diseases in European Russia, namely tick-related hospital visits and borreliosis incidence rates. It is likely that both of the aforementioned incidence rates are underestimating the true numbers of tick-related hospital visits and borreliosis incidence rates, as it is probable that people will not seek medical care and, thus, these cases will go undocumented. The Rospotrebnadzor reports are however, to our knowledge, the most complete freely available records of such occurrences in European Russia’s federal subjects.We found out that the federal regions in European Russia can be split into three clusters  characterized by similar dominating tick species, similar dynamics of normalized borreliosis incidence rates and similar dynamics of normalized hospital visits. Interestingly, we detected a strong linear growth in the normalized incidence rates in the Southern cluster of the federal regions. In agreement with our results, growing trends in tick bites and borreliosis incidence rates have been reported for Tulskaya region , which iFor each of the clusters, we identified climatic variables with significant moderate correlations with borreliosis incidence rates. In the Northern cluster, this is the snowmelt date, in the Southern cluster, the temperature sums during warm season, and in the mixed cluster, the best correlation was found to be with two parameters: median temperature of the tick season and hydrothermal index. These climatic parameters explain from 25 to 40% of the variation in borreliosis incidence rates, depending on the cluster. This is a reasonable result given that there are many unknown factors that were not considered, including human behavior, land use change, host dynamics and others. Our results emphasize that the accuracy of prediction of incidence rates based on climatic parameters and using linear regressions is below 50%. Moreover, opposite to encephalitis, we did not find any association between increasing temperature and borreliosis incidence rates in the north of European Russia.Furthermore, we investigated the correlations between the weekly dynamics of climatic parameters and weekly tick-related hospital visits. We show that all climatic parameters with pronounced seasonal cycle, such as air temperature, evaporation, vegetation indices and snow depth, show reasonably high correlations (maximum correlation coefficient between 0.7–0.8) with the hospital visits allowing for some time delays, which vary for different clusters of regions. This means a strong link between the ticks’ active season and these parameters and justifies usage of these parameters for modelling of hospital visits.Finally, we trained two LSTM neural network models in the Northern and Southern clusters to predict hospital visits in the mixed cluster and to project future hospital visits until the end of the 21st century, under the RCP8.5 climate scenario. The model trained on the set of data from the Northern cluster showed better agreement with observed hospital visits in the mixed clusters. Overall, the model captured the magnitude of the tick-related hospital visits in the Northern and mixed clusters better, whereas two peaks in the Southern clusters were underestimated by the model. However, the tick season length was captured well by the two models for all clusters. Our projections show that the tick season length in the Northern and mixed clusters of federal regions does not change significantly until the end of the century, whereas in the Southern cluster, the season is prolonged by approximately one week. This change is expected to be due to earlier onset and later end of the season, in agreement with the current warming trends.I. ricinus. This may sound as a counter-intuitive result because warming is amplified in the north and there is typically a higher fraction of I. persulcatus infected with borrelia. However, during the warm season, ticks undergo only one stage of development in the north, likely related to photoperiod [Based on our results, an increase in tick-related hospital visits and borreliosis incidence rates is expected in the southern forested areas of European Russia dominated by toperiod . Higher toperiod , 20. Thetoperiod . These ctoperiod  or humanS1 FigThe input feature tensor consisted of the feature set of the last three weeks to account for the delayed effects of temperature and precipitation.(TIF)Click here for additional data file.S2 Fig(TIF)Click here for additional data file.S3 Fig(TIF)Click here for additional data file.S4 Fig(TIF)Click here for additional data file.S5 FigThe precipitation patterns in the three clusters did not vary significantly throughout the two years examined here, thus no significant cross-correlation coefficient was found between it and the weekly tick-related hospital visits time-series.(TIF)Click here for additional data file.S1 Table(PDF)Click here for additional data file.S2 TableThe minimum and maximum temperature features were derived from the 2m temperature hourly variable for each day.(PDF)Click here for additional data file.S3 Table(PDF)Click here for additional data file.S4 TableThe clusters were divided into clusters with respect to the tick species and hospital visits dynamics found in them (Sec. 3.1).(PDF)Click here for additional data file.S5 TableThe clusters were divided into clusters with respect to the tick species found in them.(PDF)Click here for additional data file.


Easy detachment is as important as reliable an attachment to climbing robots in achieving stable climbing on vertical surfaces. To deal with the difficulty of detachment occurring in wheeled and track-type climbing robots using bio-inspired spines, a novel climbing robot utilizing spiny track and dual-rail mechanism is proposed in this paper. The spiny track consists of dozens of spiny feet, and the movement of each spiny foot is guided by the specially designed dual-rail mechanism to achieve reliable attachment and easy detachment. First, the design of the climbing robot and the dual-rail mechanism are presented. Then, the dual-rail model is constructed to analyze the attaching and detaching movements of the spiny feet, and a mechanical model is established to analyze the force distribution on the spiny track. Finally, a robot prototype is developed, and the analysis results are verified by the experiment results. Experiments on the prototype demonstrated that it could climb on various rough vertical surfaces at a speed of 36 mm/s, including sandpaper, brick surfaces, concrete walls with pebbles, and coarse stucco walls. Climbing robots could be applied in space exploration, reconnaissance, disaster rescue, and other fields due to their advanced locomotion capacity on steep or even vertical surfaces . To climNot only reliable attachment but also easy detachment is important in achieving stable climbing performance. In the design of climbing robots, the detachment of robots’ feet should be easy and rapid for efficient locomotion. In nature, because insects possess directional claws and geckos possess directional branched setae arrays on their feet, they can easily and rapidly switch their feet’s adhesion statuses by changing the motion directions of their feet. Inspired by insects’ claws and their attaching and detaching movements, several climbing robots have been developed for vertical rough surfaces. Most of these climbing robots employed legged locomotion, and their feet could easily detach from the climbing surfaces by manipulating the movements of their legs or feet. Asbeck et al. proposed a legged robot with six complaint spiny feet, SpinybotII, which was capable of climbing on hard and rough concrete and brick surfaces . Each spDifferent from the alternative climbing gait adopted by legged robots, wheeled and track-type climbing robots with minimal actuators are simpler and more effective. Several climbing robots using spiny wheels have been developed for rough vertical surfaces ,30,31. HIn our previous studies, a track-type inverted climbing robot with spiny grippers was developed , and theThe reminder of the paper is as follows. In this section, a track-type climbing robot using insect-inspired compliant spines and dual-rail mechanism is proposed. As shown in The spiny track is the critical adhesion component for maintaining the robot on vertical rough surfaces. Using the spiny track, the robot can achieve a continuous climbing motion. The spiny track is composed of sixteen spiny foot units. As shown in Serica Orientalis Motschulsky, which has been studied in our previous works [The compliant mechanism of the spiny foot plays an important role in the design of climbing robots with bio-inspired miniature spines. A spiny foot possesses ten bio-inspired spiny toes, and its compliant is achieved through the specially designed spiny toes. The detailed structure of a bio-inspired spiny toe is shown us works ,32  and g, respectively. The length of linkage is constant a. One endpoint of the linkage is located on the outer rail, and another endpoint of the linkage is located on the inner rail, and their coordinate values  and  should satisfy the following equation:The easy detachment of the spiny foot is critical in the design of track-type climbing robot. A theoretical model of the dual-rail mechanism is established to analyze the detachment of the spine foot. As shown in α + β) is the rotation matrix, c is the length of the spine, b and β are the constant parameters of the spiny foot unit, and the orientation angle of the linkage can be calculated with the following equation:The coordinate values of the spine tip can be calculated with the following equation:Combining Equations (1)–(3), we can get the detaching movements of the spiny foot and the trajectory of a spine tip during detachment. The analysis is based on the design parameters of the dual-rail mechanism shown in n is the number of attached spiny foot units, mxF and myF are the normal force and tangential force acted on the m-th spiny foot unit, respectively, nF and fF are the supporting force and friction force at the tail, G is the gravity force, h is the distance from the center of gravity to the climbing surface, l is the distance between the two adjacent spiny feet, Tl is the distance between the last attached spiny foot and the tail. If the m-th spiny foot unit provides adhesion, the normal force mxF is positive.When the robot climbs on vertical surfaces, the spiny feet move with the rotation of the spiny track, and switches between the attached phase and the swing phase. The spiny foot at the front of the robot tries to attach onto the vertical surface, and the spiny foot at the rear of the robot tries to detach from the vertical surface. The total number of attached spiny foot units switch between six and seven during climbing. yk and xk, respectively. The tangential stiffness and normal stiffness are a statistical result, which is related to such factors as wall morphology, the compliant design of a spiny toe, the number of spine toes, the adhesion probability of spine, etc. In the actual robot climbing process, the actual stiffness values vary. We assume that all the attached spiny foot units’ yk and xk are equal.Being compliant is a key characteristic of the spiny foot. The spiny foot unit’s tangential stiffness and normal stiffness are x. It means that the normal forces Fx1 to nxF are equal-difference sequences:m is from 1 to 6. The normal forces could be distributed more evenly by adjusting the supporting force nF on the tail. According to Equation (4), if the supporting force is too small, it is not enough to overcome the pitch-back moment generated by the robot’s weight. On the contrary, if the supporting force is great, the adhesion needed is greater, and more spiny feet will be under the preload state rather than the adhesive state. It means greater adhesion force should be generated by less spiny feet. Therefore, the supporting force on the tail should be moderate for reliable climbing performance.For reliable adhesion, the adhesion needed to balance the pitch-back moment and the gravity of the robot should be evenly distributed among the attached spiny feet. The normal force distribution and tangential force distribution are analyzed in Fy7 is distributed among the other spiny foot units. All these six spiny foot units have the same deformation increment Δy, which means that the tangential force Fy7 is evenly distributed among these six spiny foot units. And the same force increment is Fy7/6. Then another spiny foot unit touches the climbing surface with the climbing motion of robot, and the robot switches into the first stage again. According to the above analysis, we have the following equation:m is from 1 to 6. From Equations (4) and (6), the tangential forces on the attached spiny foot units could be calculated as:The climbing process of the robot is illustrated in As shown in In order to obtain the contact force between the spiny foot unit and climbing surface during climbing, a force-testing platform was built, as shown in As shown in nF is about 0.67 N, and the friction force fF is small enough to be ignored. In general, the normal force and tangential force on the spiny foot unit basically show a linear increasing trend. In addition, there is a sudden change of contact force in each step, as circled by the green dashed lines in mxF and myF on the seven spiny foot units attached on the climbing surface. The test results of the force distribution are consistent with the analysis results in A robot prototype was made to perform on various rough surfaces to evaluate its capacity. All the climbing surfaces are vertical, and the slope angles are 90 degrees. Considering that the climbing performance of the robot is related to some random characteristics of the climbing surfaces, such as the size of particles, the embedded depth, and the distribution of the particles, it is difficult to describe the climbing performance directly by the surface roughness or other parameters. Herein, sandpapers with different meshes are chosen as standards to judge the climbing performance of the robot, and experiment results show that the robot can climb on surfaces rougher than P120 sandpaper surfaces. The average size of the sand particles is about 120 μm. Climbing experiments on the three typical rough surfaces are illustrated in The performance of both the attachment and detachment of the spiny feet during the vertical climbing is illustrated in In this paper, a track-type climbing robot with bio-inspired spiny feet was developed for stable climbing on rough wall surfaces. To deal with the difficulty of detachment occurring in track-type climbing robots, a novel dual-rail mechanism was proposed in the robot design. The mechanical model of the specially designed dual-rail mechanism was established to analyze the attaching movements and detaching movements of a spiny foot. The analysis results and experimental results demonstrated that the spiny feet detached from the climbing surface with the reverse motion of attachment, and the detaching resistance was negligible, which means that the easy detachment of the spiny feet was achieved. Additionally, to study the force distribution among the attached spiny feet, a mechanical model was established. The analysis results of the force distribution showed that the normal force and tangential force were distributed among these spiny feet in the form of equal-difference sequences, and the contact force test results verified the analysis results. Finally, the climbing capacity of the robot prototype was experimentally demonstrated. The robot can achieve stable climbing on diverse rough surfaces, including sandpaper surfaces, brick surfaces, and some real exterior walls. Its climbing speed is about 36 mm/s, and its payload capacity can reach 200 g.The climbing performance of the track-type robot on some real exterior walls has been demonstrated in this paper. However, the locomotion function of the robot is still limited for practical applications. In the future works, the bio-inspired spiny foot will be optimized for stronger adhesion, and a robot with multi-linked spiny tracks will be developed to improve its locomotion ability, including ground-to-wall transitions, turning function, and lateral climbing ability.


In long-term use, cracks will show up on the road, delivering monetary losses and security hazards. However, the road surface with a complex background has various disturbances, so it is challenging to segment the cracks accurately. Therefore, we propose a pavement cracks segmentation method based on a conditional generative adversarial network in this paper. U-net3+ with the attention module is used in the generator to generate segmented images for pavement cracks. The attention module highlights crack features and suppresses noise features from two dimensions of channel and space, then fuses the features generated by these two dimensions to obtain more complementary crack features. The original image is stitched with the manual annotation of cracks and the generated segmented image as the input of the discriminator. The PatchGAN method is used in the discriminator. Moreover, we propose a weighted hybrid loss function to improve the segmentation accuracy by exploiting the difference between the generated and annotated images. Through alternating gaming training of the generator and the discriminator, the segmentation image of cracks generated by the generator is very close to the actual segmentation image, thus achieving the effect of crack detection. Our experimental results using the Crack500 datasets show that the proposed method can eliminate various disturbances and achieve superior performance in pavement crack detection with complex backgrounds. The road is one of the main modes of modern transportation and plays an important role in a country’s economic development. In long-term use, the pavement is prone to damage such as cracks due to vehicle crushing or natural disasters, which can reduce driving comfort and safety and shorten the lifespan of the road. Pavement cracks prevention is an important part of road maintenance. Previous road crack detection methods are mainly traditional manual detection, which require a lot of human and material resources. They are easily affected by the subjective factors of the inspectors, resulting in low detection efficiency. Therefore, the automation of crack detection has high research value and application prospects.With the development of computer vision, target detection methods based on digital image processing are applied to pavement cracks detection. Commonly used methods can be divided into threshold-based segmentation methods, edge-based segmentation methods, and region-growing-based segmentation methods. The threshold-based segmentation method is to set an appropriate threshold for division according to the grayscale difference between the crack area and the pavement, and the target area is the one that meets the threshold requirements. Akagic et al.  implemenIn recent years, as deep learning does not require manual feature extraction and has better generalization performance and good anti-noise ability, it has been gradually applied in crack segmentation. The Fully Convolutional Network (FCN) transforms the initial network for image classification into the network for image segmentation. Dung et al.  proposedThis problem can be improved by utilizing feature fusion. Yang et al.  proposedThe encoder–decoder structure can solve the problem of the loss of spatial location information well. Namely, the encoder extracts the location information and image features of cracks through operations such as convolution and pooling. The decoder parses them through operations such as deconvolution and upsampling to restore the spatial dimension of the image and the position information of the pixels. Jenkins et al.  proposedAlthough the above methods have achieved good results in pavement crack detection, there are still some problems to be solved, such as the inaccuracy of segmentation results caused by various noises in the detection process. To overcome these problems and improve the accuracy of segmentation, this paper addresses this challenge from three perspectives. First, we combined the improved U-net3+ network with the conditional generative adversarial network (CGAN) and proposed a road crack detection method based on the CGAN. The introduction of GAN in semantic segmentation tasks can enhance spatial long-distance dependence. GAN can evaluate the joint distribution of labels at each pixel position as a whole. Compared with non-adversarial training methods, it pays more attention to the high-order spatial consistency of data and can obtain more accurate and smooth segmentation results. Second, we used the U-net3+ network with the attention module as the generator of the CGAN. To solve the problem of noise interference in road images, we proposed a parallel attention mechanism, which redistributes the weight of crack features from the two dimensions of channel and space, and then fuses the redistributed features to obtain more complementary crack features and suppress the interference. Third, we used a weighted hybrid loss function to improve the accuracy of segmented images. If the segmentation performance is entirely dependent on adversarial losses, then the output segmentation mask may not always have the same global structure as the manual annotation. Therefore, the cross-entropy loss between the generated segmentation mask and the manual segmentation mask was added to the loss function of the generator to improve the segmentation performance by penalizing the distance between them.(1)U-net3+ is used as the generator of CGAN, and PatchGAN is used as the discriminator of CGAN for pavement cracks segmentation.(2)The parallel attention mechanism is embedded into the generator to highlight crack features and suppress noise features.(3)A weighted hybrid loss function is proposed to improve the segmentation accuracy of cracks.(4)This paper is organized as follows. To summarize, the contributions of this paper are threefold:At present, some non-destructive testing techniques have made some progress in crack detection. For example, Zhao et al.  used groIn recent years, deep learning methods have been gradually applied to road image segmentation because they can automatically extract target features at multiple scales. Cha et al.  proposedOlaf et al.  proposedGoodfellow et al.  proposedIn the semantic segmentation task, the role of the generator model is to map the source domain image In the convolutional neural network, feature maps of different scales describe different feature information of an image. The shallower the network, the smaller the receptive field, and the stronger the ability to represent the geometric details of the extracted shallow features, such as contour, edge, and other information. The deeper the network, the larger the receptive field, and the stronger the semantic representation ability of the extracted deep features, which can determine the attributes of objects. The U-net3+ network uses full-scale skip connections to fuse feature maps of different scales and has a better ability to extract the shallow and deep features of the image. Therefore, we choose U-net3+ as the generator in CGAN. At the same time, in order to solve the inaccurate segmentation problem caused by noise, shadows, and other interference in the image, the attention module is embedded into the U-net3+. The attention module highlights crack features, suppresses noise from the channel and space dimensions, and performs cross-dimensional feature fusion to obtain more complementary crack features so that it improves segmentation accuracy. The overall network architecture of the generator is shown in Channel attention. First, the input feature map is defined as Spatial attention. First, the input feature map is defined as Fusion. Finally, the channel weight map This paper uses the parallel attention mechanism, and the parallel structure can effectively suppress all kinds of interference, especially the interference of bright noise. As shown in The discriminator of GAN is a binary classifier to determine whether the input image is from the real world or the generator. We use PatchGAN as the discriminator of CGAN. The original road image is spliced with the manually labeled image and the generated segmented image and then input into the discriminator. The structure of the discriminator is shown in In order to improve the accuracy of segmented images, we propose a weighted hybrid loss function with two terms. The first is the adversarial loss term, and the second is the segmentation loss term. The weighted hybrid loss function can be written as:The training of the CGAN is the process of optimizing the min-max problem of the objective function, which is achieved in practice by alternately training the discriminator and the generator. When training the discriminator, it can be seen from Equation (5) that the discriminator loss is only related to the adversarial loss term, so the loss function can be written as:If only the adversarial loss term is used to train the generator, the output segmentation mask may not always have the same structure as manual annotation . TherefoThe experimental operating platform in this paper is the Ubuntu 18.04 system. The entire algorithm is implemented in the Pytorch framework using the Python3.7 environment. The CPU model is an Intel(R) Xeon(R) E5-2678 v3@2.5GHz, the GPU model is an NVIDIA GeForce RTX 2080 Ti, and the memory of the graphics card is 11 GB.The dataset used in this paper is the public crack dataset CRACK500. The dataset contains crack images under the influence of various environmental factors such as illumination, shadows, and road obstacles, which can verify whether the proposed method in this paper has a better segmentation ability for crack images with interference. The CRACK500 dataset provides 516 road crack images with a size of This paper uses indicators commonly used in semantic segmentation to evaluate the model effect, such as the Dice coefficient, pixel accuracy (Precision), pixel recall , and F1-score. The expressions of each indicator are as follows:The Dice coefficient is used to evaluate the similarity between the generated segmented image and the manually labeled image:Pixel accuracy represents the percentage of crack pixels detected correctly in detected crack pixels:Pixel recall represents the proportion of correctly detected crack pixels to all crack pixels:F1-score is used to measure the accuracy of the model:The CGAN network consists of two parts: the generator and the discriminator. The distribution training strategy is adopted during training: (1) The road crack image is spliced with the manually labeled image. Then, it is input into the discriminator network, using the real sample to train the discriminator. (2) Keeping the parameters of the generator model fixed, the segmented image generated by the generator is spliced with the road crack image to train the discriminator. (3) Keeping the parameters of the discriminator model fixed, the generated segmented image is spliced with the road crack image. It is inputted into the discriminator to obtain the adversarial loss, and the generator is trained by the adversarial loss. (4) Repeat steps 1–3. The training of the network is completed after 15,000 iterations. During training, we set the batch size to 4, the initial learning rate to 0.01, and the decay rate of the learning rate to 0.1. The learning rate is updated by tracking the loss, and the learning rate is reduced when the loss does not decrease after 500 iterations.In order to verify the performance of the parallel attention module, this paper selects four widely used attention modules: SENet, CBAM, SAM, and ECA-Net, and conducts a comparative experiment with the parallel attention module proposed in this paper. The comparison results are shown in When the cracks in the image are thin and the background interference is small, as shown in the first row of In order to verify the effectiveness of the hybrid loss function proposed in this paper on the road crack segmentation task and determine the optimal value of the weight coefficient x, x is set to 0, 0.3, 0.5, 0.7, and 1 to train the network. In the segmentation task of binary classification, in addition to the binary cross entropy (BCE) loss function we use, the loss functions often used are Dice loss, IoU loss, and so on. In order to study the influence of different loss functions on network performance, we choose Dice loss and IoU loss to perform comparative experiments with the same weight coefficient. The experimental results are shown in Dice loss is a set similarity measure function, usually used to calculate the similarity of two samples, which can be written as:The IoU loss is more focused on small targets and is more inclined to mine the foreground area during training, which can be written as:As can be seen from In order to verify the superiority of CGAN in the task of road crack segmentation, we use the Crack500 dataset to train U-Net, U-net3+, and U-net+CGAN networks and compare them with the proposed model. The result of the comparative experiment is shown in In order to verify the superiority of the proposed method in the road crack segmentation problem, we compare the proposed method with several other popular segmentation methods: DeepCrack , SegNet In order to show the superiority of the proposed method more intuitively, we select some test set images for visual segmentation comparison with other crack segmentation methods. The selected images contain complex disturbances such as manhole covers, fallen leaves, driveway lines, and shadows. The comparison results are shown in Data augmentation is a method of transforming the training set image in the training stage to improve the generalization ability of the model. At present, because there are few pavement crack images and the workload of labeling cracks at the pixel level in the image is large, the amount of data in the public dataset is small. The segmentation model trained directly on the dataset is prone to overfitting and has a poor generalization ability. Therefore, the method of data enhancement is usually used in the task of pavement crack segmentation. Because the crack image has the characteristics of variable crack direction, low brightness, and various noise, the data enhancement methods we choose include flipping transformation, contrast transformation, random cropping, and noise interference. As shown in As can be seen from We studied the problem of pavement crack segmentation in the process of road maintenance, i.e., crack segmentation is inaccurate due to complex backgrounds and various disturbances of the pavement. Therefore, we proposed a pavement crack segmentation algorithm based on CGAN to accomplish this task. The purpose of introducing CGAN into the crack segmentation task is to evaluate the crack pixels as a whole, to obtain more accurate and smooth segmentation results. The generator adopted the U-Net3+ network and introduced the parallel attention module in U-Net3+. The parallel fusion of channel attention and spatial attention can highlight crack features and suppress interference features to obtain more complementary crack features. The discriminator adopted the PatchGAN structure. In addition, a weighted hybrid loss function was used to exploit the pixel difference between the segmented image and the annotated image to improve the segmentation accuracy further. The dataset used in the experiment was Crack500. First, in order to verify the effectiveness of the proposed parallel attention, we performed an ablation experiment of the attention module. The comparison results with the benchmark model and other attention mechanisms showed that the proposed parallel attention module can effectively eliminate interference and the visual comparison result was shown. Then, in order to verify the effectiveness of the hybrid loss function and find the best weight coefficient, we selected the hybrid loss function with different weight coefficients to train the network. To explore the influence of different segmentation losses on the mixing function, a comparative experiment was performed using different segmentation loss functions. Third, for verifying the superiority of the introduction of CGAN, we conducted ablation experiments. The experimental results showed that the performance of the segmentation model is improved after the introduction of CGAN. Finally, the comparison test with other crack segmentation models showed that the proposed method could achieve superior performance over existing methods.


The simulations were used to compute interfragmentary strain and the von Mises stress distribution of the intramedullary nail for different treatments of the fibula, as well as several walking velocities  and levels of weight-bearing restriction. The simulation of the real treatment was compared to the clinical course. The results show that a high postoperative walking speed was associated with higher loads in the fracture zone. In addition, a larger number of areas in the fracture gap with forces that exceeded beneficial mechanical properties longer was observed. Moreover, the simulations showed that surgical treatment of the distal fibular fracture had an impact on the healing course, whereas the proximal fibular fracture barely mattered. Weight-bearing restrictions were beneficial in reducing excessive mechanical conditions, while it is known that it is difficult for patients to adhere to partial weight-bearing recommendations. In conclusion, it is likely that motion, weight bearing and fibular mechanics influence the biomechanical milieu in the fracture gap. Simulations may improve decisions on the choice and location of surgical implants, as well as give recommendations for loading in the postoperative course of the individual patient.Despite recent experimental and clinical progress in the treatment of tibial and fibular fractures, in clinical practice rates of delayed bone healing and non-union remain high. The aim of this study was to simulate and compare different mechanical conditions after lower leg fractures to assess the effects of postoperative motion, weight-bearing restrictions and fibular mechanics on the strain distribution and the clinical course. Based on the computed tomography (CT) data set of a real clinical case with a distal diaphyseal tibial fracture, a proximal and a distal fibular fracture, finite element simulations were run. Early postoperative motion data, recorded  Despite recent experimental and clinical progress in the treatment of tibial and fibular fractures, the rates of delayed bone healing and non-union remain challengingly high . To datevia implants may lead to a different clinical outcome . Informed consent was conducted according to the Declaration of Helsinki. The study is part of the project Smart Implants 2.0 – Weight-bearing and Gait Observation for Early Monitoring of Fracture Healing and Individualized Therapy after Trauma, funded by the Werner Siemens Foundation. It is registered in the German Clinical Trials Register (DRKS-ID: DRKS00025108).A 63-year-old male patient  suffered from a closed fracture of the lower leg with a distal diaphyseal fracture of the tibia, a proximal and a distal fibular fracture . ComputeTo simulate different mechanical conditions in the fracture gap, a simulation workflow was applied, as previously described in detail . BrieflyFor further assessment, four models were created from the patient’s lower leg data : 1) PlatThroughout the routine aftercare during the postoperative course, the patient was equipped with a pair of insoles with pressure sensors . The insoles were placed in the shoes of the patient during the first physiotherapy session shortly after surgery. In the lab, the insoles were used to monitor the gait of the patient in a standardized setting of 30 steps on a treadmill  at various velocities . For this purpose, the insoles acquired plantar pressure data that were used to compute the resulting average and maximum vertical ground reaction forces and acceleration based on the center of pressure path during the stance phase of gait and its anteroposterior/mediolateral deviation. For every parameter, the mean of all steps and the standard deviation of the measurements between all steps were calculated using the Moticon SCIENCE™ software (Moticon ReGo AG).via an inertial measuring unit (IMU)-based motion capturing system , as described previously  data format for export to the musculoskeletal simulation environment AnyBody . Together with the collected anthropometric data of the patient, a patient-specific avatar was created in the AnyBody software (AnyBody Technology) that enabled the simulation of individualized muscle forces, ligament forces, and internal joint contact forces and moments, which are essential for the understanding of the biomechanical mechanisms during human movement. These data served as personalized boundary conditions in the different FE simulations and, thereby, enabled the analysis of the effects of walking speed and weight-bearing restrictions on bone healing of the tibia according to the previously described threshold values for the different types of bone healing .The data from the musculoskeletal simulations were used in conjunction with the generated geometric models to compare the biomechanical scenarios using the software environment Abaqus . To investigate the influence of the four different fibular configurations on the von Mises stress distribution of the intramedullary nail, the four models were simulated for high axial loading (midstance of the gait cycle) and high loading with non-axial forces . In this context, von Mises equivalent stress was chosen as scalar quantity representing a measure of local loading which can be interpreted a metric for the distribution of forces. In addition, for these two loading scenarios, the mechanical stimuli and the local micromechanics in the fracture gap of the tibia were investigated with respect to the mechanical conditions for fracture healing. The simulations also allow a comparison between partial and full weight-bearing by adjusting the boundary conditions in each case, which was herein performed for the two load cases of the FibOP model. In addition, the Anybody results were used to simulate complete gait cycles for the three different velocities of the patient on the treadmill. This allowed for an analysis of the fracture healing parameters over the complete gait cycles and, thus, the dynamic influence of gait velocity on the fracture gap and its micromechanics. For this purpose, the hydrostatic strain and the octahedral shear strain were computed for each mesh cell of the callus area, and classified into the different classes with respect to the values given previously by p-value <0.05 was considered to indicate significant differences.All data are given as mean values of all steps ±standard error of the mean (SEM) as well as maximum and minimum values of each step. Data were first tested for normal distribution and the assumption of equal variance was proven. In case of parametric data, comparisons between two experimental groups were performed by an unpaired Student’s t-test, while analyses of three groups were performed by one-way ANOVA, followed by the Holm-Sidak test for all pairwise comparisons, including the correction of the α-error according to Bonferroni probabilities to compensate for multiple comparisons. In case of non-parametric data, comparisons between two experimental groups were performed by a Mann-Whitney Rank Sum Test, while analyses of three groups were performed by one-way ANOVA on Ranks, followed by a Dunn’s Test for all pairwise comparisons, which also included the correction of the α-error according to Bonferroni probabilities. The statistical analyses were performed using the SigmaPlot software 11.0 . A The clinical course of the patient was uneventful. After intramedullary nailing of the tibia and plate osteosynthesis of the distal fibula, common soft tissue healing occurred. After removal of the set screw approximately 6 weeks after surgery, the patient was allowed to increase load of the injured extremity to full weight-bearing. Due to pre-existing epilepsy, the patient’s gait pattern was already altered before the injury, but full weight-bearing was achieved. Although the tibial fracture showed osseous bridging after 6 weeks, full osseous healing of the tibia was initially delayed especially in the dorsal part of the fracture, but showed full consolidation at 6 months after the surgery. The plate on the distal fibula was removed at approximately 7 months postoperatively after full osseous consolidation. At this stage, the patient was walking free without aids and the fractures of the tibia and distal fibula were fully healed .Analysis of the data obtained from the insoles demonstrated that the patient avoided full weight-bearing of the injured left leg early after surgery . HoweverWalking on the treadmill at different velocities revealed characteristic changes during the gait cycle of the patient . While tSimulation of different weight-bearing restrictions after surgical treatment at the terminal stance phase and the pre-swing phase during the gait cycle revealed that more elements were outside the healing window when the patient was allowed to fully weight-bear compared to a partial weight-bearing restriction of 35 kg . MoreoveThe four simulations with different bone and implant configurations of the fibula as described in The present study showed that increasing walking speed and loading raised the strain in the fracture gap and in the intramedullary nail, and that the stability of the fibula affects the healing of the tibial injury. Based on the present case, FE simulations showed that fibular fractures should be stabilized surgically when they appear to be on the same level as the tibial fracture, while proximal fibular fractures above tibial injuries do not require surgical treatment. FE simulations further enable patient-specific recommendations in combination with an analysis of the patient’s gait including the range of motion of the lower extremities during the postoperative aftercare, and may, by these means, lead to novel, prognostic individual recommendations.Surgical treatment of a fibular fracture in combination with an ipsilateral tibial fracture is discussed controversially in the literature. Although additional fibular fixation has shown to improve stability and resistance to torsional forces, there has been increased potential for soft tissue-related complications and therefore the clinical impact has been questioned . The bioLately, it has been shown that the location of the fibular fracture is important to determine whether the fixation is indicated or not. Fibular osteosynthesis was considered advisable in distal metaphyseal fracture of the fibula with trans- or infrasyndesmotic lesion . In the in silico. While weight-bearing is already widely accepted to be influential for bone healing but is known to have a low compliance rate (The early postoperative treatment for trauma patients is commonly performed by general, diagnosis-related recommendations on weight-bearing and the type of mobilization. Radiographic controls are initially performed in intervals of approximately 6 weeks with increasing time spans in the later stages of the aftercare period. By these means, problems and complications such as, e.g., delayed bone healing or non-union formation during that period of treatment are mostly detected after several months, causing an individual and overall increased burden of disease . In ordence rate , we shownce rate . Moreovence rate . Bendingnce rate . Accordince rate .We are aware that these conclusions are drawn from only one case in the present study, and more cases will have to be provided to further analyze these associations. However, for individualized aftercare the herein introduced monitoring of the walking speed in combination with changes of the shape of the gait cycle curve in consideration of potential weight-bearing restrictions, as well as potentially other kinetic and kinematic parameters may be helpful novel tools for clinical follow-up controls of bone healing. In fact, this is in line with previous studies analyzing the relevance of gait patterns on bone healing in an experimental setting . Also, iIn future finite element analyses, it might be of interest to implement callus growth over time and stiffness changes in the fracture gap throughout the course of healing to study the development of strain and stress maxima over time . In addiIn conclusion, we demonstrated an individualized simulation of a complex fracture of the lower leg based on the patient’s image and motion data. Together with patient-specific early postoperative motion data, the current findings might enable the treating trauma surgeon to estimate individually the healing course within the first days after surgery. Moreover, the impact of walking speed on the mechanical condition monitored by the gait cycle curve during the postoperative course is demonstrated and highlights the need to find thresholds for the duration and the volume of elements in good healing conditions in future studies. Walking speed in combination with changes of the shape of the gait cycle curve and differences of the range of motion in the involved joints may be helpful novel tools for follow-up visits in clinical practice. This may help to develop individual, prospective and radiation-free recommendations on postoperative treatment in the future.


R2 = 0.48 and a standard deviation STD = 0.10, with a total system average delay of 192 ms. As the average interval between different gait phases in human gait is 250–1000 ms, a 192 ms delay is still acceptable for daily walking requirements. For this reason, compared with the previous study, the newly proposed system presents a higher accuracy and is better suitable for real-time leg muscle activity estimation during walking.To give people more specific information on the quality of their daily motion, it is necessary to continuously measure muscular activity during everyday occupations in an easy way. The traditional methods to measure muscle activity using a combination of surface electromyography (sEMG) sensors and optical motion capture system are expensive and not suitable for non-technical users and unstructured environment. For this reason, in our group we are researching methods to estimate leg muscle activity using non-contact wearable sensors, improving ease of movement and system usability. In a previous study, we developed a method to estimate muscle activity via only a single inertial measurement unit (IMU) on the shank. In this study, we describe a method to estimate muscle activity during walking via two IMU sensors, using an original sensing system and specifically developed estimation algorithms based on ANN techniques. The muscle activity estimation results, estimated by the proposed algorithm after optimization, showed a relatively high estimation accuracy with a correlation efficient of  Knee extensors play an essential role in human mobility . In factIn the past few years, the development of wearable measurement systems improved fast because people started paying more attention to their health status and had begun monitoring their own daily activity ,8,9. As In medical settings, muscular activity is usually assessed using surface electromyography (sEMG) ,11,12. THowever, all these methods for measuring leg muscle activity have severe limitations, which limit their suitability for widespread adoption as muscle activity estimation systems during ADL. sEMG sensors are invasive as they require adequate skin preparation, such as shaving and removing the sebaceous film by alcohol, to ensure good electrode contact; and the signal processing phase is critical to remove noise and interferences which can corrupt the signal and lead to calculation errors . OpticalFor this reason, in a past study, we have developed a system for kinematic model-based leg muscular activity estimation using only one IMU sensor placed on the shank. However, the preliminary results of system testing showed that knee extensors activity estimation using a traditional human lower limb kinematic model was not sufficiently accurate for real-time analysis.In this study, we present an updated measurement system and muscle activity estimation algorithm that overcomes all the limitations previously mentioned. The sensor system uses two IMUs, one on the shank and one on the knee joint. The IMUs collect motion data for the calculation of four gait parameters includes acceleration, velocity, angular velocity, and stride length to reconstruct a more precise gait model. ANN techniques, which are often used to solve predictive system problems ,23, are This paper is structured as following: The system overview is shown in The sensor system consisted of sEMG sensors and IMU sensors, both designed by our group. In particular, data from sEMG sensors were used only for neural network training and for the final algorithm validation, to compare the estimated muscle activity with the actual muscle activity directly measured via sEMG.The sEMG sensor, named Waseda Bioinstrumentation sEMG (WB-EMMG) a 24], w, w24], wThe 9-axis IMU, named Waseda Bioinstrumentation Ver.4 IMU (WB-4 IMU) b 28] in in28] inA diagram of the whole sensor system used in this study is shown in In this study, 14 healthy subjects  participated in the data collection experiment. Subjects’ relevant physical parameters are shown in T1: 5 m normal flat walking at self-pace for five timesT2: 5 m normal flat walking at brisk pace for five timesT3: MVC of knee extension muscle (Left)T4: MVC of knee extension muscle (Right)T5: Measurement of lower limb ratio and BMIThe experiment protocol was composed of 5 phases as follows:Phases T1 and T2 were conducted on a flat surface indoors . Gait daAfter the walking experiment, in phases T3 and T4 we measured the maximum voluntary contraction (MVC) of calf and thigh of subjects. In T3 a, subjecFinally, in T5 the body measurements, weight, height, length of the calf, and length of the full leg, were taken to calculate the lower limb ratio and BMI. These parameters were calculated using the following formulas:The target knee extensor muscles in this study were the rectus femoris muscle (RFM) in the thigh and the outer gastrocnemius muscle (OGM) in the calf. For this reason, the sensors were placed as in Z-axis was perpendicular to the facing direction of the subjects. Unlike the EMG sensors, IMU sensors do not require direct contact with the skin. After positioning the sensors, the whole system was turned on and connected to the GUI on the data processor PC via Bluetooth.IMU sensors were positioned at the knee and the ankle, oriented so that the sensor The muscle activity estimation algorithm was divided into two main parts. A block diagram of the algorithm is shown in To estimate the leg muscles activity from the motion data measured via IMU, gait parameters must be calculated first. A block diagram of the gait parameters calculation algorithm is shown in Heel Strike (HS): the instant in which the foot heel touches the soil. It is the start point of the stance phase and the end point of the swing phase;Foot Flat (FF): the instant in which the foot lays flat on the soil;Toe Off (TF): the instant in which the toe leaves the soil after the foot is thrusted backward. It is both the end point of the stance phase and the start point of the swing phase;Mid Swing (MS): The instant in the middle of the period in which the foot is in the air.Swing phase and stance phase detection: the angular velocity is used to determine the initial and terminal contact of the foot using the algorithm described in . The gaiThe gait parameters calculation algorithm consists of the following steps:An example of IMU gait data for one foot and gait parameter computation is shown in The stance phase is defined as the period from HS to TO while the swing phase is defined as the period from TO to the subsequent HS. The stance time and the swing time can then be calculated.2.Peak detection in swing phase: The peak values of acceleration and angular velocity vectors during swing phase are calculated using the following formulas:Notably, an R-adaptive extended Kalman filter (EKF) was applied to the sensor data to increase the computational accuracy . R is thIn the prediction phase, the angle in the following instant in time is calculated by the current angle and angular velocity. In the update phase, the predicted angle is compensated by the angle calculated from the actual accelerometer data. The accuracy of the angle calculated from accelerometer data depends on the measurement condition, whether static or dynamic. To overcome this problem, the R-adaptive process minimizes the uncertainty in each measurement condition.The formula for computing R is shown in Equations (5) and (6) :(5)R=[σxN is the number of samples of the temporal window; k; R-adaptive is the acceleration norm minus gravity where 3.Coordinate transformation: To calculate the actual motion of the ankle, the motion data measured by the IMU sensor in its local coordinate must be transformed into the global coordinate as shown in 4.Zero Velocity Update: To cancel the integration error, we used the Zero Velocity Update (ZUPT)  algorith5.Stride length calculation: the stride length was calculated from the walking distance in global coordinated, which is computed by integrating the velocity in time (Equation (11)). The walking distance computed using data from the sensor on the shank is shown in Since the distance in global coordinates traveled by the sensor at the ankle is the distance traveled by the ankle itself, the sensor is stationary during the stance phase and moves forward during the swing phase. Therefore, the graph in In this study, we used machine learning techniques to build an adaptive model of muscle activity. In particular, we used an Artificial Neural Network (ANN) for model training. There are in fact many works using ANN for human analysis and adaptive modeling. In the initial stage of model training, the model was trained within a full-connected network. The problems of the trained model were recorded and analyzed after obtaining the estimation results, and the model was refined. After a few refining iterations, having reached satisfactory training results, the model hyper-parameters were optimized. The network structure was also modified to improve the estimation accuracy. The ratio between training datasets and validation datasets is 8:2, while the validation datasets were selected by assigned random seed in algorithm.Since the problems discussed in this study are value prediction problems, we are essentially facing a standard regression problem for classification. The model training in this study was conducted locally. The training framework and the software environments are shown in The CUDA library is a parallel computing library that allows software to use certain types of graphics processing units (GPUs) for general purpose processing. It is applied in the project to get acceleration in machine learning. Tensorflow is a free and open-source platform for machine learning. Keras is also an open-source library that acts as an interface for the Tensorflow to enable fast experimentation with deep neural networks.In the training phase, we used normalized MVC data From the EMG of one gait cycle the Root Mean Square (RMS) with 100 ms of window size (Equation (13)) is calculated, and subsequently normalized by MVC.In this study, a new method to estimate muscle activity of two muscles used in walking, the rectus femoris muscle in the thigh (RFM), and the outer gastrocnemius muscle in the calf (OGM) is presented. The real-time EMG muscle activity measurement was taken as the ground truth against which the model performance was evaluated. We compared the performance of the muscle activity estimation model with the actual muscle activity measured via EMG during walking. We used the coefficient of determination For each muscle, the average muscle activity and the peak performance in one gait cycle were estimated. The results of the system performance experiment are shown in The real-time performance was shown as By default, the hyper-parameters of the model at this stage have not been optimized. From the views of results, the accuracy of the model is notably high, but weird where accuracy testing is 30% different between the training dataset and the testing dataset. Such a situation shows that the model is with low robustness at the same time. We achieved the model optimization to improve the robustness.Optimization of the Hyper-parameters: Hyper-parameters refer to the parameters that control the process of machine learning: loss function, learning rate, and optimization algorithm. The model optimization used in this study is “GrindSearch” . This me(a)the number of hidden layers;(b)the number of nodes in the hidden layer;(c)the activation function of hidden layers;(d)the optimizer;(e)the loss function.The subset of hyperparameters optimized in this study is:For (number of the training = 3) in the given range combination, the final selected hyperparameter values are shown in This set of optimized hyperparameters leads to a relatively good performance, but also to a more complex structure of the model, which indirectly increases the difficulty of model training. For this reason, the optimized model is not suitable for online learning predictions. However, the model discussed in this study is only used for offline learning.2.Optimization of the network structure: the most common network structure for ANN is full connection, meaning that the nodes of each layer are connected to all the nodes of the adjacent layers. The advantage of such a structure is that it can learn the training data characteristics more comprehensively. However, sometimes, a full connected ANN might give too much weight to less important or interfering features, e.g., signal processing noise. In such cases, the resulting trained model is less robust, e.g., overly sensitive to noise, and easy to overfit if trained on a small dataset, which leads to lack of practical application value of the model.Therefore, in this study we used the “Dropout” method, to randomly discard some nodes connections . This prThe performance gap between the model on the training dataset and the test dataset is within 5%. In other words, after this optimization process, the general performance of the model has been improved.The results estimated by the proposed algorithm after optimization in  showed aIt is important to note that the performance of real-time muscle activity estimation is affected by the signal transmission and processing delay of the current system. The measured average processing delay of the system is 72 ms, while the wireless communication average delay is 120 ms, with a total system average delay of 192 ms. Human regular walking frequency is between 1 Hz and 4 Hz , which mFrom the results, us study , the actThe model used in the study uses the EMG sensors measurements as the ground truth for MVC calculation. However, EMG measurement itself has intrinsic errors. For this reason, further improvements in prediction accuracy are required, possibly using more than one method to measure muscle activation and calculate the MVC;The system in this study has data processing limitations. Before importing the model, the measured data must be filtered to remove noise and interference. As a result, the system must rely on adequate data preprocessing for correct real-time estimation of MVC values.During this study, some limitations have been discovered:In this study, a real-time muscular activity measurement system using non-contact sensors was developed. The system uses two IMU sensors to collect the motion data during normal walking to estimate the knee extensor activity. An Artificial Neural Network was used in estimation model training. An evaluation experiment was also conducted to validate the estimation algorithm.In this study, we have shown that it is possible to estimate knee extensors activity during simple walking at different speeds, without an EMG or a Ground Reaction Force (GRF) sensor. The results of this study reveal the possibility of muscular activity estimation only by IMU.Improving the accuracy of the model: subsequent research will consider using multiple sensors to estimate the MVC reference value rather than a single EMG sensor to provide the MVC reference value in model training. The fusion of multiple sensors can complement the observation results, and it is expected to raise the upper limit of prediction accuracy due to the EMG sensor;Optimizing the effect of real-time prediction: subsequent research will consider simplifying the complexity of calculations and seeking more efficient algorithms, using feature extraction methods without gait cycle estimation, if it is possible;Working on applications: the proposed system can not only work as measurement to offer feedback to human, but also send muscle activity data to a smart training device or exoskeleton. We have developed a knee extensor training device with active components to apply load on the knee joint during normal walking. With more detailed, real-time muscle activity data, the device can adjust its load according to the current muscle activation to avoid insufficient or excessive training.Future works will mainly focus on the following three aspects:


Dense water production in the seas around Antarctica is a key process for century-scale carbon storage, slowing global warming. Results from an advanced new model reveal the prospect of system changes that may greatly reduce the efficiency of this carbon storage by the end of this century. The Southern Ocean around Antarctica is disproportionately important to this, due partly to its unique pattern of circulation that facilitates the efficient transfer of waters from the surface to depth. Regions such as the Weddell Sea are key to this carbon sequestration. Here, dense waters sink from the shelf and spread to flood the global abyss, isolating transported carbon from the atmosphere at depths of several kilometers for centuries to millennia.As our planet continues to overheat, understanding how the ocean absorbs and stores carbon takes on ever-increasing significance. At present, global warming is attenuated by the reduction of greenhouse gases in the atmosphere as CONature Communications, Nissen et al.1 present results from simulations of the Weddell Sea that include ice shelf cavities and that resolve both physical and biological processes involved in deep-ocean carbon accumulation. When run under the high-emission SSP5-8.5 climate change scenario, changes to the rate and mechanism of dense water production emerge, with the potential to trigger abrupt shifts in the transfer of carbon to the greatest depths. If realised, this could impact the future partitioning of carbon between the atmosphere and ocean, potentially shortening the timescale upon which sequestered carbon can return to the surface and exacerbating climate change.Writing in 2 emissions since the 1980s2, thus buffering other parts of the Earth System from the worst impacts of global warming. A key factor in delivering this “climate favour” is the overturning circulation of the ocean , a constant-climate simulation, and a third simulation with constant climate but varying atmospheric CO6, but it was then largely absent until recent reports suggested a tentative resumption7. The intermittency and future intensity of dense water production by this mechanism is not well understood, and the simulations presented by Nissen et al. highlight that generating improved understanding and predictive capability is important, with added significance deriving from its impact on deep-ocean carbon storage.Of particular note is that, when driven under the high-emissions scenario, the model demonstrates an ability to generate some unexpectedly rapid shifts in the rate of deep-ocean carbon accumulation. These appear as an increase in the accumulation rate in the 2080s, followed by an abrupt decrease in the 2090s. These changes are traced to an intermittent switch in the mechanism by which the model produces dense waters, with the declining production on the Antarctic shelves being temporarily overcompensated by open-ocean convection over the deep Weddell Sea. Such open-ocean convection—the sinking and mixing down of dense water directly from the surface to great depth away from the continental shelf—is rare in the observational record. A pronounced example occurred in the 1970s8, illustrate clearly that this region is one where sudden system shifts are possible, with long-term consequences. It should be noted that Nissen et al. use a very high emission scenario (SSP5-8.5). A lower greenhouse-gas future is looking achievable, especially given recent commitments and pledges to reduce carbon emissions9, but action is required to translate commitments into action. Further work is needed to ascertain the likelihood of shifts in deep-ocean ventilation under more likely emissions scenarios. The possible emergence of such shifts does, however, point to potentially more variable and less predictable deep-ocean carbon sequestration in future, strengthening the imperative for further modelling and sustained observational programs in this most challenging of environments.The simulations by Nissen et al. have generated new insight into the potential role of changing dense water production around Antarctica in future climate change. They highlight that prospective changes in the dominant mechanisms could drive rapid shifts in the accumulation rate of deep-ocean carbon. Whilst such changes occur in their model at the end of this century, superposed on more gradual decoupling of the shelf and deep ocean environments, it is far from certain at what point the dominant modes of deep-ocean ventilation may switch in reality, or whether a threshold exists beyond which a new mode will dominate. Several recent changes around Antarctica, such as the dramatic retreat of sea ice