Multilevel thresholding segmentation of color images plays an important role in many fields. The pivotal procedure of this technique is determining the specific threshold of the images. In this paper, a hybrid preaching optimization algorithm (HPOA) for color image segmentation is proposed. Firstly, the evolutionary state strategy is adopted to evaluate the evolutionary factors in each iteration. With the introduction of the evolutionary state, the proposed algorithm has more balanced exploration-exploitation compared with the original POA. Secondly, in order to prevent premature convergence, a randomly occurring time-delay is introduced into HPOA in a distributed manner. The expression of the time-delay is inspired by particle swarm optimization and reflects the history of previous personal optimum and global optimum. To better verify the effectiveness of the proposed method, eight well-known benchmark functions are employed to evaluate HPOA. In the interim, seven state-of-the-art algorithms are utilized to compare with HPOA in the terms of accuracy, convergence, and statistical analysis. On this basis, an excellent multilevel thresholding image segmentation method is proposed in this paper. Finally, to further illustrate the potential, experiments are respectively conducted on three different groups of Berkeley images. The quality of a segmented image is evaluated by an array of metrics including feature similarity index (FSIM), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and Kapur entropy values. The experimental results reveal that the proposed method significantly outperforms other algorithms and has remarkable and promising performance for multilevel thresholding color image segmentation. Image segmentation is a vital processing stage in object location and pattern recognition . It can In the past few years, researchers have proffered a range of methods to achieve image segmentation, which can be summarized as threshold-based method , edge-baNumerous techniques based on respective criteria have been developed for getting appropriate thresholds . As a primary and available method, Otsu has long been highly valued and applied . In addiThe essence of image segmentation can be regarded as an optimization problem. Still, the computation complexity increases explosively with the number of thresholds increases. For this reason, researchers combine heuristic algorithms with image segmentation methods creatively. In , an imprThe preaching optimization algorithm (POA) is a novel meta-heuristic algorithm proposed in 2021, which simulates the behavior of preachers in religious communication . PreacheAs the no free lunch (NFL) theorem for optimization is proposed , people However, as with the above-mentioned heuristic algorithms, there are also some drawbacks of the standard POA algorithm mentioned as follows: unbalanced exploration-exploitation and easy to fall into local optimization. According to the above research of the successful employment of distributed time-delay, a natural idea is to introduce it into POA to enhance the performance to a certain extent. In , the mecMotivated by the above discussions, the purpose of this paper is to propose an HPOA-based segmentation algorithm for color images. The main contributions of this paper can be summarized as follows:(1) A novel HPOA algorithm is proposed where (a) the distributed time-delay contributes to a substantial reduction of premature convergence; (b) the hybridizing with PSO provides a thorough exploration of the entire search space; (c) the evolutionary state supplies a significant balance between the local and global search abilities.(2) An HPOA-based color image segmentation algorithm is obtained by combining HPOA with the Kapur entropy algorithm. The proposed HPOA-based segmentation algorithm searches for a more exact threshold thereby facilitating a better components partition of the color image.(3) The performances of HPOA and HPOA-based color image segmentation algorithm are investigated in detail: (a) eight classical single-objective benchmark functions are employed to assess the performance of HPOA on various types of problems. (b) A total of 24 Berkeley color images are utilized to verify the effectiveness of the HPOA-based segmentation algorithm on multiple complex images.The remainder of this article is organized as follows: POA is a novel swarm intelligence algorithm proposed in 2021 . The maiWhen a preacher inherits the religious knowledge to his inheritors, the inheritors will follow around him as follows:Cultural competition occurs between all the inheritors. The elite individuals with the best fitness are selected directly to the next generation, and the remaining are considered according to the comprehensive ranking of location distribution and fitness function:The new preachers will develop their religion through study tours. The idea of the Levy-flight and normal distribution are employed to implement the behaviors. If they get a better position, then the original position will be updated.Although the POA algorithm outperforms various popular algorithms in solving engineering problems, there are still some defects existing such as insufficient thorough exploration for the entire search space and easy to fall into local optimum when facing practical problems, thus it needs to be improved.In this section, a novel HPOA algorithm is proposed to further strengthen the competence of the traditional POA algorithm. The main innovation of this algorithm lies in the introduction of the distributed time delay and the hybridization of the PSO algorithm. In the interim, the evolutionary state strategy is utilized. More specifically, the proposed HPOA algorithm enables the search agents to learn from historical personal or global optimal depending on their evolutionary states. Compared with other algorithms, HPOA pursues stronger capability, avoids being trapped by local optimum, and maintains a balance between convergence and diversity.The search process of heuristic algorithms is frequently phased. For example, agents are more likely to explore promising areas in the early stage of the search, and more inclined to exploit discovered solutions and potentially surrounding areas in the later stage of optimization. According to ,51, thisWhen a biota explores the search space, the distance between different individuals can be used to measure the search state of the population as a whole part. To give an illustration, when the individuals are scattered far away, it means that they are looking for prey. When they gather closer, it’s besieging the excellent targets. The distance Then the best individual is selected and its distance Finally, the evolutionary state of the population is classified:The estimation of the evolutionary state enables individuals to evaluate their search ability accurately. In the following part, adaptive search strategies will be added for different individuals based on their evolutionary states.In religious competition sessions, inheritors are comprehensively considered to decide whether to become new preachers or not. The new preachers selected by this procedure only consider the fitness and location of an individual in the current iteration. Still and all, the procedure ignores the exchange of information with themselves and the population experience. The search process can be more accurate and efficient if we can make better use of the previous knowledge that individuals and other individuals have searched. As a consequence, the randomly distributed time-delay is introduced into the search model. This strategy enables preachers to enhance the use of the accumulated with better accuracy and pursue a stronger capability of avoiding local trapping problems. Hence, the idea of distributed time-delay based on the PSO algorithm is introduced to the location update formula of POA.On the basis of the evolutionary state and distributed time-delay proposed above, this paper proposes a new location update strategy for POA. The novel orientation adjustment strategy consists of four different states, depending on different directions of historical information. The advantage of the method is to make individuals with adaptive capabilities more targeted to search and keep a proper balance between exploration and exploitation. In the exploration state, preachers are expected to search the entire search space as comprehensively as possible to get more optimal solutions. The historic global optimal solution contains a wealth of information, which is distributed in different locations in the search space. Therefore, the distribution time-delay which leads to randomly selected global optimal solutions is added to preachers’ position update formula. Eventually, the orientation adjustment factor is set as In the exploitation state, preachers should focus on the optimal solution which has been found. They improve search efficiency by learning from the individual optimal solution quickly. The record of the preachers’ behavior in a valuable location is stored in the individual optimal solution, which can improve the search efficiency of the new individual greatly. Therefore, the distribution time-delay which leads to randomly selected individual optimal solutions is added to preachers’ position update formula. The orientation adjustment factor is set as In the convergence state, preachers are encouraged to gather around into the global optimal region as soon as possible. To implement this procedure, they ought to reduce the proximity of the other direction, which can be indicated by setting the orientation adjustment factor in the other direction to zero. The orientation adjustment factor is set as In the escape state, preachers are trying to escape from the region around the local optimal. As a consequence, they learn from the entire history to make enough movement. This procedure expresses by double approximation to the individual and global optimal. The orientation adjustment factor is set as It is worth noting that since the proposed strategy ought to be combined with iteration-based algorithms, it needs historical iteration messages of the individuals as the current agents are guided by previous optimums. The mechanism is shown in (1)Initialize the parameters including the population size, the max number of iterations, the search dimension, the number of inheritors, and the number of elite individuals.(2)Initialize the population.(3)Record global and local optimal historical information.(4)Transmit the location information to inheritors using Equation (1).(5)Select new preachers by the mechanism of religion competition using Equation (2).(6)Estimating the evolutionary state of the new preachers using Equations (4)–(6).(7)Adding distributed time-delay to the preachers based on their evolutionary state.(8)Search for a new solution by the mechanism of religion development by Equation (3).(9)Repeat Steps 3 to 5 till the algorithm meets the max number of iterations.(10)Output the result.The implementation process of HPOA is as follows:In this section, the HPOA algorithm is utilized to optimize the basic Kapur entropy algorithm to improve the defects. Firstly, we present a brief description of the multilevel thresholding segmentation. Then, we describe the Kapur entropy and the fitness function. In the end, we present the proposed HPOA-based color image segmentation method.Multilevel thresholding segmentation utilizes a threshold group to divide the pixels of each gray level into different categories. This method can not only distinguish the foreground and background of the image but also achieve great results when the image is complex and need to be extracted. In the interim, this method is also suitable for color images. In this method, if the threshold group describe as Kapur entropy is an automatic threshold selection technique based on the maximization of entropy. It has clear mathematical meaning and can retain the small details excellent, which makes it extensively applied in complex image segmentation. Assuming that The combination maximizing To obtain the segmentation threshold more quickly and accurately, the HPOA algorithm is employed to optimize the Kapur entropy. The powerful capability of HPOA obtains more accurate segmentation thresholds, with the segmentation accuracy improving. The flow chart of the proposed segmentation algorithm based on HPOA is shown in To verify the performance of the proposed algorithm, eight well-known benchmark functions are adopted. These functions are categorized into three groups: (1) multimodal, (2) fixed dimension multimodal, and (3) unimodal problems.In the test set, functions F1 to F3 are multimodal with a large number of local optima. The multimodal problems are ordinarily employed to evaluate the exploration ability since a large number of local optima increases the probability of stagnation; Functions F4 to F6 are multimodal but with low dimensions. The fixed dimension multimodal problems have fewer local optima as the dimension is less as compared to the multimodal problems. These problems examine the balance between local and global search abilities; Functions F7 to F8 are unimodal, only one global optimum is present. The unimodal problems evaluate the capability of exploitation. These functions have been used to evaluate algorithms in ,42,52,53(1)Traditional POA algorithm .(2)The state-of-the-art WOA algorithm, in which is flexible and requires fewer parameters to be adjusted .(3)The classical representative of swarm intelligence: PSO .(4)A newly proposed algorithm named SCA, containing several adaptive variables to ensure a balance between exploration and development .(5)A novel natural heuristic algorithm: MVO, designed for engineering structure design .(6)MFO, inspired by moth navigation which has advantages in solving unknown space problems .(7)An interesting algorithm, ALO, along with characteristics of few adjusting parameters and high accuracy .All of the algorithms are developed by using Matlab R2016b and implemented on Windows 7 environment on a computer having Intel CPU @2.20 GHz and 12 GB memory. The proposed HPOA is compared with several well-known heuristic algorithms. Each of them contains different characteristics, including:More precisely, the population size of all algorithms is set as 20, and the max number of iterations is 1000. Each algorithm runs 10 times to avoid contingency.As mentioned above, eight benchmark functions are used to evaluate the performance of the proposed HPOA algorithm. In this paper, the comprehensive performance of the algorithm ought to be analyzed by three kinds of criteria: (1) accuracy, (2) convergence, and (3) statistical analysis.The accuracy criterion of each algorithm is determined by average value and standard deviation. The competitive results between HPOA and other algorithms are discussed as follows.In terms of accuracy, a higher average value signifies better capability. It is found that HPOA is outstanding to comparison algorithms from In terms of convergence, it is observed that HPOA is also the most competitive from In terms of statistical analysis, we conduct statistical tests to verify whether the improved algorithm is significantly better than the original algorithm, which is proposed in . And welBased on above demonstration, HPOA achieves the best performance on In this section, the HPOA-based segmentation algorithm is employed in color images. The purpose of the experiments is to investigate whether the proposed method is competent in producing high-quality segmented images.We conduct the experiments on three groups of bench images in The quality of the segmented images can be evaluated by the image evaluation metrics as follows:FSIM is an quality assessment (IQA) metrics to measure the image quality automatically . The basPSNR is a renowned image assessment index, which computes the peak signal-to-noise ratio between two images . This raSSIM is an index to measure the similarity of two images, which takes into account various factors such as brightness, contrast, and structural similarity . It findTo evaluate the result of the algorithms, the quality of the segmented images is quantitatively analyzed by FSIM, PSNR, and SSIM. The proposed method provides segmented results in RGB channels. Considering of these metrics require that the compared images ought to have the same classes, the segmented images are grayed to match the ground truths during the evaluation. The results of FSIM, PSNR, and SSIM of each algorithm are presented in (1) In the FSIM table, the proposed algorithm obtains the most competitive results in almost all cases (66 out of 72 cases). These values indicate the performance of the proposed algorithm is the most outstanding. It is observed that the images segmented by the proposed method have higher similarity and lower distortion with the reference images.(2) In the PSNR table, although there are only small differences between the algorithms in low dimensions (Dim = 5), HPOA still shows superiority over the others on nearly all the images (21 out of 24 cases). With the number of thresholds increasing, the results become diverse. As exhibited that HPOA can commonly provide the best results as well , and the PSNR values of HPOA significantly increase.(3) In the SSIM table, it is perceived that the proposed method outperforms all the other algorithms on various bench images, since the SSIM index obtains the highest values for majority of cases (68 out of 72 cases). The result indicates that the images segmented by IPOA are more similar to the human segmentation images in structural similarity.Except for the image evaluation indicators, the value of the function fitness can also be a significant index to evaluate the performance of algorithms. Based on the above demonstration, the competitive values of FSIM, PSNR, SSIM, and fitness values prove the high accuracy of the HPOA-based color image segmentation algorithm. Notably, the superiority of the proposed algorithm becomes more and more remarkable as the number of thresholds increases compared to other algorithms. For this reason, the HPOA-based segmentation algorithm can accomplish the complex tasks of color image segmentation effectively, as well as provide a more precise technique for multilevel segmentation. The segmentation images of the proposed algorithm are shown in This paper presents a hybrid preaching optimization algorithm based on Kapur entropy for complex image segmentation problems. HPOA evaluates the evolutionary state of each population and adjusts the updating model adaptively. Even more noteworthy is that distributed time-delay containing historical information of previous personal and global best is introduced into HPOA. The expression of time-delay draws lessons from the PSO algorithm. Therefore, this strategy strengthens the diversity of the population and prevents premature convergence efficaciously. Eight classical test functions are employed to evaluate the comprehensive performance of HPOA. The validity and stability of the hybrid algorithm are verified by qualitative and quantitative methods. The experiment results reveal that HPOA has better accuracy, stability, and a faster convergence speed than other algorithms. Simultaneously, it has a statistically significant improvement. Eventually, combining HPOA with conventional Kapur entropy, an HPOA-based color image segmentation algorithm is proposed. All segmentation experiments are performed on three categories of images from the Berkeley dataset , includiAs future work, our goal is to further improve HPOA for the MRI image segmentation problem . We also


To improve the path tracking accuracy and yaw stability of distributed drive autonomous vehicles (DDAVs) under extreme working conditions, a cooperative lateral motion control method based on the dynamic boundary is proposed to prevent different road adhesion conditions from affecting the motion stability of DDAVs. Based on the analysis of the DDAV lateral dynamics system coordination mechanism, a dynamic boundary considering the pavement adhesion coefficient is proposed, and the Lateral Motion Synergistic Control System (LMSCS) is designed. The LMSCS is divided into the coordination, control, and executive layers. The coordination layer divides the control domain into the stable, quasi-stable, and unstable domains by the dynamic boundary, and coordinates the control strength of the path following control and yaw stability control. In the control layer, the path following control and yaw stability control laws are designed based on the global fast terminal sliding mode. The executive layer estimates the expected steering wheel angle and expected additional wheel torque. Joint simulations under double line shifting conditions confirmed that LMSCS reflects the impact of the road attachment conditions and improves the path tracking accuracy and vehicle yaw stability. The LMSCS has better overall performance than existing lateral motion control methods. Additionally, DDAVs combine torque vector control and path tracking control to improve the vehicle yaw stability and lateral movement safety while improving the path tracking accuracy. As the research objects of distributed drive vehicles and autonomous vehicles, torque vector control4 and path tracking control6 have become research hotspots in the field of DDAV lateral dynamics control8. How the torque vector control and path tracking control work together to improve the comprehensive lateral dynamics performance of DDAV has become a challenge for researchers.With the rapid growth of car ownership, social problems such as traffic safety, traffic congestion, and environmental pollution have emerged. Moreover, the requirements with regard to the safety, efficiency, and mobility of automobiles are constantly expanding9 and Stanley10 methods are classic path tracking control algorithms based on automotive kinematics, and have simple and real-time characteristics, but do not consider the vehicle’s dynamic performance, which is generally achieved at low speed and under other simple working conditions. To adapt to the automatic vehicle’s complicated and variable driving conditions and consider the nonlinear dynamic characteristics of automatic driving vehicles, the path-based path tracking control method has been extensively investigated12. The dynamic path tracking control method often uses a conventional linear two-degrees-of-freedom (2-DOF) model as a reference model14, and the employed control algorithm includes the PID algorithm15, model prediction algorithm16, sliding mode control5, and reinforcement learning algorithm17.As a key technology for automatic driving, path tracking control focuses on controlling the car steering system to drive the car along the expected path while ensuring the car’s stability and travel comfort. The pure pursuit18. Many studies on the yaw stability control of distributed drive vehicles have been conducted26. The yaw speed and sideslip angle of the traditional linear 2-DOF vehicle dynamics model under steady-state steering are typically considered as the control reference targets, and the objective of improving the yaw stability of the vehicle is achieved by reducing the yaw speed and sideslip angle under complex conditions26. The typical algorithms used in Yaw Stability Control (YSC) research include PID control20, fuzzy control22, adaptive control23, sliding mode control25, and optimization control26.Distributed drive vehicles achieve yaw stability control without affecting the longitudinal speed through torque vector control28. In27, the adaptive high-order control law and pseudo-inverse low-order control law are designed for DDAV path tracking control. An adaptive fuzzy sliding mode control law is designed for DDAV yaw stability control. However, less consideration is given to the coordinated control between path tracking and yaw stability. In13, a hierarchical control scheme is proposed to improve the trajectory tracking ability and lateral stability of DDAVs, and the working areas of PFC and YSC are coordinated based on the working conditions of tires. In28, a robust DDAV path following control strategy based on the nonsingular terminal sliding mode and active disturbance rejection control is presented. The simulation results obtained under the double line change condition reveal that the proposed control strategy ensures that the vehicle quickly and accurately follows the reference path, and has strong robustness. Currently, there are many algorithms for PFC and YSC integrated control. The sliding mode control has the advantages of fast response, being insensitive to parameter changes, no online identification of the system, and simple physical implementation29, which make sliding mode control suitable to vehicle dynamic performance control. Therefore, sliding mode control has been extensively investigated, and many new control algorithms based on sliding mode control theory, such as Global Fast Terminal Sliding Mode (GFTSM)31, have emerged. The GFTSM algorithm has the following features: the system can converge to the equilibrium state within a limited amount of time, and the convergence time can be designed. The continuous control law is adopted, and does not include a switching term, which is beneficial for reducing the chattering phenomenon of sliding mode control. The GFTSM has good robustness to system uncertainties and disturbances29, and should be further applied to practical DDAV dynamic performance control.Although the above-mentioned path tracking control method or yaw stability control method improves the path tracking accuracy or yaw stability of DDAVs, the integrated control of Path Following Control (PFC) and YSC more effectively improves the lateral motion performance of DDAVsThis paper proposes a DDAV coordinated control method based on the dynamic boundary, and a DDAV coordinated control system is designed. The main contributions of this study are as follows: (1) In existing studies, the yaw velocity and sideslip angle of the traditional linear 2-DOF vehicle dynamics model under steady-state steering are typically considered as reference targets for stability control, without considering the road conditions. To solve this problem, a 2-DOF vehicle dynamics model with a road adhesion coefficient is designed. Based on the dynamic model and tire force limit, the dynamic boundary describing the stable state of DDAV lateral movement is derived to consider the influence of additional variables. The control domain is divided into a stable, quasi-stable, and non-stable region through the dynamic boundary to more accurately identify the stable state of vehicle lateral movement. (2) Based on the recognition of DDAV’s stable state of lateral motion, a coordinated control law for PFC and YSC is constructed to fully active the DDAV's multi-degree-of-freedom control advantage such that LMSCS could improve the path tracking performance and vehicle yaw stability simultaneously. (3) The GFTSM-based wheel angle control law and additional yaw moment control law are designed. Compared with the existing lateral motion control method based on the traditional sliding mode algorithm, it is demonstrated that the adoption of GFTSM in LMSCS can achieve better lateral motion control. (4) The effectiveness and superiority of the proposed DDAV lateral motion co-control method based on the dynamic boundary are verified by simulation under different vehicle speeds, road adhesion coefficients, and double lane shifting conditions.The rest of this paper is organized as follows: In “The dynamic-boundary-based LMSCS is a model-based control strategy. The premise of LMSCS design is to construct a suitable DDAV dynamic model and analyze the dynamic cooperative control mechanism of PFC and YSC to provide the theoretical basis for realizing synergistic control.In path tracking control, the vehicle dynamics model should be able to describe the DDAV lateral motion, yaw motion, and relationship between the heading angle and above-mentioned motions. Therefore, the 2-DOF linear vehicle dynamics model is adopted. This model has a simple expression and reflects the most basic characteristics of the car’s curve motion, and can be used to calculate the main parameters reflecting the vehicle’s lateral motion performance. The model satisfies the design requirements of the path tracking control law and ensures that the real-time performance of the algorithm is satisfactory.32:The differential equations of the 2-DOF vehicle motion are as followsPath tracking control mainly aims to limit the lateral deviation between the vehicle and given reference path. To facilitate the description of the relationship between the vehicle and the road, the road coordinate system is established as follows: The origin mentclass2pt{minimThe DDAV speed along the 32:To construct the DDAV yaw stability control law, a 2-DOF vehicle dynamics model, which contains the tire force of each wheel and an additional yaw torque that reflects the lateral and yaw motion of the vehicle, respectively, is established as follows33, which is an analytical model derived from the force balance relationship, has fewer custom parameters, expresses the nonlinear characteristics of the tire, and is widely used for vehicle lateral motion control35.The tire force contained in the 3-DOF vehicle dynamics model can be calculated using the Dugoff nonlinear tire modelThe longitudinal tire force is expressed as follows:The lateral tire force is expressed as follows:The expression of The objective of DDAV path tracking control is to make the driving position and direction as consistent as possible with the target path, and the DDAV typically realizes the target path direction tracking by controlling the vehicle’s yaw angle. In DDAV yaw stability control, the yaw angular velocity is typically considered as a control objective, and can be obtained from the definition of the yaw angle and yaw angular velocity, as follows:According to Eq. , PFC andIn this study, the dynamic boundary, which introduces the pavement adhesion coefficient, is used to assess the stable state of the DDAV lateral motion and provide the vehicle stable state criterion for YSC and PFC synergistic control. The dynamic boundary is described by the yaw rate and sideslip angle to evaluate the lateral stability of the DDAV. Under general driving conditions, the lateral dynamic response of the vehicle is within the linear range and exhibits good handling stability. The steady-state response of the vehicle in the linear range is determined as the stability boundary. When the vehicle is subjected to increasingly more severe working conditions, the lateral dynamic response exhibits increasingly more obvious nonlinearity, and the vehicle is eventually exposed to lateral instability. The state wherein the tire lateral force reaches saturation and is about to lose lateral stability is determined as the unstable boundary. Both the stable boundary and unsteady boundary model introduce road adhesion coefficients such that the designed dynamic boundary can more reasonably describe the stability of the DDAV under different road conditions. As shown in Fig. 36. The μ-2-DOF vehicle dynamics model was used to derive the stability boundary.To introduce the road adhesion coefficient into the stable boundary of the dynamic boundary, a 2-DOF vehicle dynamics model considering the road adhesion coefficient was derived. In this paper, the new 2-DOF vehicle dynamics model is referred to as the μ-2-DOF vehicle dynamics model. Based on Dugoff tire dynamics model, the μ-2-DOF vehicle dynamics model was derived by referring to the method of establishing a linear 2-DOF vehicle dynamics model36, the longitudinal velocity of the vehicle is constant and the driving force of the wheel is small. Therefore, the wheel slip rate is small and the slip rate is ignored. According to Eqs. 19) can b36:The sideslip angles of the front and rear tires are expressed as follows36, δ is small, therefore, According to the assumptionsing Eqs.  and 22)36, δ is ing Eqs. , the \doBy combining Eqs.  and 27)27), whenHere, 37 are introduced:37, and have been used as the maximum limit of the yaw rate and sideslip angle in many yaw stability control studies. In37, the road attachment condition of the side slip angle and the yaw rate when the tire lateral force reaches saturation is proposed and determined as the maximum limit, which coincides with the definition of the unstable boundary in this study. Therefore, In this study, the unstable boundary is defined when the lateral force of the tire reaches saturation and the vehicle is about to become laterally unstable under the condition of road adhesion. In this state, the vehicle dynamic response is strongly nonlinear, and the assumption of the stable boundary derived above is no longer valid. Hence, it is very difficult to establish the unstable boundary through theoretical derivation, and the following empirical formulasTo ensure the clarity of the control system structure and the independence of each sub-strategy function, and facilitate the system organization, LMSCS adopts the layered control architecture, which is divided into the coordination, control, and execution layers. The architecture of the designed DDAV lateral motion control system is shown in Fig. The main objective of the coordination layer is to identify the control domain wherein the current vehicle state is located with a dynamic boundary, and then allocate the control strength of PFC and YSC.Combined with the stable boundary and unstable boundary, the LMC control domain is identified. When the vehicle is in the stable domain, the following relationship holds:When the vehicle is in the unstable region, the following relationship holds:In other cases, the vehicles are in the quasi-stable region.The YSC ensures that the vehicle is driven within the stability domain as much as possible. The control rules of YSC design are as follows: when the vehicle is driven within the stability domain, YSC control is not carried out. When the vehicle travels in the quasi-stable region, YSC controls the vehicle to enter the stable region; when the vehicle enters the unstable domain, it is first controlled to enter the quasi-stable domain. According to the above-mentioned rules, the objectives of YSC based on dynamic boundaries are designed as follows:Path tracking is the basic function of the DDAV for normal driving, and yaw stability is a prerequisite for safe driving. The design criterion of LMSCS is to improve the path tracking accuracy of the DDAV to ensure the vehicle yaw stability. The yaw stability of the DDAV is evaluated by the overall error of the yaw rate and sideslip angle tracking, as follows:The yaw rate focuses on the oscillation and stability of the vehicle itself, and the sideslip angle focuses on the trajectory maintenance of the vehicle. The yaw rate also reflects the vehicle’s ability of driving in the direction of the desired path at the next moment. Controlling the yaw rate to track the ideal signal improves the vehicle yaw stability and the path tracking accuracy. To improve the comprehensive performance of DDAV lateral motion, the coordination of path tracking and yaw stability control in LMSCS is carried out as follows. The PFC is completely aimed at improving the path tracking accuracy to ensure that the vehicle is driven on the road. The YSC determines the control target according to the vehicle yaw stability state. When the yaw rate and sideslip angle are both in the stable region, the vehicle does not require yaw stability control. The YSC is assisted to improve the DDAV path tracking accuracy as the main objective, without side slip angle control. When the yaw rate is in the quasi-stable region and the slip angle is in the stable region, YSC improves the DDAV path tracking accuracy and yaw stability simultaneously, and side slip angle control is not required. In other cases, the main objective of YSC is yaw stability control. Generally, the sideslip angle is small, and even if slight disturbance exists, the sideslip angle automatically returns to a stable state. At this time, the yaw stability of the vehicle is mainly determined by the yaw rate and the yaw rate should mainly be controlled. When the sideslip angle is large, the operation of the steering wheel hardly makes the vehicle produce additional yaw moment, the vehicle is difficult to control, and the sideslip angle should mainly be controlled. With consideration to the above-mentioned factors, the values of In Eq. , the squd by Eq. . The cooThe control layer solves the design problems of the wheel angle control law and direct yaw moment control law. The sliding mode control law is designed based on the response results of the coordination layer and GFTSM.Sliding mode design:The exponential approaching law is adopted to enhance the system’s anti-chattering performance, as follows:29, and the saturated function is expressed as follows:When the modeling uncertainty and disturbance are large, the switching gain Equation  can be rAccording to Eq. , the accAccording to Eq. , the folCombined with Eqs. , 41), a, a41), aIn Eq. , when \dGlobal fast terminal sliding mode arrival time analysis:According to Eq. , the fol29, in the process of reaching the sliding mode, the time of convergence from any initial state According toConsidering the exponential approaching law, the symbol ≈ is used in the equation. By setting parameters Stability analysis of PFC based on GFTSM:The Lyapunov function is defined as follows:33.Through derivation and calculation, Sliding mode design:By combining Eqs.  and 51)51), the Global fast terminal sliding mode arrival time analysis:According to Eq. , the folSimilarly, in the process of reaching the sliding mode, the time of convergence from any initial state By setting parameters Stability analysis of YSC based on GFTSM:The Lyapunov function is defined as follows:33.Through derivation and calculation, The actuator control target distribution is realized in the executive layer. The steering wheel angle is calculated according to the average front wheel angle calculated by the control layer. The additional driving torque of each wheel is calculated according to the additional yaw torque of the body calculated by the control layer.Figure With regard to the coupling relationship of the tire forces, generally, as the wheel load increases, the maximum longitudinal or maximum lateral force becomes greater. To avoid one of the tire forces reaching saturation first, a four-wheel additional torque distribution strategy based on the wheel load is adopted. The longitudinal forces generated by the additional driving torque of the wheels are as follows:By combining Eq. , the comTo verify the effectiveness and superiority of the LMSCS based on the dynamic boundary and GFTSM, a sample DDAV model was used to verify the simulation. The main parameters of the DDAV model are listed in Table The double-side double-line shifting condition was adopted as the simulation test condition, and the length of the double-line shifting on both sides is different. The designed path is shown in Fig. For the convenience of description, the DDAV lateral motion control system, which only adopts the PFC based on GFTSM, is considered as the Primary Lateral Motion Control System (PLMCS). By comparing the simulation results obtained for PLMCS and LMSCS, it is confirmed that LMSCS fully enables the lateral dynamic performance control potential brought by the multi-degree-of-freedom controllable advantages of the DDAV. Finally, the DDAV path tracking accuracy and yaw stability are improved simultaneously. Due to the limited space of the paper, only the data curves under some working conditions are shown below, and the data statistics under all working conditions are shown in Table When PLMCS is used, the estimated DDAV As shown in Figs. When DDAV is driving in unstable conditions, when PLMCS is used, the yaw rate γ and sideslip angle β are shown in Figs. .By sorting and comparing the simulation results obtained using PLMCS and LMSCS, respectively, the summary of the simulation is presented in Table entclass1pt{minimaThe above conclusions reveal that the DDAV lateral motion cooperative control method based on the proposed dynamic boundary fully activates the advantages of DDAV active steering and distributed drive multi-degree-of-freedom cooperative control. Thus, LMSCS improves the path tracking performance and vehicle yaw stability simultaneously.To date, many studies have successfully applied the lateral motion control method based on the Quasi Sliding Mode (QSM) for the lateral motion control of autonomous vehicles or distributed driving vehicles, and the effectiveness of the algorithm has been verified by simulation or tests on actual vehicles. In this study, the vehicle lateral motion control system based on QSM was considered as the Lateral Motion Traditional Control System (LMTCS). To further verify the superiority of LMSCS, the simulation results of LMSCS and LMTCS were compared and analyzed.Through a calculation process similar to that of GFTSM-based PFC, the front wheel average angle control law in LMTCS can be obtained as follows:The YSC in LMTCS also adopts the exponential approach law, and the saturated function r as Eq. . ThroughParameter A is expressed as follows:27.The DDAV lateral dynamics control performance can be evaluated by the control errors Integrate the absolute value of the error in the simulation time period:The absolute value of the error is weighted by time and integrated within the simulation time period:Integrate the absolute value of control actuation within the simulation period:In Eqs.  and 67)67), \doc. In Eq. , \docume38.To fully describe the control performance of LMTCS and LMSCS and carry out a reasonable evaluation, ITAE is applied to supplement the evaluation effect of IAE, because, although IAE reflects the sum of control errors in the control process, it cannot reflect the speed of control convergenceThe simulation results reveal that both LMTCS and LMSCS realize DDAV lateral motion control. The results of various evaluation indices obtained by simulation are presented in Table This paper proposes a DDAV lateral motion cooperative control method based on the dynamic boundary and GFTSM. The dynamic boundary of the proposed method considers the influence of road adhesion and wheel speed on the tire sidetracking characteristics, which is beneficial for accurately identifying the stable state of vehicle lateral motion under different road conditions and dividing the control domain. The PFC and YSC cooperative mechanism was designed based on the recognition results obtained for the vehicle’s lateral motion stability. The PFC and YSC control laws based on GFTSM were designed. The simulation results confirm that the designed control method improves the vehicle path tracking when the DDAV runs stably, and improves the vehicle’s yaw stability when the DDAV runs under unfavorable conditions and is about to become unstable. Compared with the DDAV lateral motion control system based on the traditional sliding mode algorithm, the DDAV lateral motion control system has better comprehensive DDAV lateral motion control performance. In future work, the dynamic boundary will be extended, and additional DDAV motion stability state evaluation parameters will be introduced based on the yaw velocity and sideslip angle such that the dynamic boundary can more comprehensively and accurately identify the vehicle stability state and provide state criteria for vehicle motion control.


Aging is associated with changes in the sensory-motor system that could lead to dynamic instability. In fact, postural control deficits have been proposed as an early indicator of frailty. Measurements of the displacement of the center of pressure (COP) using pressure mat data are useful tools to determine postural steadiness. Companion dogs represent a powerful model to study aging in people because they share our environment and experience similar age-related diseases. To date, the effect of aging on postural control in dogs has not yet been evaluated. The aim of this study was to determine the correlation between age and the displacement of the COP in dogs during quiet standing. Due to the diversity of life expectancy in dogs according to their body size, age was normalized as a fraction of the predicted life expectancy. Dogs older than 75% of their life expectancy (n=18) were asked to stand on a pressure mat for 8 seconds per trial during at least five trials. Only the frames where the dogs were standing still and facing forward were analyzed. Age as a fraction of life expectancy was significantly correlated (p<0.05) with the Medio-lateral Range, Root-Mean-Square Distance, 95% Confidence Ellipse, and Total Sway Area of the COP. These results show that, as in humans, aging in dogs is associated with postural control deficits and therefore reinforce the dog as a suitable model for translational studies of aging and postural steadiness.


Deep-learning (DL)-based methods are of growing importance in the field of single image super-resolution (SISR). The practical application of these DL-based models is a remaining problem due to the requirement of heavy computation and huge storage resources. The powerful feature maps of hidden layers in convolutional neural networks (CNN) help the model learn useful information. However, there exists redundancy among feature maps, which can be further exploited. To address these issues, this paper proposes a lightweight efficient feature generating network (EFGN) for SISR by constructing the efficient feature generating block (EFGB). Specifically, the EFGB can conduct plain operations on the original features to produce more feature maps with parameters slightly increasing. With the help of these extra feature maps, the network can extract more useful information from low resolution (LR) images to reconstruct the desired high resolution (HR) images. Experiments conducted on the benchmark datasets demonstrate that the proposed EFGN can outperform other deep-learning based methods in most cases and possess relatively lower model complexity. Additionally, the running time measurement indicates the feasibility of real-time monitoring. Environment research plays an important role in human daily activities. According to the results of environmental research, further analysis and effective solutions to existing problems can be made ,2. ImageWith the resurgence of convolutional neural networks (CNNs) , deep leIf one visualizes the feature maps of each convolutional layer of a well-trained CNN, it’s easy to find out that there is a certain relationship among the feature maps generated by the same convolutional layer, and this relationship is manifested by the similarity between some feature maps. As shown in In order to fill this gap, we design the efficient feature generating block (EFGB), which can leverage the intrinsic features to produce more feature maps in an economical way. Furthermore, to improve the performance of the model, we introduce the staged information refinement unit (SIRU) for better feature extraction. We establish the local residual module (LRM) based on the SIRU and the main body of proposed efficient feature generating network (EFGN) is constructed by stacking several LRMs. More details will be described in The efficient feature generating block is proposed, which can generate more feature maps in an efficient way, so the network can achieve high performance while keeping low computation complexity.The Super Resolution Efficient Feature Generating Network is proposed, which introduces the staged information refinement unit to further boost the network performance.In summary, there are two main contributions of this work:The rest of the paper is organized as follows: Thanks to the rapid development of high-performance computing devices and the appearing of large-scale image datasets ,17, traiWith the emergency requirements of applying well-trained CNNs to embedded devices, lots of lightweight models have been proposed. In SqueezeNet , the dimbileNets  designedbileNets  used chabileNets  with intIn this section, we give the overall network architecture and introduce the workflow of proposed method at first. Then we describe the local residual module in a top-down manner, which is the core part of our proposed method. At the beginning, we give a concise description of the symbols in As depicted in k-th LRM, k-th LRM respectively. In the RM, all outputs from previous LRMs are collected by the feature gathering block with concatenation operation, then a Previous studies ,21,22,26As shown in k-th LRM as s-th EFGB of first SIRU in the k-th LRM, the s-th stage, which is the concatenation along channel dimension of the input and output of k-th LRM can be obtained byk-th LRM. As shown in In deep CNN, feature maps from relative shallow layers usually contain abundant texture information. It’s vital to use these informative feature maps when we are processing reconstruction tasks. Moreover, previous works ,27 have c denotes the number of channels, w and h are the width and height of X respectively. An ordinary convolution layer is firstly applied on the input data to yield the intrinsic feature maps M is transformed to its potential counterpart. In practice, the transformation is implemented by depthwise convolution (DWC). The DWC is a special case of group convolution, which process each feature map separately. In other words, the group of DWC is equal to the number of input channels. Finally, the output Y of EFGB is consist of the identity mapping of M and its variant, it can be expressed asThe mainstream SR methods using conventional convolution would produce redundant feature maps as shown in ge SR in . As illuSince the DIV2K  dataset k is the kernel size of convolution filter and T LRMs, the input and output both are feature maps with channel number The detailed structure of LRM is described in es. ReLU  functionhttps://github.com/gokuson77/EFGN (accessed on 17 May 2021).The LR training images are downsampled from their HR counterparts with bicubic interpolation. We crop the image patches with a size of ptimizer  is applic denotes the number of channels and For a more intuitive comparison, we calculate the parameters and computational cost of a conventional convolution layer and our proposed EFGB. Assuming both of the input and output data have a shape of As the core of our proposed network, the LRM is worthy to be comprehensively investigated. Specifically, the parameter setting of SIRU in each LRM and the number of LRM  in the network are discussed.To find a proper parameter setting, we fix the input channel of first EFGB in SIRU as 64 and T as 4. Then we divide the model to three types according to the output of EFGB in SIRU. Concretely, the output channel of EFGB in expansion model is increased by 32. On the contrary, the reduction model reduces the output channel of EFGB by 32 while the basic model maintains the number of output channel as the same of input channel. The experiment results are presented in We then investigate the impact of the number of LRM. As illustrated in To verify the effectiveness of proposed staged information refinement unit (SIRU) and efficient feature generating block (EFGB), we construct two networks for ablation study. It is worth noting that both of the ablation models follow the same training settings as the EFGN to ensure the credibility of experimental results.The first one called EFGN_NS has the modified SIRU as illustrated in The other model named EFGN_NE uses ordinary convolution layers for features extraction instead of the EFGB. As we have analyzed in We compare our method with several DL-based SR algorithms. The quantitative comparison is evaluated by metrics mentioned above (PSNR and SSIM). For a comprehensive study, we also investigate the network parameters and computational costs   as the oIn this paper, we propose a lightweight image super-resolution network for real-time environment monitoring. Concretely, a novel efficient feature generating block is designed to fully utilize the redundancy among feature maps of the same layer. Additionally, the staged information refinement unit is introduced to explore hierarchical information reuse, which can further boost the reconstruction performance. Extensive experiments conducted on the benchmarks demonstrate that the proposed EFGN surpasses other DL-based methods while balancing the SR reconstruction performance with network parameters and computational costs. Meanwhile, our method is capable of real-time monitoring since it has fast running speed. Although the model shows excellent performance, a room of it to improve still exists. First and foremost, we only use L1 loss to train our network, which cannot restore the high frequency details in some cases. In order to produce more visually friendly images, we plan to incorporate generative adversarial network (GAN) and use joint loss to train our network. Besides, the gradient information in LR images have been proven to be helpful in the image SR task. In the future, we will make an attempt to exploit the informative gradient features for better reconstruction.


The Track-Hold System (THS) project, developed in a healthcare facility and therefore in a controlled and protected healthcare environment, contributes to the more general and broad context of Robotic-Assisted Therapy (RAT). RAT represents an advanced and innovative rehabilitation method, both motor and cognitive, and uses active, passive, and facilitating robotic devices. RAT devices can be equipped with sensors to detect and track voluntary and involuntary movements. They can work in synergy with multimedia protocols developed ad hoc to achieve the highest possible level of functional re-education. The THS is based on a passive robotic arm capable of recording and facilitating the movements of the upper limbs. An operational interface completes the device for its use in the clinical setting. In the form of a case study, the researchers conducted the experimentation in the former Tabarracci hospital . The case study develops a motor and cognitive rehabilitation protocol. The chosen subjects suffered from post-stroke outcomes affecting the right upper limb, including strength deficits, tremors, incoordination, and motor apraxia. During the first stage of the enrolment, the researchers worked with seven patients. The researchers completed the pilot with four patients because three of them got a stroke recurrence. The collaboration with four patients permitted the generation of an enlarged case report to collect preliminary data. The preliminary clinical results of the Track-Hold System Project demonstrated good compliance by patients with robotic-assisted rehabilitation; in particular, patients underwent a gradual path of functional recovery of the upper limb using the implemented interface. The Track-Hold System (THS) project, developed in a healthcare facility and therefore in a controlled and protected healthcare environment, is carried out in the broader framework of Robotic-Assisted Therapy (RAT). RAT represents an advanced and innovative rehabilitation method, both motor and cognitive, and makes use of active, passive, and facilitating robotic devices. Sensors often equip robotic devices for detecting and tracking both voluntary and involuntary movements, and specially developed multimedia protocols to achieve the highest possible level of functional re-education.Like conventional rehabilitation methods , the RAT requires close collaboration between the members of the Multidisciplinary Team, which in the RAT, besides the Doctor and the Therapist, requires the Physiologist and Biomedical and Computer Engineer .Track-Hold is a passive robotic arm capable of recording and facilitating the movements of the upper limbs. The robotic device was created in its mechanical and sensor components by the “Wearable Robotics” Laboratory, a spin-off of the Institute of Biorobotics of the Sant’Anna School of Advanced Studies (Pisa), and completed by an operational interface for its use in the clinical field, by the Institute of Science and Technology (ISTI) “Faedo” of the National Research Council (CNR) of Pisa, based on specialist clinical advice.in the absence of gravitational force or with opposing gravitational pull ;in the presence of the natural force of gravitation;in an intermediate condition.Thanks to a solution based on levers and weights, the arm allows to move a person’s upper limb with a push from the bottom up, varying the weight. Track-Hold records the movements of the limb of the subject under various conditions and multiple settings. Namely, Track-Hold settings permit the movement of the subject’s limb in the following conditions:The levers, the weights, and the limb constitute a set of scales that allow it to function even in equilibrium conditions. These characteristics, especially the functioning with the limb in weightlessness, make it suitable for the rehabilitation and passive training of subjects with motor difficulties of the upper limbs, resulting from previous ischaemic events, progressive neurological pathologies, and extreme loss of strength—reduction in muscle mass or, again, coordination or difficulty in organizing purposeful movements.The execution of the limb movements of the patient to be re-educated is facilitated by the passive mechanics of the balance wheel, as the upward thrust allows the patient to move the limb with minimal muscular effort. The clinical evaluation made by the physician is used to adapt the amount of support, determining how much weight should be relieved. The device consists of arms connected by joints. Each of the joints is equipped with a patented precision angle sensor .The integrated Track-Hold technology allows recording the arm’s movements through a two-way association between the actions of the upper limb and those of the robotic arm. The recorded kinematic data are objective and can be used, among other things, to estimate improvements and progress of personalized therapeutic programs.The manipulator of the device, consisting of a handle, is equipped with a pressure sensor that allows detecting when it is pressed with the hand . The sampling rate of the positions is 100 Hz, which is sufficient for correct motion detection. The device is connected to a computer via a USB interface.Track-Hold has been equipped with an agent software that acts as an operational interface. CNR-ISTI Signals and Images laboratory (SI-Lab) set up the interface in the laboratory in collaboration with the Clinicians (this activity took place between July 2020 and October 2020). The operational interface offers a wide range of exercises that the patient must perform under the control of the operators.The research work, which took place in the period indicated above, had among the main objectives the creation of special software which uses the potential of the device technology in response to the clinical-rehabilitation needs.Robot Assisted Therapy (RAT) with both passive and active devices, is an innovative form of rehabilitation that enables highly repetitive, intensive, adaptive, and quantifiable physical training. It has been increasingly used to restore loss of motor function, mainly in stroke survivors suffering from an upper limb paresis. Multiple studies collated in a growing number of review articles showed the positive effects on motor impairment, less clearly on functional limitations . RecoverIn this context, Robotic Assisted Therapy (RAT) is an innovative approach to upper limb rehabilitation. The RAT involves intensive and repetitive exercise, with interactive and individualized practice, which could also imply the execution of oriented tasks or dual-task paradigms, as was shown in our preliminary work .The RAT encompasses technological solutions, such as computerized control systems, and mechanical devices, with both active and passive functioning, that could promote motor learning .Promoting processes of motor learning and cross-education from unaffected to affected upper limb  is the mThe devices that implement RAT, such as wearable sensors and free-contact systems for gesture tracking, are highly reliable to capture and measure kinematic and dynamic parameters of the upper limb, such as movement quality, speed, direction, strength and range of joints motion ,8,9.Results of previous studies ,10 showeA recent metanalysis  shows thNotably, it has been reported that RAT, with individualized and customized protocol of exercises, could improve in post-stroke patients also neurophysiological aspects of the upper limb, mainly enhancing shoulder and elbow range of motion ,14,15,16The most important paradigm of RAT is not only to maximize the number of task repetitions but also to increase patient attention and effort as well .Indeed, it is known that repetitive and monotonous exercises provide worse retention of motor skills, compared with alternate training .Furthermore, it has been speculated that RAT could decrease recovery of motor functions if it encourages assuaging since the patients could decrease effort and attention due to the use of adaptive algorithms, i.e., during training with active robotic devices .Indeed, clinical evidence shows that, since learning is error-based, a faster improvement of motor skills may be achieved when the rate of error is increased .Cognitive and motor rehabilitation after stroke and other neurological diseases is firmly based on intensive and repetitive training and task-specific learning for promoting neural reorganization and subsequent recovery . ConventMoreover, studies with stroke survivors have shown a differential pattern of motor outcomes, depending on the cognitive domain impaired .Rehabilitation methods, based on robotic systems, implemented with exergames, provided positive results in patients with post-stroke disabilities of upper limb .Exergames, used as task-oriented motor functions, have been used in RAT to improve certain cognitive functions, promoting the activity of specific domains, especially those related to executive functions and attention, that regulate decision making, movement programming, and time reaction .As described above, state of the art indicates that the combination of RAT with exergames might have a beneficial outcome. In consideration of our previous work  in whichOur study was an extended case report with an enrollment of a small sample of patients.The enrolled patients signed an informed consent drafted according to the guidelines of the local Ethics Committee, furthermore, the procedures were carried out in accordance with the provisions of the Declaration of Helsinki on clinical trials on humans.SI-Lab staff, the practitioner in neuro-rehabilitation, and the physiotherapists operating in the ICARE Healthcare Facility of the former Tabarracci hospital (Viareggio) collaborated on the THScase study experimentation.The case study mainly had the objective of developing a motor and cognitive rehabilitation protocol in subjects with post-stroke outcomes affecting the right upper limb, including strength deficits, tremor, incoordination, and motor apraxia.The protocol consisted of a 4-week training period for subjects with a motor deficit of the right upper limb; enrolled patients performed guided motor exercises with weight relief using the TrackHold device. Patients performed the proposed tasks following a criterion of progressive difficulty (achievement of a greater number of key points). Furthermore, based on the progressive recovery of muscle strength, weight relief was slowly reduced, clinically assessing each patient’s level of muscle strength session by session.Four patients, two men, and two women, completed the minimal proposed tasks proposed in this study. The patients were enrolled after neurological assessment and neuropsychological screening. Patients description, clinical data and anthropometric parameters have been reported in the A phase for the evaluation of subjects eligible for the rehabilitation protocol;An execution phase in which THS sessions were carried out;A follow-up phasefor the assessment of the clinical outcome.The extended case report was carried out in a period of three months, during which we ran the experimental procedures. There were 3 phases of the experimentation:During this phase of the trial, the clinicians evaluated the patients to establish patients’ eligibility in the protocol and estimate the level of disability.The presence of moderate-to-severe cognitive impairment, assessed by the Mini screening test Mental State Evaluation (MMSE), was considered a fundamental criterion for excluding patients from the present experimental protocol.To be enrolled, subjects had a score equal or bigger to uggested .A clinical and pharmacological anamnesis was then carried out, along with the neurological physical examination and the survey of the main anthropometric parameters , lengths, and circumferences of the limb upper right). The clinicians selected patients in acute and subacute post-stroke, veterans of the event, vascular affecting the left hemisphere, with a motor disability of the right upper limb.0) using standardized clinical scales (listed below) to establish the level of muscle strength, motor ability, and spasticity of the upper limb.The screening phase required a baseline assessment (TAfter four weeks of training at follow-up (T 1), the clinicians performed the same type of clinical evaluation.MRC scale: assesses the level of motor disability. MRC scale is a grading system and provides the following scores: 0, for paralysis; 1, only a trace or flicker of muscle contraction is seen or felt; 2, muscle movement is possible with gravity eliminated; 3, muscle movement is possible against gravity; 4, muscle strength is reduced but the movement against is possible . MaximumMotor index: evaluates movement against gravity and resistance. Motor Index (or Motricity Index) can be used to assess the motor impairment in a patient who has had a stroke. Tests for each arm start with assessing inch grip using a 2.5 cm cube between the thumb and forefinger. Scores are as follows: 19 points are given if the subject is able to grip cube but not hold it against gravity, 22 points are given if able to hold cube against gravity but not against a weak pull, and 26 points are given if able to hold the cube against a weak pull but strength is weaker than normal. In the second part of the test, an elbow flexion from 90° is required, so that the arm touches the shoulder. Scores are as follows: 14 points are given if movement is seen with the elbow out and the arm horizontal. In the third part of the test, shoulder abduction moving the flexed elbow is required from off the chest. In this case, 19 points are given when the shoulder is abducted to more than 90° beyond the horizontal against gravity but not against resistance . MaximumAshworth scale: assesses the severity of the spasticity. Ashworth scale is the most accepted clinical tool to measure the increase of the muscle tone and spasticity; it is a six-point scale (for upper and lower limbs). Scores range is from 0 to 4, where lower scores represent normal muscle tone . MaximumThe evaluation scales used are the following:The exercises carried out with the Track-Task aim to make movements valuable and necessary for the execution of daily life activities.The movements functional to each exercise, carried out by manipulating the device, are composed of sub-movements, which consist in making a cursor on the screen placed in front of the patient reach key-points, characterized by a particular angle and a position. The movement of the manipulator in the frontal plane (XY) finds an immediate correspondence with the position on the screen (easy to identify even by the patient who is performing the exercises). As for the movement in-depth , it was chosen to represent it with a variation in the size of the cursor: pushing the manipulator forward (Z coordinate increase) decreases its size, the object moves away. This metaphor is very intuitive and allows, by diversifying the size of the target key points and consequently their position on the Z-axis, to guide the subject to perform the movements in three dimensions. These features have been explained with great attention to patients in order to obtain the best for the best possible performance.The Track-Task design allows working in a double-screen configuration. The first large screen faces the patient who has to do the rehabilitation or training session. The second screen faces the operator who enters the patient’s identification data and selects the training according to the planned rehabilitation therapy by selecting suitable buttons on the control panel. The configuration with two screens allows the system to not confuse or distract the patient with the exercise preparation operations.Total time: total time elapsed, in seconds.Distance overhead: the percentage that indicates the actual distance traveled by the cursor in space, compared to the minimum possible distance between the key points to be reached.LF index, HF index: these data are produced by a frequency analysis on the trajectories. The analysis is very important as it allows to detect some indications of tremors associated with neuromotor and/or neurological disorders. Without going into detail here, we can simply say that those at a relatively low frequency (5–8 Hz) are symptoms of neuromotor problems deriving from problematic damage inherent in the post-stroke phase or parkinsonian-type pathologies. Those with a higher frequency (8–15 Hz) can be classified as systemic or physiological tremors, the extent of which may be a function of the subject’s age. The Track-Task application then performs a frequency analysis of the movement trajectories in order to distinguish these two bands.During the rehabilitation sessions the following data relating to the movements made by the subject were acquired:The Track-Task has a protocol organized in exercises having increasing duration and difficulty. The proposed exercises guide the patient to perform a functional movement due to its characteristics, as it recalls precise gestures corresponding to daily actions.Each protocol contains a certain number of tasks; each task is composed of sub-movements divided into segments. The subject has to act its limb forward to some oriented key points placed in three-dimensional space, moving the manipulator and following the cursor’s movements on the monitor.Pushing a door: the act of pushing a door involves reaching two key points.Turning off a switch: the gesture of turning off a switch involves reaching two key points.Opening the door to the left by grasping the handle: the gesture of opening the door by turning the handle involves reaching three key points.Opening the sliding door of a wardrobe: the gesture of opening a door or sliding door involves reaching three key points.Bring a bottle of water to the mouth: the gesture of bringing the bottle towards the mouth to drink implies the achievement of three key points.Mixing the contents of a container clockwise: the gesture of mixing the contents of a container involves reaching 4 key points.Cleaning a surface or glass: the gesture of cleaning a surface with large movements that go from left to right and vice versa implies the achievement of 7 key points.The Tasks included in the protocols are the following:The system has three different protocols having increasing duration and difficulty. The subject executes various protocols in rehabilitation training, according to the indication of the clinician. The adopted protocols are based on the assessment of the level of motor skills, strength, and muscle tone that the clinician evaluates in correspondence with pre-established follow-up. More in detail, the three configurable protocols include: (i) a short session composed of three tasks, (ii) a medium one containing four tasks, and (iii) a long one having seven tasks. Before executing each protocol, an additional functionality of the system is used to acquire, under the guidance of the operator, the motor parameters of the patients, namely the “Range Of Movement” of the upper limb joints (ROM).The four patients enrolled underwent an overall training period of 4 weeks. A training session preceded the actual training to learn how to use the device correctly.The robotic arm needs weight for proper use. Clinicians measured the patient’s muscle strength to calculate the weight relief to use with the robotic arm at the start of each training session. Clinicians used a dynamometer and the arm curl test to take the measurements. At the end of the same session, the patient indicated the level of perceived fatigue using a Visual-Analogue Scale (VAS). Each rehabilitation session lasted about 30 min.the patient’s subjective motor learning ability;muscle strength recovery of the patient.Each patient performed specific tasks. The clinician chose the task difficulty for each session for each patient with the following criteria:Dynamic muscle strength: evaluation of the muscle strength of the upper limb using a dynamometric platform (test time = 10 s);Arm curl: This test measures the strength and endurance of the lower limb. The clinician loaded the Track-Hold loaded with 1 kg for the female and 2 kg for the male. During the test, the subject has to perform as many arm curls as possible. The test length is 30 s; the patient executes the test on the side of the dominant arm.The clinician assessed the last point (muscle strength recovery) for each patient at the start of each session using dynamic strength tests:At the end of the session, the clinician assessed the level of psycho-physical fatigue using the index of fatigue by presenting an analogical scale to the subject. The subject uses this analogic scale by pointing to the area corresponding to the perceived fatigue. The score is from 0 to 10.To obtain weight relief, clinicians used three weights installing them on the Track-Hold device: (1) 976 g, (2) 1429 g, (3) 3894 g.The doctor and the physiotherapist assisted the patient during the rehabilitation setting .Patient 1 participated in 12 rehabilitation sessions, performing a total of 47 tasks. The training led to a visible improvement in dynamic strength from one session to another and consequent progressive reduction in weight relief. Patient 2 participated in a total of 4 sessions, carrying out 14 tasks. The performance improvement was less noticeable in terms of weight relief, dynamic strength, and perceived fatigue. Patient 3 participated in 11 rehabilitation sessions, carrying out a total of 35 tasks. The results show a marked reduction in relief from a moderate increase in dynamic strength and a reduction in the VAS index of fatigue score from 6 to 3. Patient 4 performed seven sessions (an intermediate number) during 28 tasks. The data gained during the tasks evidence an improvement in motor performance and effort tolerance. The clinical results of the THS experimental protocol show that this type of training, based on the performance of motor tasks on a guided cognitive paradigm and with progressive relief of severity, resulted in a specific functional recovery of the right upper limb in the short term.This recovery positively affects the acquisition of muscle strength, motor skills, and less perception of physical and mental fatigue in the patient.scoring method for pliers grip:–0 = no movement–11 = start of pressure, some movements of thumb and forefinger–19 = grip is possible but not against gravity–22 = grip is possible against gravity but not against resistance–26 = grip possible against resistance but weaker than left side–33 = normal socketscoring method for elbow bend and shoulder abduction:–0 = no movement–9 = muscle contraction is possible without appreciable movement–14 = movement is possible but not for all ranges of joint motion or against gravity–19 = movement is possible throughout the range of joint motion against gravity but not against resistance–25 = movement is possible against resistance but is weaker than the left side–33 = normal movementTHS training seems to have improved the overall motor skill of the right upper limb, particularly elbow and shoulder mobility, as reported in Our preliminary results show that THS training could be effective to improve upper limb motricity in the acute and subacute post-stroke stage. Preliminary clinical results of the Track-Hold System demonstrated good patient compliance with this kind of robotic rehabilitation. The interfaces for users and operators made it possible to establish an effective recovery path for patients who were able to gradually resume (even if not completely) the functions of the upper limb. In the form of exergames, the guiding exercises of varying difficulty made it possible to emulate the gestures of daily activities. Following the exergames, the patients learned the lost movement capacity during the workouts; then, they have reproduced the movements in their daily acts, thus approaching the quality of life before the disease.Further experimentation is necessary for a large sample of patients to confirm the effectiveness of this cognitive and motor rehabilitation approach.This case study has shown that exercises carried out using a passive robotic device equipped with weight relief allow partial recovery of the functional capacity of the upper limb even three months after a stroke episode. The next step of this study involves a comparison between this rehabilitation approach and a clinical method that is currently used for the functional recovery of stroke survivors, namely Action Observation Therapy (AOT) . The AOT


Apis mellifera) has long been a popular model in the search for neural correlates of complex behaviors and exhibits extraordinary navigational capabilities. However, the neural basis for bee navigation has not yet been explored under natural conditions. Here, we propose a novel methodology to record from the brain of a copter-mounted honey bee. This way, the animal experiences natural multimodal sensory inputs in a natural environment that is familiar to her. We have developed a miniaturized electrophysiology recording system which is able to record spikes in the presence of time-varying electric noise from the copter's motors and rotors, and devised an experimental procedure to record from mushroom body extrinsic neurons (MBENs). We analyze the resulting electrophysiological data combined with a reconstruction of the animal's visual perception and find that the neural activity of MBENs is linked to sharp turns, possibly related to the relative motion of visual features. This method is a significant technological step toward recording brain activity of navigating honey bees under natural conditions. By providing all system specifications in an online repository, we hope to close a methodological gap and stimulate further research informing future computational models of insect navigation.Navigating animals combine multiple perceptual faculties, learn during exploration, retrieve multi-facetted memory contents, and exhibit goal-directedness as an expression of their current needs and motivations. Navigation in insects has been linked to a variety of underlying strategies such as path integration, view familiarity, visual beaconing, and goal-directed orientation with respect to previously learned ground structures. Most works, however, study navigation either from a field perspective, analyzing purely behavioral observations, or combine computational models with neurophysiological evidence obtained from lab experiments. The honey bee ( Honey bees are remarkable navigators. Foragers learn to orient in complex environments and perform accurate goal-directed flights in areas several square kilometers in size Collett, . A rangeIn insects, we can identify two main approaches: animals may either move freely in small confined arenas, such that their brain is accessible with wire electrodes or imaging techniques . The field site was rich in visual landmarks, both on the ground  and the horizon (see the map in Apis mellifera) workers from three groups  were attached to a quadcopter  via an extension arm , pulling the bee from the foam ball. For the control group C, the ball was pulled manually, without any motor activity of the copter. Due to the tarsis reflex, the bees started beating their wings instantaneously. Bees in group H were lifted upwards to ~2 m altitude , with negligible rotatory or horizontal movement. Bees in group F were flown forwards, continuously gaining altitude (up to 2 m) and distance to the lift-off point. The copter was controlled manually and brought back after no wing beating was observed anymore or a maximum of one minute of flight time had passed. Flight forward velocity was 10 m/s  or put in an opaque box on the top face of the copter  such that it could not perceive the flight path visually. The animal was tethered with a small clamp in the box, similar to the mechanism depicted in We investigated if bees extract information relevant for homing when being transported on the copter. Bees were trained to a sugar dish 400 m east of their hive , and a microcontroller board with an SD card for data storage. The amplifier see  was baseFigure 8). The amplified and filtered  signals were digitized and read out by a microcontroller board  was equipped with the miniaturized recording system and an extension arm to attach the animal and recording equipment. A camera observed the tip of the arm from below and provided a view of the environment  at Julius-Kühn-Institute Berlin, Germany . All inflight neurophysiological recordings were conducted at this site.A hive was set up at Free University Berlin , and bees were trained to collect sucrose from a feeder on a field  and reconstructed the depth map in high-resolution (12 cm/pixel) for a close neighborhood around the field and in low resolution (~4 m/pixel) for a larger surrounding area. The three maps were combined in Blender , resulting in a surface depth map. Due to regulations, we were not allowed to fly over the surrounding areas. We extracted freely available image data (Google Earth) in virtual flyovers for the surrounding field  was positioned in the middle of the field  was repeated twice, and the copter then flew back to the start location to repeat this flight pattern until the battery of the copter was too low to continue . The copter was then landed manually at the start site for battery replacement, with consistent rest times between trials. Each time, we downloaded the data to a laptop and then performed the preflight check again. The process was repeated until either the animal died or no spikes could be registered anymore.U-test was used to test the null hypothesis that there are no significant differences in observed flight behavior duration between the three groups .Statistical hypothesis testing was performed to analyze the behavioral data. For the wing duration experiment (section 2.1.1), a one-sided Mann-Whitney U-test was used to test the null hypothesis that there are no significant differences in the duration of homing flights after the release from the copter between the two groups .For the homing experiment (section 2.1.2), a one-sided Mann-Whitney The recorded data consisted of two channels of neuronal signals timestamped by GPS signals. The GPS signal was also used to timestamp the telemetry of the copters flight path, which would be used to synchronize the data with sub-millisecond accuracy. The telemetry data was saved with 100 samples per second, including the speed, height, GPS coordinates, acceleration, and orientation of the copter. The data were then merged using the GPS timestamps.xt = [xt − median(X)]/mad(X), where xt is the amplitude of the signal at time t and mad is the median absolute deviation of the signal. The differential of the two recordings was then computed to improve the signal-to-noise ratio in the data. Furthermore, a local robust normalization was applied with a sliding window size of one second to reduce the effect of the time-varying signal-to-noise ratio caused by the quadcopter's motors and rotors on the quality of the extracted signal.The electrophysiological recordings were analyzed using the Python scientific software stack ·k (k = 1.4826). The threshold for spike detection was set to Thr = 4·n , for each detected neuron, the median spike shape was determined, and the sliding Pearson correlation of this shape with the normalized input signal was computed. The spike detection steps were then repeated on the correlation coefficient, i.e., a threshold was computed, and local minima beyond this threshold were detected. This pattern matching spike detection increased the number of detected non-outlier spikes from 13,861 to 17,106 in a recording of ~14 min.Figure 8, a rolling mean with a window size of 3 s was used to smoothen the trajectory.Spike trains were binned in intervals of 100 ms, and spike rates were calculated as the sum of detected peaks during each interval. For the visualization of the rates in The trefoil path was repeated multiple times per flight, and it seems possible that the neuronal signals reflect these repetitions, irrespective of which sensory properties the units we record from encode. To verify that hypothesis, we calculated the Pearson correlation coefficient of the spike rate time series that corresponds to a single repetition of the trefoil trajectory in a sliding window over the whole flight's recording. If the bee's brain signals reflect the repetitive flight patterns, we expect to see peaks denoting the beginning of every trefoil pattern.In the data analysis, the copter telemetry data (GPS and compass readings) were used to reconstruct the flight path in the 3-D map of the environment. We previously published a software package to reconstruct bees' visual perception . Groups C and H did not differ significantly . See The forward flight group T returned home after a significantly shorter amount of time . See Bees in the treatment group Before using the new recording system on the copter, we tested its functionality with artificial signals and signals from a honey bee brain under laboratory conditions. Activity from the same source was recorded with both the copter's amplifier system and a commercial system . We found no significant differences in the data obtained by these two systems when comparing spikes from bee brains as well as sweeping through frequencies generated artificially.In-flight, we successfully recorded uninterrupted single-unit activity from MBENs for multiple repetitions of the flight trajectory. The electrodes picked up significant amounts of EM noise produced by motor controllers, motors, and propellers. We observed that the noise levels differed between channels, probably due to differences in impedance. However, the differential recording allowed removing much of it when carefully adjusting the respective digital gain factor for one of the channels. For each experiment, the factor was set manually after the fact. Once this tuning was complete, spike shapes emerged. The amplitude of the monopolar input channels was around 100 times larger than the resulting spikes from the differentiated channel. These recordings were then sorted. We calculated interspike intervals and confirmed that the refractory period of 4 ms was rarely undercut, indicating the successful sorting of single spike sources. For more details, see Experiments with neuronal recordings on the copter took place during the fall of 2018 and summer of 2019. Starting with around 200 animals, we successfully implanted electrodes into ~50 animals, all producing neural activity upon light stimulation. However, only 10 animals continued to provide spike data after transport to the field, and only three of these recordings passed the post-experiment quality control check. Half of the recordings showed too much noise over the entire flight list with doubtful spike sorting results, and two recordings were excluded due to variable signal-to-noise ratios throughout the experiment.The neural recordings are consistent for repeated environmental stimuli. We find strong autocorrelation of the spike rates for single trefoil flight patterns in individual bees see . This inWe found that episodes with high spike rates coincide with turning maneuvers see , though Visual inspection of the spike rates revealed no apparent correlation to the bee's spatial relation toward the feeder or the hive. We used a model of the bee compound eye to map the spike rate back to the map of the area for each position along the flight path see . While bThe mapping of spike rate activity to the field of view of the individuals in general revealed regions associated with high spike rates that were varying over multiple repetitions of the same trajectory, even within one individual. Interestingly, even when excluding the turns in this mapping, the regions near the turning maneuvers tended to show the strongest activity see . We founWe propose a novel methodology in the search for the neural correlates of navigation in bees. In contrast to reproducing realistic conditions in virtual environments, we propose moving the lab to the field. While this approach comes with its own challenges, we show that recording neural activity from MBENs in honey bees is feasible on a quadcopter in flight.We miniaturized the recording hardware, and substantially reduced motor noise picked up by the electrodes with various strategies, from grounding and shielding to differential recordings and respective hardware design decisions.Those bees that survived the implant and were transported to the field survived multiple repetitions of the flight trajectory. The bees that entered analysis showed no baseline shifts indicating electrode movement nor subsequent loss of units. The electrophysiological data are of high quality, and the spikes and their properties are close to those recorded under lab conditions  and survival time of the animal. The success rate under laboratory conditions varies between 30% in bees mounted to a tube , we need to further improve the signal-to-noise ratio. On the one hand, this could be accomplished with better shielding and adaptive impedance matching for the channel subtraction. On the other, the assessment procedure that determined whether spike magnitudes are sufficient would benefit from a realistic simulation of anticipated copter noise in the lab.The main problem for interpreting the results is that spike rates can only be meaningfully compared within the same animal due to potential differences in the neuronal connectivity and electrode implant locations between individuals. Unfortunately, only a finite amount of data can be recorded from one individual, making the interpretation of the results difficult. It may be possible to assess the exact recording location via imaging techniques. To improve the repeatability of the implant and reduce variability in the resulting signals, the electrode production, insertion, and sealing process could be automated in future studies.An important question to be addressed more accurately in future work will be to relate the localization of the recording electrodes to subsets of MBEN or brain structures such as the central complex. Extracellular recording techniques come with the unavoidable limitation of spatial location. Therefore, the preparatory steps during the selection of the recorded neurons become extremely important. Technical improvements that allow extending the recording time substantially will help characterize the selected neurons physiologically by probing batteries of more complex stimulus conditions before the preparation is fixed to the copter. So far, we selected for stable responses to simple movement stimuli before the bee was fixed to the copter. Thus, it is not surprising that the MBENs analyzed here correlate with turning motions.Our data analysis includes the reproduction of the bee's visual perception using a three-dimensional map created before the experiments. In contrast to recording synchronized video approximating the field of view of honey bees directly on the copter, our approach drastically reduces the amount of data recorded in each experimental run. However, our model of the honey bee vision using a photogrammetric model of the environment can not simulate the dynamic nature of vegetation, celestial cues, and weather conditions.Bees have shown flight behavior in a virtual reality setup (Luu et al., Our system complements the toolkit for studying the neural correlates of natural navigation in bees. While future developments of lab-based setups may need to focus on a realistic, multimodal reproduction of the environment, drone-based setups are confronted with more complex control tasks. Since, to our knowledge, there is not yet a virtual reality system capable of recording brain activity in flying bees, our system can serve as an alternative starting point. To encourage the continuation of this effort, we are sharing this proof of concept, as presented here, in its entirety with the community.https://github.com/BioroboticsLab/neuronal_correlates_honeybee_navigation.The datasets and codes presented in this study can be found in this online repository: BP, RM, and TL conceived of the idea. BP and JPe developed and tested the miniaturized recording system. JPe and TL conceived hardware additions to the copter. BP, JPe, and TM conducted the wing beat duration experiment. TL, BW, and RM conducted the homing experiment. JPe, BP, OK, HD, TL, and IF conducted the neurophysiological experiments. JPe and TM created the drone maps. BW, TS, JPe, BP, and JPo analyzed the data. BP, BW, TL, and RM wrote the manuscript. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.