This study aims to explore the possibility of estimating a multitude of kinematic and dynamic quantities using subject-specific musculoskeletal models in real-time. The framework was designed to operate with marker-based and inertial measurement units enabling extensions far beyond dedicated motion capture laboratories. We present the technical details for calculating the kinematics, generalized forces, muscle forces, joint reaction loads, and predicting ground reaction wrenches during walking. Emphasis was given to reduce computational latency while maintaining accuracy as compared to the offline counterpart. Notably, we highlight the influence of adequate filtering and differentiation under noisy conditions and its importance for consequent dynamic calculations. Real-time estimates of the joint moments, muscle forces, and reaction loads closely resemble OpenSim’s offline analyses. Model-based estimation of ground reaction wrenches demonstrates that even a small error can negatively affect other estimated quantities. An application of the developed system is demonstrated in the context of rehabilitation and gait retraining. We expect that such a system will find numerous applications in laboratory settings and outdoor conditions with the advent of predicting or sensing environment interactions. Therefore, we hope that this open-source framework will be a significant milestone for solving this grand challenge. Movement is essential for human well-being, and diseases that affect it can significantly deteriorate life quality. Advances in mobile sensors ), video technologies (markerless Motion Capture (MOCAP)), and traditional marker-based systems are used for real-world assessment of movement ,2. The agait2392 model [OpenSim [OpenSim tool-chain that is well-adopted by the biomechanics community.Different research groups have approached the problem of estimating the kinematics and dynamics of movement using musculoskeletal models. Notably, in  the auth92 model  within t[OpenSim . Statist[OpenSim . Howeverrtosim), that relies on OpenSim, capable of performing Inverse Kinematics (IK) and Inverse Dynamics (ID) calculations was presented in [An open-source solution  to carry out further dynamic analyses. In particular, we focused on estimating the Ground Reaction Forces and Moments (GRF&M) during walking using only kinematics. There have been numerous studies ,22,23,24In the following sections, we will closely examine the challenges mentioned above. In w are the weights (default is 1) associated with the different error terms. The superscript T over a vector or matrix denotes the transpose operation. Please note that this implementation can handle models that contain holonomic constraints due to the first term in the optimization that penalizes these errors.The current implementation of IK can handle marker position and IMU orientation tracking. The implementation of IK solves an optimization problem that minimizes the holonomic (position) constraints, the weight marker, and orientation errorOpenSim, a transformation Concerning IMU integration, a calibration procedure is defined to match the sensor orientations with the corresponding body segment orientations. The subject is asked to stand in a pose that matches the default pose of the virtual model for a few seconds. The sensor orientations NGIMU from x-io Technologies Limited) to record the upper- or lower-limb movement and test the real-time system. Unfortunately, this introduced many technical challenges related to obtaining accurate orientation from multiple sensors that are out of the current study’s scope. We experienced problems with the accumulation of errors due to bias and drifting in the orientations, which are sensitive to magnetic interference. Synchronization of sensor data  was accommodated through a data structure that stores the arriving frames, which are then resampled (constant sampling frequency) using each sensor’s synchronized timestamp. Timestamp synchronization was performed at the beginning of each session. Despite the technical challenges, we were able to test whether one could successfully use IMU information as an input to our system and verify the Inverse Kinematics (IK) method.In our experiments, we made use of a custom-build IMU MOCAP system . Calculating derivatives is a challenging task in the presence of noise. If the noise is permitted in the ID calculations, we will obtain unrealistic estimates of the joint moments because the acceleration term is multiplied with the inertia mass matrix. To cope with this issue, we usually apply a low pass filter  and theM is termed as the memory of the filter. A low-pass (M (same as memory size) is applied to reject the high-frequency noise. The filter’s kernel and window functions are sinc and Hamming, respectively. The filtered signal is used to construct a generalized cross-validation spline of order D is the lag of the real-time filter) instead of t to improve the filtering result. Therefore, the proposed filter is characterized by four hyper-parameters . Depending on the application and type of task, one can fine-tune these parameters to achieve the right trade-off between delay and accuracy.The proposed filter  assumes To perform dynamic analysis, one must estimate the external GRF&M and include them in the calculations. These are typically measured using dedicated equipment . However, this can significantly limit the scope and extent of any dynamic analysis only when such measurements are available. There have been numerous studies ,22,23,24N the total number of body segments, In this work, we implement a GRF&M prediction method, where the total forces and moments are computed using the Newton–Euler equations of motion:Assuming the GRF&M are the only external forces and moments are applied to the subject during gait, the During the single support phase, the GRF&M applied to the limb in contact with the ground is equal to the total GRF&M computed. Whereas, during the double stance phase, the indeterminacy problem is resolved following the smooth transition assumption . In thisstance or swing) based on a predefined threshold value k consecutive states. The leading leg is assigned to the lower limb corresponding to the last heel-strike event until the next heel-strike event of the contralateral leg. After computing the gait-event-related parameters, the GRF&M applied on each lower limb are calculated following the rules defined in An import prerequisite for using the transition functions is determining time events during the gait cycle. In particular, the heel-strike  diagram . State cThe CoP is approximated as described in . We scalThe computed vector Estimation of muscle forces imposes several challenges because there is not a unique solution due to the muscle redundancy problem  (more unp is the power exponent (set to 2 or 3), In our implementation, we formulate the optimization problem as follows:stencies . The aboSolving Equation  in real-In the previous subsections, we presented a pipeline for estimating the kinematics, dynamics, and internal forces (muscle and joint) of the musculoskeletal system in real-time. The pipeline serves as the backbone (back-end) for implementing rehabilitation solutions that could provide real-time feedback to the user and clinicians (instructor). In this subsection, we present a gait retraining system (front-end) for reducing the knee joint’s loading and reduce the pain. The gait retraining system aims to engage the users in a series of exercises that can teach them how to adapt their gait optimally.Unity3D game engine. Unity3D was selected because it can permit the implementation of solutions that can target typical projection screens and utilize augmented or virtual reality output devices. Our system could support interactive scenarios that could engage the user and make the training process more fun and effective. Exchange of data with the simulation back-end was established through shared memory to improve efficiency, reduce latency, and decouple implementation specifics .The gait retraining system was implemented using the In The extension of this system is not only limited to the particular case presented here. One can imagine that similar rehabilitation scenarios could be implemented in line with the applications’ needs. The significance of the back-end is that it permits calculating any kinematic and dynamic quantity  from the musculoskeletal model in real-time. Depending on the application, different combinations of these quantities will be needed. The front-end can then transform the raw information and visualize it appropriately. The user’s interaction may not be only limited to primitive projection screens but can also utilize haptic, augmented, and virtual reality devices. Therefore, rehabilitation scenarios could be designed so that they are more engaging and effective. All this would not be possible without an accurate and reactive back-end.In this section, we present the comparison of the real-time filtering method’s performance and justify how to determine the different hyper-parameters. Next, the comparison of the offline and real-time generalized forces computed through ID is presented by highlighting the importance of determining the kinematics’ first and second derivatives. We will further present the estimated GRF&M and quantify their impact on the ID results. A comparison between the offline and real-time calculations of muscle forces is examined. Computation delays for the various modules are presented in the text. Finally, we outline a use case scenario of the proposed system in the context of rehabilitation and gait retraining to reduce the knee’s joint reaction loads.OpenSim’s API [gait2392 [Our framework utilizes the im’s API ,10,34 angait2392  with somgait2392 . The genM, and delay D, characterize the filter. The type of movement determines the cutoff frequency. For slow movements like gait, we chose M and D is discussed below.In this subsection, we demonstrate the real-time filter’s performance and outline the procedure for determining the optimal values of its parameters. The filter operates on the generalized coordinates obtained from IK since the optimization may not result in a smooth solution. The mean computational delay of the IK module is less than 0.1 ms. Four parameters, namely the cutoff frequency  studies ,36. We aM and D), we examined the total Root Mean Squared Error (RMSE) , there exists an optimal value for M. We determined that M reduces the computational latency of the filter. Similarly, The purpose of the filter is to reject the noise and calculate the first and second derivatives of the kinematics necessary for consecutive dynamic analysis. To determine the optimal values of the two hyper-parameters (r (RMSE)  of the fOpenSim’s offline kinematics analysis (ground truth) against the proposed filter and the “spatial” filter as implemented in rtosim [OpenSim’s offline time-series. While the spatial filter can approximate the original signal well, it underestimates its first and second derivatives for fast-moving coordinates  to reduce latency while maintaining accuracy.We will briefly discuss the performance of the real-time ID module with respect to OpenSim and real-time ID utilizing the two real-time filters presented previously  is illustrated. A comparison of all coordinates is also presented in M samples to begin the processing. The real-time curves are smoother and present fewer oscillations due to the non-causal implementation related to the D parameter. We also observed that the proposed filter outperforms the spatial filter in terms of RMSE. The spatial filter can underestimate the moments due to the kinematic mismatch at the coordinate velocity and acceleration, as presented in the previous subsection. Therefore, this can have a significant impact on the joint moments.Moreover, we present the effect of online filtering on the calculated generalized forces from ID and why proper filtering is important. In x) and vertical (y) directions closely resemble the measured ones. Lateral (z) forces follow the overall shape. The x component of the CoP was accurately predicted, but the lateral z direction is hard to determine. Less confluence is observed in the moments. The mean computation delay of this module is 2 ms. The double support phase during walking leads to infinite possible solutions (close kinematic chain). Modeling assumptions, such as the distribution of mass and joint parameterization, can affect results due to the method’s nature.Predicting ground reaction wrenches in real-time can enable applications that calculate dynamic quantities  during movement. In Next, we quantified the influence of the predicted GRF&M on joint moments calculated from ID. In In this section, the performance of the real-time muscle optimization module is presented. The determination of muscle forces can find many applications in sports biomechanics, ergonomics, and rehabilitation. For example, estimation of muscle forces is necessary to determine the joint reaction loads affected by muscles spanning the joints. Resolving the muscle redundancy is the bottleneck of the whole pipeline requiring an average of 10 ms to execute with a musculoskeletal model with 92 muscles. Things that have improved the performance of the muscle optimization were (i) the formulation of the problem to reduce the constraints and decision variables, (ii) the analytical representation of the moment arm matrix , (iii) optimization initialized using the previous solution, and (iv) the efficiency and parallelism of the interior point optimization method.OpenSim’s Static Optimization (SO) and proposed real-time muscle optimization method. Results show forces for major muscles  of the lower-limb. A comparison of all 92 muscles is presented in the Dr. Good has experience with the gait retraining system, and their clinic just received a patient that has been diagnosed with mild knee osteoarthritis in her right leg. He already knows that the patient needs to adopt a gait pattern that alleviates the knee joint’s medial loading to reduce the pain. The primary indicator to assess the loading of the joint is the medial reaction force at the knee. One strategy is to adjust the foot progression angle, defined as the angle between the line of walking progression and the foot’s longitudinal axis. Therefore, he defines the vertical reaction force as an objective to be minimized. The decision variable is the foot progression angle. However, he also decides to experiment with the step length and step width, providing simple and intuitive visual feedback to the patient. The system can adjust the decision variables’ target value  using onFirst, he tried to familiarize the patient with the motion analysis lab by explaining the different visualization elements and by presenting the current session’s goal. Since this is her first session, the patient is ordered to walk normally to gather information on her walking habits and calibrate the model. The clinician checks if the system works properly by comparing the reaction loads’ real-time and offline calculations at the knee . Based oIn this subsection, we would like to highlight the usefulness of the proposed system in the context of gait retraining and rehabilitation. We will outline a possible use case scenario that might appeal to researchers and clinicians in the field of rehabilitation.In this work, we presented the implementation and validation of a system capable of calculating various kinematic and dynamic quantities from a musculoskeletal model in real-time. We rigorously proved that the real-time calculations closely resemble their offline counterpart. This permitted us to examine the use of such a system for gait retraining to provide real-time biofeedback to the user. The added value of a gait retraining system needs to be further examined in the long-run, especially for reducing joint reaction loads in patients with osteoarthritis ,38. NeveXSENS instead of our custom-build solution and compare the performance of the system against marker-based solutions.Being able to accurately determine the kinematics using either a marker- or IMU-based analysis systems is an important step that, if not done correctly, can affect calculations at the next stages. On the one hand, marker-based solutions are well-established and can accurately determine the movement of the reflective markers. However, in real-time scenarios, there might be instances of missing markers due to occlusion , which mt using prior or subsequent information. Using subsequent information dramatically improves the filtering process without distorting the original signal. This is essentially what our proposed filter does. It introduces a small lag  in the data to predict the signal more accurately. Moreover, we use splines instead of finite differences to calculate the derivatives. If the motion capture operates at higher frequencies , then a small delay will be negligible to the user. The drawback is that this filter can slightly increase the computation time and thus reduce the pipeline’s throughput. However, the benefits are that the kinematics are accurately determined, and therefore real-time filtering does not distort calculations at the following stages. It is also worth noting that one could determine the optimal filter parameters for a given type of task, such that they minimize the inconsistencies between the offline and online analysis, as shown in this study.In order to perform dynamics calculations, we must determine the first and second derivatives of the kinematics. This proved to be a challenging task for real-time applications because of the inherent noise in the measurements that can easily pollute the meaningful signal. This is not an issue in an offline setting because we can calculate the value of a signal at Estimating the GRF&M in real-time under any movement context is an essential step towards moving in out-of-the-lab settings. This holds for any interaction of the user with the environment. In this study, we used a model-based approach to estimate the GRF&M based on first principles. While these wrenches approximate the measurements well, we observed that even a small mismatch could easily manifest in the calculated joint moments. Interestingly, some components of the ground reaction wrench influence these moments differently. A model-based approach might be challenging to implement and probably hard to generalize to different types of movements. Furthermore, more accurate models might result in further computation latencies. Given that rich data sets of movements can be recorded, we think that a machine learning method might appeal better in the context of real-time applications.OpenSim. The limitation of the current implementation lies in the optimization routine’s ability to solve the problem efficiently. Nevertheless, we are exploring how to improve on this and possibly include additional constraints that could originate from direct measurements such as surface EMG to personalize the obtained solutions optionally.Predicting muscle forces are an essential aspect of utilizing musculoskeletal models because direct measurements are challenging and sometimes impossible. Invasive techniques raise many issues, whereas the mapping of muscle activity to muscle force is not trivial. Non-invasive assessment of muscle-tendon loads through vibration tracking  is a proCalculation of muscle forces permits the estimation of joint reaction loads. The plethora of gait retraining systems rely on estimating the knee adduction moment using the ID method to meet applications’ real-time constraints ,42,43. TIn this work, we presented a framework for calculating a multitude of kinematic and dynamic quantities using musculoskeletal models in real-time. Such a system can find application in rehabilitation by providing biofeedback to the users during the training sessions. Notably, the system is designed to work with marker- and IMU-based MOCAP solutions. The second allows exploring scenarios where users’ habits can be monitored and analyzed outside a dedicated and well-equipped laboratory (outdoor activities). The out-of-the-lab setting is still an ambitious goal that is only possible for kinematic analysis. Performing dynamic analysis without additional sensing devices is challenging due to the complexity of modeling any user’s interactions with the environment . Within this vision, we think that the future lies in this direction, where interactions can be accurately predicted, and kinematic and dynamic analysis can then be performed in real-time. This technological leap will unveil new research directions that could target not only applications related to healthcare and human well-being but also translate to other fields as well. We hope that the methods outlined here and the publicly available source code can help meet this grand challenge.


Methods for autonomous navigation systems using sonars in air traditionally use the time-of-flight technique for obstacle detection and environment mapping. However, this technique suffers from constructive and destructive interference of ultrasonic reflections from multiple obstacles in the environment, requiring several acquisitions for proper mapping. This paper presents a novel approach for obstacle detection and localisation using inverse problems and compressed sensing concepts. Experiments were conducted with multiple obstacles present in a controlled environment using a hardware platform with four transducers, which was specially designed for sending, receiving and acquiring raw ultrasonic signals. A comparison between the performance of compressed sensing using Orthogonal Matching Pursuit and two traditional image reconstruction methods was conducted. The reconstructed 2D images representing the cross-section of the sensed environment were quantitatively assessed, showing promising results for robotic mapping tasks using compressed sensing. Navigation systems such as autonomous or remotely operated vehicles in general, including mobile robots, traditionally use lidar, radar or sonar sensors for object localisation and environment mapping. Lidar sensing provides high angular resolution when compared to radar and sonar sensing, but has difficulties in dealing with dusty, smoky or foggy environments, highly polished surfaces or optically transparent materials, such as glass ,2. On thThe conventional detection method in sonar-based autonomous navigation uses the time-of-flight (TOF) concept, which consists of measuring the travel time between the signal burst sent to the environment and the detection of its reflections (echoes) from the existing obstacles ,6. This Traditional approaches to solve these problems focus on minimising crosstalk and noise using probabilistic methods  such as In related work,  uses a cIn , a discuented in , in whicThe present paper addresses obstacle detection and localisation for robotic mapping applications using compressed sensing concepts and sonar sensors operating in air. A specific hardware platform was designed and implemented to send, receive and simultaneously acquire raw reflections from four ultrasonic transducers. Experiments were conducted in a real environment containing one, two and three standard cylindrical obstacles  placed in different configurations. The OMP algorithm  was implAll algorithms were implemented using both real- and complex-valued (Hilbert transform ) models The remainder of the paper is structured as follows: Environment mapping using sonar sensors and the inverse problem approach can be mathematically modelled using Equation :(1)g=Hf+e vector .H matrix is modelled by placing these responses-m samples of the signal acquired along time for each of the n ROI mapped locations-in each of its columns. For systems with more than one transducer, the PSF is formed by concatenation of the reflection signals received from each transducer. This process is based on medical ultrasound imaging research and is detailed in [In sonar imaging, the point spread function (PSF) defines the response of ultrasonic reflections received from a particular obstacle to a given position in a region of interest (ROI) . The H mailed in ,25,26,27H matrix, such that H matrix has an inverse e is known. Reconstruction methods aim to find a solution to the resulting inverse problem by estimating a vector f that satisfies the linear system. Traditional methods are based on minimising a cost function, such as the least squares (LS) approximation [A simple technique to solve the linear system in Equation  is by inximation :(2)f^=arH and H [A well-known solution is based on a deterministic model that estimates the vector Equation  describerix of H .H, this deterministic model becomes unstable, and noise amplified, causing artefacts in the reconstructed image [However, due to ill-conditioning of ed image . To dealed image  and Tikhed image , are useEquation :(5)f^= may be used as (||f||2) ,33. Therlidation  and the lidation . In thisCompressed sensing methods rely on the sparse characteristics of the signal of interest and small inaccurate measurements in the acquired signal ,36. ThisCompressed sensing provides several approaches to solve the reconstruction problem ,38, suchThe mathematical formulation of the method is given by:H matrix in order to select the column k that corresponds to the maximum correlation [The greedy algorithm used in this work to solve Equation  was the relation ,38, as sk  is added to a support matrix S and the signal is reconstructed using the pseudo-inverse solution, as shown in Equation  was used to trigger the transducers, acquire the received signals after amplification and sampling, and send the acquired data to the PC for storage and processing. The hardware has an array of four 50 kHz electrostatic transducers, circuits for transducer excitation and circuits for amplification and sampling of received signals. The Texas Instruments MSP-EXP432P401R LaunchpadIn this work, the hardware was configured to send 16 burst cycles from individual transducers and to acquire the reflected signal in all of the four available transducers. The signals received in each transducer were sampled at 200 kHz and composed signal vectors of 1024 samples, corresponding to a period of 5.12 ms, which allows the detection of objects located as far as 165 cm from the transducers. In this configuration, the period of the ultrasound echo corresponds to 64 samples in the acquired signal vector.The controlled experimental environment defined in this work consists of a surface area of 30 cm H matrix, was performed by triggering each transducer individually and acquiring the corresponding reflections from all of the four available transducers simultaneously. The four acquired signals corresponding to each obstacle location in the grid were concatenated into single column vectors and appended side by side as columns of the H matrix. The process was performed for all 25 ROI positions and repeated for each transducer that was triggered, resulting in an H matrix with f) and each pixel represents a location in the ROI, the number of columns (n) in H corresponds to the number of reconstructed image pixels in f.The linear system identification, i.e., construction of the H matrix are depicted in graphic format in The first 300 rows of the H matrix , Tikhonov (TIKH) and Orthogonal Matching Pursuit (OMP). A Hilbert Transform was performed in the raw signal for enhancing algorithms results. The algorithms were executed using real-valued data (*-R) and complex-valued data (*-C) obtained from the Hilbert Transform .The regularisation parameter value was e method -the sameThe number of iterations used as stop criterion was set for a maximum of 10 iterations, but none of the OMP algorithms has ever reached this number of iterations.The grey scale images resulting from the reconstruction process represent the cross-section of the sensed environment. While white pixels represent empty spaces in the grid, black pixels indicate high probability of a real obstacle being present in the corresponding grid coordinates. H matrix). The compressed sensing OMP algorithm, on the other hand, minimised the effects of noise and the presence of artefacts. All algorithms using the real-valued model reconstructed the environment with an additional non existing obstacle when compared to the ground truth. The OMP algorithm was able to reconstruct an image in complete agreement with the expected ground truth when using the complex-valued model.For the specific position shown in Examples of reconstructed images with two obstacles in the environment  with the highest correlation with the acquired input signal, drastically reducing artefacts in the reconstructed images. For a fair comparison analysis of results, focusing on the localisation of the obstacles, the reconstructed images by the PINV and TIKH algorithms were thresholded using the values shown in Both traditional methods, PINV and TIKH, use all columns of The reconstructed images after thresholding, for the experiments with two obstacles in the environment, are shown in The experiment with three obstacles in the environment presented similar results to the other experiments, as shown in Three different performance analyses were conducted. The first assessment aims at quantifying the similarity between the reconstructed images and the actual environment. The second is intended to quantify reconstruction errors that were preliminarily noticed—for that, obstacles in the reconstructed images were characterised as true positives or false positives. Finally, the Euclidean distance between the coordinates of reconstructed obstacles and the coordinates of actual obstacles in the environment was computed in the third assessment. A non-parametric test was also performed to determine the statistically significant differences between the results.The first assessment consists of a similarity measurement between the reconstructed image and the expected ground truth. A comparison between the thresholded version of the reconstructed image and the ground-truth yields a binary result for this assessment statistic, valued one in case of exact image reconstruction or zero otherwise. All individual comparison results are then added up to compute a single reconstruction score for each algorithm.For the experiments with a single obstacle in the environment, a perfect reconstruction score would be 25, as there are 25 possible locations for a single object in the grid see . Figure All images reconstructed by the algorithms using the complex-valued model are identical to their respective ground-truth images and correspond to accurate representations of the environment (perfect reconstruction scores) in all cases. On the other hand, reconstructions using the real-valued model fail to be perfect in more than two thirds of cases. The results in The reconstruction scores for three obstacles in the environment can be seen in The first assessment indicated that the use of the complex-valued model results in a more accurate image reconstruction when compared to the use of the real-valued model. A rate of The second assessment analysed the number of true positives and false positives regarding the obstacles detected by the algorithms. A true positive is the occurrence of a black pixel indicating the presence of an obstacle at the correct location of the reconstructed image. On the other hand, the occurrence of a black pixel indicating the presence of an obstacle at an incorrect location of the reconstructed image is a false positive. The reconstructed images using the real-valued model present false positives due to the presence of noisy artefacts. The PINV-R and TIKH-R algorithms have shown a large percentage of false positives in the experiments with three obstacles, revealing weaknesses of these algorithms when the number of objects in the environment increases.The algorithms that used the complex-valued model correctly positioned the obstacle in all 25 reconstructed images in the single obstacle experiments, as already verified in the first assessment. In the experiments with two and three obstacles, no false positives were observed, although the results of true positives were Further analysis of the reconstructed images revealed that missing obstacle errors occur when the obstacles in the environment are in the same longitudinal axis (one behind the other from the point of view of the transducers). In this particular spatial configuration of the objects in the environment, the algorithms were able to reconstruct only the closest obstacle to the transducers. The kind of reconstruction error shown in Regarding false positives, which appear in reconstructions with the algorithms using the real-valued model, they are due to the presence of noisy artefacts and justify the low score shown by the algorithms in the first performance assessment (reconstruction score). The algorithms using the real-valued model are not able to deal with noise adequately, especially when there is an increase in the number of obstacles in the environment. The results obtained by the algorithms using the complex-valued model, on the other hand, had no noise artefacts and consequently had no false positives either. However, obstacles were missed when they shared the same longitudinal axis in the environment.The third assessment was an analysis of the Euclidean distance between pixel coordinates of reconstructed obstacles and the actual coordinates of obstacles in the ground-truth. The pixels with the highest intensity and only the exact number of obstacles present in the environment were considered in this assessment. Each pixel in the reconstructed image corresponds to a distance of The results for the experiments with two obstacles in the environment were similar to the ones for the experiments with a single obstacle, as shown in the Finally, the results for the experiments with three obstacles are shown in The non-parametric Kruskal–Wallis test was performed to determine whether there are statistically significant differences between performances regarding the Euclidean distance assessment. The statistical analysis for the experiment with a single obstacle in the environment showed no difference between the algorithms using the real-valued model, but there is statistically significant difference between results obtained using the real- and complex-valued models. The same effect was observed in experiments with two obstacles. Concerning the experiment with three obstacles, the statistical analysis of the Euclidean distance assessment revealed three statistically different performance groups: (1) PINV-R and TIKH-R had the worst performance; (2) OMP-R showed better performance than PINV-R and TIKH-R; and (3) PINV-C, TIKH-C and OMP-C yielded the best results.The assessments demonstrated that the algorithms using the real-valued model, PINV-R, TIKH-R, and OMP-R are not able to deal efficiently with signal acquisition noise and consequently are not capable of reconstructing the experimental environments correctly. Noise artefacts  and misplaced obstacles are observed even after PINV and TIKH reconstructed images were thresholded. The regularisation parameter used by the TIKH-R algorithm did not improve performance, with results close to ones obtained for PINV-R, and despite the suppressed artefacts by OMP-R, the algorithm still fails in most reconstructions.On the other hand, the algorithms using the complex-valued model consistently presented the best results in all assessments, albeit they presented a very specific problem—only the closest obstacle was detected in reconstructions of the environments with two and three obstacles present when they shared the same longitudinal axis. Although this problem influences the analyses of results, it can be addressed in robotic mapping, for example, by moving the robot position and acquiring a second environment reading. Another solution is to use time correction for reflections. However, due to the lack of prior knowledge of the environment, time correction was not used in the experiments.Autonomous systems must allow reliable navigation without colliding with obstacles. Environment mapping using sonar in the air traditionally use the time-of-flight technique to detect obstacles, but suffers from measurement errors caused by interference from multiple obstacles present in the environment. Consequently, the represented environment maps have low resolution and need multiple scans of the environment for proper mapping.A novel approach initiated in  has showReconstructions were performed for one, two and three obstacles in a controlled testing environment using a platform developed for acquisition of ultrasonic reflections. Three algorithms were implemented and the images reconstructed using real and complex-valued models. The experiments were analysed using three assessment methods and presented promising results for use in obstacle localisation. PINV-C, TIKH-C and OMP-C algorithms, all of them using complex-valued models, presented similar results. However, due to better treatment of sparsity by compressed sensing algorithms and the absence of post-processing, OMP-C is considered the best algorithm for obstacle localisation using sonars in air.Future work aims to overcome the existing detection problem that arises when obstacles are positioned precisely in the same longitudinal axis, including experiments with other greedy algorithms such as Orthogonal Least Squares (OLS) , especia


An electric field is the space surrounding an electric charge, within which it is capable of exerting a perceptible force on another electric charge. Especially under high voltage, electric fields induce various electrostatic phenomena, some of which could be utilized to provide remarkable pest control measures. The main focus of the present study was to introduce an attractive force generated by a surface charge on an insulated electrified conductor, which was successfully used to construct an electric field screen that prevented airborne nuisances  from entering the interiors of various facilities. Another focus was the disinclination of insects to enter the electric field, thus, giving the electric field screen the ability to repel insects. Charges accumulated on the surfaces of non-insulated conductors are mobile through discharge, based on their potential difference. Such arc discharge was strong enough to destroy insects that were exposed to it. Some precedent illustrative examples are cited to explain the principles of attraction, dielectrophoretic movement of spores, and discharge-mediated positive electrification of insects, and to discuss how electric fields are generated and used in electric field-based pest control strategies. Pesticides have been the dominant method used for control of all classes of pests . However, the overuse of pesticides has led to the rapid evolution of pesticide-resistant pests. Given the pesticide-resistance problem and consistent public concern to reduce overall pesticide use, pesticide-alternative strategies are urgently required. The protection of crop plants from infection and/or attack by pathogens and pests via safe and environmentally benign methods has been a long-standing goal. Much effort has focused on developing biological and chemical methods to achieve this, including the production of resistant crop plants using conventional and new biotechnological techniques, biocontrol of pathogens and pests using antibacterial, antifungal, and entomopathogenic microorganisms, and the screening of biologically synthesized compounds that inhibit growth of the pathogens ,2. DespiOnce a realistic and reasonable research objective can be settled, it would be possible to advance our steps steadily through a cyclical repetition of creation and destruction of a working hypothesis formulated from the reproducible experimental results and eventually, to propose new conceptions for an electric field screen. An electric field screen is an air-shielding apparatus devised on the basis of principles and techniques of applied electrostatic engineering and was first introduced in a famous American specialist journal ‘Phytopathology’ in 2006 . At the A number of physical methods, especially exploiting electrostatic phenomena, can generate physical forces sufficiently strong to catch airborne fungal spores or small flying insects that may pass through the conventional insect netting used to protect greenhouse crop plants. If an effective force could be generated, such an electrostatic approach would be an extremely promising tool to provide a spore-free and pest-free space for crop plants in a greenhouse environment . In thisPhysical methods could provide an alternative means of managing the pests, since they would be compatible with other components of integrated pest management, have little impact on the environment, and reduce pesticide use, thus, slowing the development of pesticide resistance. In conclusion, the present work demonstrates that the basic electrostatic knowledge brought about a physical tool contributable to social benefits. Thus, the concepts and technologies originating in university laboratory are made readily available to the public in an effort to fulfill scientists′ mission of contributing to society by helping to solve real-world problems and move science forward.The purpose of the present review work is to introduce some parts of the interesting phenomena of high voltage electric fields, which anyone can produce easily with a simple voltage generator, especially in terms of physical control of insect pests. From this point of view, previous works of this field could be substantial precedents for subsuming electric field-based techniques into the existing theory of pest management strategy. More importantly, touching the real phenomena of the electric field by the readers could be a trigger or chance to introduce a new approach to pest control or to improve the apparatus in response to special needs of insect pest problems. These challenging approaches are essential to clarify the total picture of what could happen to the insects in the electric field and how we should utilize these phenomena for constructing a new system of pest control approach. This review paper was prepared to make a first step for this purpose.Electric field is defined as the space surrounding an electric charge, within which it is capable of exerting a perceptible force on another electric charge . The eleA negative voltage generator draws negative electricity, free electrons, from the ground  and supplies electrons to the conducting wire . NegativThe amount of electricity required for electrification is proportional to the applied voltage of the voltage generator. As the applied voltage increases, the amount of electricity increases. The applied voltage corresponds to the potential difference with respect to earth ground. A larger potential difference creates more remarkable electrostatic phenomena as a result of the larger attractive or repulsive force generated. This allows for the capture of insect pests.The application of negative voltage to an insulated conductor allows negative charge to accumulate on the conductor surface, which, in turn, produces an electric field in the surrounding space. This field induces surface charges in the insulating coating of the conductor. In our example, in which the insulator is an acrylic cylinder, opposite charges are created on the inner and outer surfaces of the cylinder. The outer surface charge of the cylinder (a monopole) produces an electrostatic field in the surrounding space , i.e., t® tube, Kokugo, Tokyo, Japan). The most serious drawback of using soft polyvinyl chloride tube is that its volume resistivity is considerably low, thus, limiting the ability to charge the material. Unexpectedly, this limitation was the motivating force to bring about a new method to create the necessary capture capabilities with a lower voltage application . As. As5]. AIn the actual greenhouse experiment, the greenhouse was separated into two rooms by a partition. In one of the rooms, all of the lateral windows were furnished with single-charged dipolar electric field screens. The screens were negatively charged at 4.2 kV, as described in the previous section. In the second room, there were no screens installed. We examined the entry of insect pests into both rooms. The period of each experiment was two weeks, and six experiments were conducted in total. The ventilating fan was also covered with an electric field screen box to prevent pests from invading via the opening left by the fan, especially when the fan stopped.Over the course of the experiment, there were no occurrences of green peach aphid or Western flower thrips. However, moderate to severe whitefly and tomato leaf-minor fly invasions were evident over the same time period. As expected, both pests were completely prevented from getting into the screen-installed room in all series of experiments. In these demonstrative experiments, the practicality of the electric field screen was clearly proven. The severity of the pest invasion was clear from the number of pests trapped by the yellow-colored adhesive plates hung inside the room that was not furnished with an electric field screen. That an intense invasion occurred in one of the half rooms of the same greenhouse indicates that a similar invasion would have been likely in the other half room if preventive measures had not been put into place ,5. Our rDischarge is defined as electric current generation between opposite poles as a result of the dielectric breakdown of gases in the electric field by the potential difference between the poles ,18. The The state of the corona depends on the polarity of the needle pole (a distinction between positive- and negative-charging), the potential difference between poles, and the pole distance. In the case that the needle pole is positively charged, a corona discharge (positive corona discharge) can change from a glow discharge (corona discharge surrounding the tip portion) to a streamer discharge via a brush-like discharge by increasing the applied voltage and/or decreasing the distance between the poles. The discharge finally breaks down with an arc discharge between the two poles  19]. If. If19]. The arc discharge-generating screen consists of two stainless steel nets . One netThe principle is also simple. An electric field is formed between oppositely charged metal nets. One of the nets is negatively charged (negative voltage applied) to create the corona discharge. The glow corona is detected at the convex surfaces of the net (making for a shorter pole distance), which act as needle poles. Over the limit of voltage for corona discharge, arc discharge  occurs along portions of the net. This screen is an arc-discharge-generating type; however, it does not necessarily require mechanical arc discharge. When the insects enter the electric field between the nets, at any location, the insects effectively become intermediate poles. As such, these insects are subjected to an arc discharge from the negatively charged net due to their conductive cuticle outer layer.Eventually, the electricity is transferred to the insect and then, to the earthed net via a two-step arc discharge B. This tThe purpose of the present study was to introduce the potential abilities of high voltage electric fields to workers in the pest control research field. The most notable phenomenon regarding electric fields was the strong attractive force generated in non-discharging electric fields such as electrostatic and static electric fields. This attractive force formed the basis of the electric field screen function. The electric field screen was constructed as an air-shielding apparatus devised by harnessing the nature of the electric field. This apparatus was used to prevent airborne spores of pathogens, flying insect pests, pollen grains that cause pollenosis, and fine smoke particles from entering the inside of various facilities. The electric field screen is applicable to facilities such as domestic housing, hospitals and school buildings, greenhouses for crop production, warehouses and processing plants for post-harvest crops, and animal husbandry facilities. A wide variety of electric field screen structures can be customized to prevent the entry of biotic and abiotic environmental nuisances. All screens have a simple, common structure; therefore, the cost of production is relatively low. A single-charged, dipolar type of electric field screen is now commercially available because the conductors, which are the crucial components of the electric field screen, are coated with polyvinyl chloride resin to prolong screen operation in outdoor environments with minimal deterioration. This type of electric field screen is built and sold by a joint-stock company , which undertakes the design, installation, inspection, and maintenance of the electric field screens. Another application of attractive force was in the production of an electrostatic insect sweeper , which iAnother prominent feature of electric fields is their role in atmospheric events mediated through coronas, streamers, and arc discharge in dynamic electric fields. Using non-insulated conductors, discharge-generating electric field screens were constructed using the same method and configuration. The corona discharge-generating electric field screen produced negative ions to bind with fine particles of tobacco blown into the electric field, by which fine particles of tobacco smoke became trapped by the positively charged earthed metal net of the screen . The eleThe fact that insects avoid entering static electric fields indicates that single-charged electric field screens possess an insect-repelling function as well as insect-capturing ability . The insOur special perspective of future research ranges over many topics. The first is an application of the electric field screen to an unmanned aerial vehicle for remote-controlled monitoring of insects, especially ubiquitous flies transmitting plant and human pathogens over postharvest crops in the facilities of a food supply chain. For this purpose, the replacement of heavy conductor metal wires passed through an insulator tube with proper light materials was essential for considerable weight reduction of the screen. Coating of the inner surface of the insulator tube with conductive paste may be a potential alternative. The use of water in an insulative tube of the screen is a possible alternative due to its considerably higher electric conductivity. Interestingly, various water-soluble colorings provide the screen with a luminous body, which may be useful to analyze photoselectivity by insects. The combination of insulated and non-insulated conductive fibers may make it possible to weave them into a plastic sheet of the electric field screen. Discharge-generating techniques may provide a new tool to detect and dismember the insects nesting in dried grain under the inspection of postharvest crops, or to selectively kill some kinds of flies emerging from underground pupae, which may be useful for an organic farming system. These are only several instances among many possible applications of electric field-based pest control measures. Hopefully, this paper can be an opening to gather valuable research works in this new field.


Orthogonal frequency division multiplexing (OFDM) chirp waveform, which is composed of two or more successive identical linear frequency modulated sub pulses, is a newly proposed orthogonal waveform scheme for multi-input multi-output (MIMO) synthetic aperture radar (SAR) systems. However, according to the waveform model, there will be range ambiguity if the mapping width exceeds the maximum unambiguous width determined by the transmitted signal. This greatly limits its application in high-resolution wide-swath (HRWS) remote sensing. The traditional system divides the echoes by digital beam forming (DBF) to solve this problem, but the energy utilization rate is low. A MIMO-SAR system using simultaneous digital beam forming of both transceiver and receiver to avoid range ambiguity is designed in this paper. Compared with traditional system, the novel system designed in this paper obtain higher energy utilization and waveform orthogonality. Synthetic aperture radar (SAR), as an all-time all-weather, high-resolution active microwave imaging radar, has unmatched advantages over visible light and infrared detection systems. It has strengths in military reconnaissance, disaster monitoring and terrain measurement. It has been extensively used in many fields such as surveying, mapping, marine research and resource exploration. The basic principle is to use the relative movement between the azimuth antenna and the target for virtually synthesize a large aperture, so as to obtain higher azimuth resolution .For the traditional SAR, to increase the mapping width, it is necessary to reduce the aperture of the antenna. The reduction of the antenna aperture will reduce the gain of the antenna, making the power aperture product smaller and the imaging capability worse. In addition, high azimuth resolution requires SAR systems to have high pulse repetition frequency. High pulse repetition frequency will lead to the increase of pulse blind area, which further limits the mapping width. Therefore, single-channel SAR and single-transmit and multiple-receive (SIMO) SAR cannot satisfy the growing demand for mapping. Multiple-input multiple-output synthetic aperture radar (MIMO-SAR) provides more channels and large aperture of the antenna . So, MIMIn MIMO communication and MIMO radar, code division signal is widely used as an orthogonal signal, and has achieved good results. Therefore, scholars have been conducted research on the application of code division signal in MIMO-SAR. Linear frequency modulation (LFM) signal has the advantages of large time-width bandwidth product and ideal correlation performance. It is the most commonly used signal in SAR. Up and down chirp signals are a pair of code division signals. Therefore, MIMO-SAR mostly uses up and down chirp signals as the basic form, combining different modulation methods to form code division signals . RegardlSince the cross-correlation between the frequency division signals is 0, some scholars use them in MIMO-SAR systems. Different antennas are used to transmit LFM signals with stepped center frequencies. The echoes transmitted by different antenna can be separate by band-pass filtering. Subsequently, the separated echoes are rearranged in the azimuth direction to increase the number of spatial samples, thereby improving the azimuth resolution of the system with the same pulse repeat frequency (PRF). However, the above method causes different signal frequencies at different azimuth positions, which will inevitably lead to changes in the scattering characteristics of the target. It will result in a reduction in imaging quality. Some scholars proposed a multi-sub-band concurrent MIMO-SAR HRWS imaging method, which uses multiple antennas to transmit and receive stepped frequency signals at the same time, uses synthesis in time domain to synthesize the received echoes of sub-bands in the range direction . CombineSpace time coding (STC) signals introduce time and space correlations in the signals transmitted by different antennas and improve the diversity gain and coding gain of the system without increasing the bandwidth. STC signals have been widely used in wireless communications . Some scThe azimuth phase modulation signal mainly includes azimuth phase coding (APC) signals and inter-pulse phase modulation signals ,22. AzimOrthogonal frequency division multiplexing (OFDM) is the waveform used in 4G/5G communication. It has large bandwidth and great orthogonality, so there are a lot of researches on the combination of OFDM and MIMO-SAR in recent years ,29,30,31The OFDM chirp signal performs well in both orthogonality and range resolution, but there is an inherent range ambiguity due to modulation and demodulation methods . The traThe remainder of this paper is organized as follows: The MIMO-SAR system model which uses OFDM chirp signal is presented in In this section, the MIMO-SAR model is first introduced, taking four transmitters and four receivers as an example. Then, several typical orthogonal waveforms are introduced. Their advantages and disadvantages are analyzed, respectively. Finally, the reason for the range ambiguity of OFDM chirp signal is analyzed theoretically.As shown in ulfilled . There aV in the azimuth direction, and the origin locates at the center of the antenna array at the slow time η = 0. The coordinate of ground point target T is The platform moves at velocity Through the above analysis and discussions, it can be found that each waveform has its pros and cons. As an orthogonal waveform, the OFDM chirp waveform has been proven to be effective. In this part, the principle of OFDM chirp signal and the cause of range ambiguity will be introduced.The chirp signal is modulated to different carrier frequencies of the OFDM signal. Two-channel improved OFDM chirp signals have been proposed in . In thisTransform (2) into frequency domain and we get:It can be seen from (3), the signal from each group of subcarriers is equal to the chirp signal multiplied by a carrier term. After demodulation, a result equivalent to the chirp signal can be obtained after processing each group of sub-carriers.Taking the four-channel signals as an example, the modulation method is as shown in The chirp signal is separately modulated into four different subcarriers in frequency domain, and then transmitted through the transmitting antenna, respectively. As all the subcarrier frequencies are orthogonal, each chirp signal can be demodulated by extracting the weight of its own subcarrier frequencies from the combination of the four OFDM waveforms.In theory, since each subcarrier is orthogonal, this coding method can obtain four sets of orthogonal signals. However, because of the information is not fully utilized during demodulation, range ambiguity occurs.T. Transform (4) into frequency domain and we get:Equation (4) is the form of OFDM chirp signal in time domain:It can be known from the expression in frequency domain that the four signals are completely orthogonal in frequency domain, and when the information of the corresponding frequency points is extracted during demodulation, the corresponding transmitted signals can be completely recovered.When the signal is received, assuming that the sampling rate is The position of the target is determined by a radar based on the peak of time domain of the signal after pulse compression. In the case where the form of the signal is determined, the cyclic shift of the signal in time domain is equal to a phase shift in frequency domain. As shown in m-th frequency point.The relationship between them is as following:When extracting in frequency domain, the interval between adjacent frequency points is It means that, if the time difference between the arrival of the echoes of two targets is Assume that the transmitted OFDM chirp signal has subcarrier frequency interval of For the limitation of range ambiguity, DBF can be used to limit the width to maximum unambiguous swath width. However, this method uses a small antenna to send a wide beam. The transmission of a wide beam needs to reduce the aperture of the antenna, which reduces the gain of antenna and cannot meet the requirement of signal-to-noise ratio. If a wide beam is not used, the swath width of each imaging strip is seriously limited. If a large mapping width is required, it can only be achieved by inter-pulse scanning or multiple flights. Inter-pulse scanning requires more azimuth PRF, and multiple flight requires more time cost, both of them cannot meet the needs of high-resolution wide-swath mapping.This paper designs a MIMO-SAR system using simultaneous digital beam forming of both transceiver and receiver to avoid range ambiguity and increase the energy efficiency. For multiple transmitting antennas in azimuth, the orthogonality is guaranteed by the generator matrix of the OFDM chirp signal. For the transmitting antenna in each azimuth, the orthogonality between multiple sub-bands in the range direction is guaranteed by frequency diversity and DBF. The schematic diagram of each azimuth antenna in the range direction to achieve multiple sub-bands mapping is shown in Multiple sub-beams are transmitted by digital beam forming in range, each sub-beam adopts signals of different frequencies, and satisfies the demodulation limitation of the OFDM chirp signal. The principle of suppressing the range ambiguity is shown in The signals can be transmitted by a phased array antenna. The following will discuss how to implement this method with 4 sub-bands as an example.Step 1:Set the relevant system parameters. Step 2:According to the number of azimuth transmitting antennas, design Step 3:According to the number of the sub-bands in the range, design the transmission waveform in the range direction of each array element.The steps of the MIMO-SAR imaging method proposed in this paper based on multi-dimensional waveform coded signals and simultaneous digital beamforming of both transceiver and receiver are:The beamforming can be implemented in digital domain or in analog domain. In analog domain, multiple beams can be realized through a passive power synthesis network. In digital domain, multiple signals with different phase offsets can be generated by a digital signal generator. In the system designed in this paper, each orthogonal signal in different transmit antenna is composed of chirp signal with different phase modulation. Chirp signal has constant modulus. Therefore, the nonlinear characteristics of the power amplifier will not affect the characteristics of the signals. Multiple beams can be realized through a passive power synthesis network. In this case, the characteristics of the signal will not be changed. If the signal is synthesized directly in digital domain, it is necessary for the system to ensure a high linearity. The following will introduce the simultaneous digital beamforming technology in digital domain in detail.As shown in L, the spacing of the array elements is When The transmitted signal of the range to each array element is The synthesized normalized antenna pattern can be expressed as:Step 4:The signal is transmitted through the transmitting module.Step 5:Receive the signal through the signal receiving module, then sample the signal.Step 6:Separate the reflected echoes with different range to different subswath through spatial filtering.L array elements in range, so as to obtain signals corresponding to the beam pointing. The corresponding weighting matrix can be expressed as:The spatial filtering method is to perform weight processing on the signals received by Step 7:After filtering in frequency domain, the reflected echoes of different subswath are further separated to improve the isolation between the signals of the respective subswath.Assuming that the received signal of each array element is Step 8:Demodulate each of the separated subswath signal to obtain the receiving echo corresponding to the transmitting antenna in each azimuth. The transmitted signals in frequency domain is expressed as (3).The filtering in frequency domain is performed on the signal after filtering in spatial domain, and the obtained signal is Step 9:Use traditional MIMO-SAR imaging algorithm to process the echoes after steps 6–8.After FFT is performed on the received echoes, the corresponding frequency points in frequency domain are extracted to demodulate the corresponding azimuth orthogonal signal.In order to verify the effectiveness of the proposed system, some simulations were designed. Suppose there are two targets, the distance from the reference point of target one is 100 m and 630 m for target two. Value of parameters is shown in Simulation results of real-scene are as shown in It can be seen from In the above simulation, we only considered the effect of range ambiguity suppression when the uniform linear array is not weighted. If the receiver antenna elevation pattern was tapered by a Dolph–Chebyshev window, the range ambiguity can be further suppressed. It is worth mentioning that, this does not affect the range ambiguity suppression ability of the proposed scheme, since DBF is also used on the transmitter. In The above simulation shows the effectiveness of the proposed scheme. In order to further illustrate the advantages of the proposed scheme, we compare our scheme with the scheme in . Except The performance of the MIMO-SAR system is compared with  in termsNESZ is a parameter which indicates the system radiometric sensitivity. It is defined as the backscattering coefficient corresponding to an SNR that is equal to unity. In MIMO-SAR, NESZ is as follows:As can be seen from The RASR is used to indicate the range ambiguity effect. For the MIMO-SAR system using OFDM chirp signals, this parameter is defined as follows :(15)RASRAs can be seen from In this paper, firstly, the two-channels orthogonal OFDM chirp signals are extended to OFDM chirp signal is sensitive to Doppler-shift, it is a disadvantage. Some scholars have studied how to compensate for the Doppler-shift . However


Singular value decomposition (SVD) methods have aroused wide concern to extract the periodic impulses for bearing fault diagnosis. The state-of-the-art SVD methods mainly focus on the low rank property of the Hankel matrix for the fault feature, which cannot achieve satisfied performance when the background noise is strong. Different to the existing low rank-based approaches, we proposed a simultaneously low rank and group sparse decomposition (SLRGSD) method for bearing fault diagnosis. The major contribution is that the simultaneously low rank and group sparse (SLRGS) property of the Hankel matrix for fault feature is first revealed to improve performance of the proposed method. Firstly, we exploit the SLRGS property of the Hankel matrix for the fault feature. On this basis, a regularization model is formulated to construct the new diagnostic framework. Furthermore, the incremental proximal algorithm is adopted to achieve a stationary solution. Finally, the effectiveness of the SLRGSD method for enhancing the fault feature are profoundly validated by the numerical analysis, the artificial bearing fault experiment and the wind turbine bearing fault experiment. Simulation and experimental results indicate that the SLRGSD method can obtain superior results of extracting the incipient fault feature in both performance and visual quality as compared with the state-of-the-art methods. Rolling element bearings (REB) are key components in the industrial mechanical systems such as wind turbine, high speed railway, aeroengine, etc., which are designed for long-term safe working -corrosion, some potential failures might occur on the rolling bearing surface . Once thOne of the bottle-neck problems is to remove the substantial noise  from the original vibration fault signal ,12. OverA preliminary endeavor of SVD based feature extraction method is to establish the low rank domain for the Hankel matrix of the original signal through SVD operation. Therefore, singular components (SCs) with different characteristics are reliably projected into different lower-rank spaces . This pre et al.  put forwe et al.  proposede et al.  presentee et al.  presentee et al.  proposede et al.  proposede et al.  proposede et al.  revamped(1)We explore the intrinsic structures for the Hankel matrix of the fault feature pattern, indicating that it has the SLRGS property. To the best of our knowledge, it is the first time that the Hankel matrix of the vibration fault signal is modeled as a simultaneously low rank and group sparse matrix estimation problem.(2)A novel SLRGSD framework is proposed for bearing fault diagnosis. The new framework is formulated to promote the SLRGS property. Moreover, the periodic impulses and interference components are split based on the PI index in the low rank domain. The PI is proved to have better fault feature discrimination ability than the PMI.(3)The essential of the SLRGS property for improving the performance of SLRGSD method is verified by the numerical analysis and bearing fault diagnosis experiment. The results indicate that the proposed SLRGSD method can effectively enhance the fault feature.According to above discussion, we can conclude that there are several major shortcomings in the previous studies, which can be remarked as follows: Firstly, the state-of-the-art methods mainly focus on the low rank property for the Hankel matrix of the fault feature pattern. Thus, the structure property is not sufficiently exploited, which cannot provide better estimation result of the periodic impulse fault feature. Secondly, the fault feature may be undermined in the low rank space. This may be attributed that the indicators such as kurtosis and PMI are not ideal enough to discriminate the fault feature pattern with heavy background noise. Thus, the fault feature in the low rank subspace is not the optimal. To address the above issues, a novel SLRGSD method for bearing fault diagnosis is proposed. Different from the current low-rank oriented fault diagnosis approach, the new diagnostic framework is proposed to prompt SLRGS property for the Hankel matrix of the fault feature pattern. Specifically, the periodic impulse fault feature is heightened by the periodic information (PI) indicator in the low rank space. Thus, the fault feature of the rolling bearing extracted by SLRGSD can be enhanced as compared with the current low-rank oriented fault diagnosis approaches. We summarize the main insights and contributions of this work as follows:The outline of this paper is as follows. In In general, the rolling bearings usually work in the harsh environment. A typical observed signal is constituted by the quasi-periodic transients generated from the inner race, outer race and ball fault of bearing, the interference harmonics and the background noise, which can be expressed as:Step 1:Construction of the Hankel matrixThe collected signal is expressed as a time series, which cannot be directly processed by the SVD operation. In consequence, the 1-D collected time sequence is supposed to reshape into a 2-D matrix first for SVD processing. The Toeplitz matrix, cycle matrix and Hankel matrix and were proposed to fulfill this task. Because of the wavelet-like characteristics and zero phase shift property, the Hankel matrix is broadly applied . In the Step 2:Hankel matrix decompositionFor the time sequence Step 3:Reconstruction for feature extractionAfter SVD operation, the signals with different characteristics can be projected into different spaces. For current Hankel based SVD method, the core strategy is to split the fault feature related SCs from interference harmonics and the noise related SCs in the low rank domain. The most energetic SCs preservation method ,34, the Based on above observation, we can find that the current Hankel based SVD methods are designed based on the low rank property of the fault feature. However, it should be noted that the intrinsic structure property for the Hankel matrix of the fault feature is not sufficiently exploited. Moreover, the kurtosis and PMI are not ideal enough to discriminate the fault feature with heavy background noise, which will undermine the fault feature in the low rank space. To obtain better estimation result, it is essential to explore a new framework for bearing fault diagnosis. The highlight of the SLRGSD approach is to establish a new fault diagnosis framework based on the SLRGS property for the Hankel matrix of the fault feature. The high-level steps of the proposed framework are described in The DET has aroused wide attention as a powerful tool for analyzing and recovering the time sequences, which reconstructs the attractor from the observed time-series signals . The delatisfies :(4)vec, we can reveal the SLRGS property for the Hankel matrix of the periodic impulses fault feature, which are detailed elaborated as follows.Low-Rank Property: The low rank property for the Hankel matrix of the bearing fault signal has been demonstrated by several studies such as Ref. ,40,41,42Group sparse property: Another assumption believes that the periodic fault feature has the group sparse property, which were confirmed by the researches from Ref. ,47,48. OBased on above analysis, it is not hard to conclude that the fault feature has the SLRGS property. Thus, we propose a new SLRGSD framework for bearing fault feature extraction. In this framework, we try to construct a new denoising model that in addition to well-fitting the fault feature signal with least square, it has low rank nuclear norm and the groups sparse for the Hankel matrix of the fault feature signal. To finish this task, we define the following optimization problem as follows:In this section, we exploit the solution of Equation (9) using the proximal gradient descend. According to Ref. , if the The  on Ref. , Equatio in Ref.  can be dBased on Equation (11), the derivation process for proximal operator of the For the proximal operator of the low rank term, it can also be gained by incremental proximal algorithm. Before further analysis, we first define soft thresholding operator of the matrix as follows:r values , which cNote that another challenge should be resolved, that is, the valuable SCs should be determined through an appropriate rule. Concerning the periodic impulsive characteristic for the fault feature of the rolling bearing, we redefine an equivalent function of In Equation (16), the PI indicator is proposed to quantitatively measure the fault information in the vibration signal. The PI index  is synthMoreover, for a clear expression, we define the groups sparse term as follows:As indicated in Ref. ,54, the Algorithm 1 Proximal gradient descends for solving SLRGSD.Require: Hankel matrix 1. 2. for 3.4. update 5. update 6. until convergence7. end for8. Obtain the estimated matrix Combining the closed form solutions of Equation (12), (14) and (19), we can gain a satisfying stationary result using the proximal gradient descends for solving the SLRGSD approach. The detailed processes are summarized in Algorithm 1.To extract the weak fault feature from the signal with heavy background noise, a fault diagnosis procedure for the rolling bearing based on SLRGSD is proposed. The detailed processes are as follows:Firstly, the original vibration fault signal In this section, the performance of the proposed SLRGSD approach is evaluated by the simulation signal with heavy background noise. Firstly, we introduce the simulation signals and the performance metrics. Then, the assessment of the PI index and the regularization parameters configuration strategy are conducted. Moreover, the role of the SLRGS property is investigated with the assistance of AIHN indicator. Finally, the effectiveness of the proposed SLRGRD method and its comparison with other methods are depicted in detail. In the numerical experiments, the observed simulation signals generated by the model mentioned in The first term of In summary, the parameters of Equations (21) and (22) are explained as follows: We adopt two indicators, namely the approximate entropy (ApEn)  and the Before subsequent analysis, we examine the advantages of PI index for constructing the weighted nuclear norm. The simulation signal with SNR = −13.2 dB generated in We further define a new indicator to quantitatively evaluate the discrimination ability of fault feature information for the PMI and PI. Inspired by the definition of SNR, we defined feature discrimination indicator (FDI) as follows: SLRGSD method contains three key parameters, including the delay time w signal . We assu to Ref. , the groWe first analyze the influence of parameter Regarding the tuning parameter The highlight of the SLRGSD method for enhancing the incipient fault feature is to prompt the SLRGS property for the Hankel matrix of the fault feature as compared with the current low-rank based methods. Note that the penalty parameter Moreover, we compare the SLGRS property with the low rank-oriented (LRO) strategy for extracting the fault feature. For a fair comparison, the implementation processes of the LRO strategy are shown as follows: Firstly, the Hankel matrix of the original signal is decomposed into several SCs via SVD operation. After that, the PI values of all SCs are calculated for evaluation. Finally. the SC with maximum PI value is selected to reconstruct the fault feature signal. The AIHN values for the LRO strategy of the simulation signals with different noise level are also displayed in In summary, we can draw the conclusions that the SLGRS property is a critical and indispensable strategy to enhance the weak fault feature for the proposed method. Therefore, it is reasonable to affirm that SLGRSH method provides a more robust and effective way of extracting the fault feature for bearing fault diagnosis.In this section, the fault simulation signal generated in RSVD: RSVD method decomposes the Hankel matrix of the original signal into several SCs via SVD operation. Then the information of periodic impulses in each SC is measured through the PMI indicator. When PMI > 1, the relevant SCs was selected to reconstruct the fault feature signal. Hence, RSVD method is developed based on the low rank property for Hankel matrix of fault feature signal. In this research, it should be noted that the reconstructed fault feature is determined by the SC with maximum PMI as the fault feature is contaminate by the heavy noise.PO-VMD: VMD method has attracted wide attention for extracting the bearing fault feature. The fundamental ideal is to decompose the raw input fault signals into a serial of sparse discrete modes. The sensitive sub-components can be affirmed based on tailored statistical indicators such as the kurtosis, the entropy, etc. The parameters such as the mode frequency bandwidth control parameter  in Ref. , the motThe SNR of the simulation signal is set to −12.80 dB. The time waveform and its frequency spectrum of the simulation signal are depicted in Initially, we can obtain the extracted fault feature signal and its envelope spectrum using the proposed SLRGSD algorithm, as displayed in The same simulation signal is processed by the baseline RSVD method. To make a fair comparison, the delay time of RSVD is also set as Furthermore, the parameters optimized VMD (PO-VMD)  method iFinally, the ApEn and SNR output indicators introduced in In this section, two kinds of experiments are employed to illustrate effectiveness and robustness of the proposed SLRGSD method for bearing fault diagnosis. The first one is the artificial inner and outer race fault experiment of the rolling bearing, which is used to evaluate the performance of proposed method under heavy background noise. The second one is the wind turbine bearing early fault experiment, aiming at assessing the incipient fault detecting ability of the proposed method. In addition, the outperformance of the proposed SLRGSD method over the start-of-the-art methods are demonstrated. In this experiment, the artificial fault generated by the electric discharge is seeded at the inner race and outer race of the rolling bearing. The shaft hosts one test bearing and three support bearings for the testing rig, as displayed in  formula ,10, whicWe first analyze the vibration signal with inner race fault, where a segment of original signal with 1s is employed to validate the performance of the SLRGSD algorithm. Note that the Gaussian noise with intensity of 17.75 dBm is added into the original signal. The time waveform of the signal and its envelope spectrum are displayed in The proposed SLRGSD method is performed to identify the fault feature of the vibration signal. To illustrate the outperformance of the SLRGSD method, the state-of the art RSVD method and the PO-VMD method are also to accomplish the same task. According to the rule proposed in To further evaluate the role of SLRGS property and verify the superiority of the SLRGSD method, we compare it with the baseline RSVD method. The delay time To demonstrate the superiority of proposed SLRGSD method, the PO-VMD method is employed to process same vibration signal. According to the parameter optimization strategy, the MFO convergence curve for finding the minimum −PI is shown in To further prove the effectiveness of the proposed SLRGSD algorithm, the outer race fault vibration signal is also employed for analysis. Similarly, the Gaussian noise with intensity of 11.73 dBm is added into the original signal. The time waveform and the envelope spectrum of the signal are shown in Therefore, the proposed SLRGSD method is taken to identify the fault feature. Similarly, parameters We plot the fault feature obtained by SLRGSD method based on the above parameters. The results are shown in In a similar way, the PO-VMD method is applied to analyze the vibration signal with outer race fault for comparison. The MFO convergence curve of searching the minimum −PI is shown in In this section, the bearing fault signal of the wind turbine is employed to verify the effectiveness of the proposed SLRGSD method. The test rig is shown in  to Ref. , the rolFirstly, the signal described in For comparison, the RSVD method is employed to fulfil the same fault detection task. We first calculated the AIHN value of the extracted fault feature using the RSVD method. Notably, the AIHN equals to 0.0174, which is roughly one-third of the maximum AIHN value obtained by SLRGSD method. In addition, as shown in Lastly, the PO-VMD method is adopted to process the bearing fault signal of the wind turbine to verify the superiority of the proposed method. The MFO convergence curve for finding the minimum-PI is shown in To address the defect of the current Hankel matrix based SVD method for bearing fault diagnosis, a novel SLRGSD method is proposed in this research. The highlight of the proposed SLRGSD approach is that the SLRGS property of fault feature is first revealed for enhancing the fault diagnosis performance. Specifically, the proposed framework combines the weighted nuclear norm and the groups sparse to promote the SLRGS property for bearing fault diagnosis. The incremental-proximal descend algorithm is further utilized to obtain the closed solution for the optimization problem. The PI index evaluation and the configuration of the main regularization parameters are discussed in the numerical analysis. Moreover, leveraging the AIHN indicator, the role of the SLRGS property is profoundly demonstrated in the numerical simulation. The results validate that the SLRGS property is a key cornerstone to enhance the performance of SLRGSD method, which is the main motivation and contribution of this research. The proposed SLRGSD method has been successfully applied to process the vibration signals generated by the bearing fault test rig and the wind turbine bearing fault platform. The superiority over the state-of-the-art methods such as RSVD and PO-VMD is verified, indicating that the proposed SLRGSD method can be a protentional tool to enhance the incipient fault feature for bearing fault diagnosis.


Corneal topography is a mandatory investigation in the preoperative evaluation of the patient candidate for laser keratorefractive surgery, in order to assess the corneal shape, to determine the radii of curvature and the corneal thickness. Abnormal corneal topography is the most important identifiable risk factor for corneal ectasia. This paper reviews the principles of successive generations of topographers and illustrates several normal and abnormal corneal topographies.  Therefore, corneal topography is probably the most important investigation in preoperative screening and should be performed in all patients addressed for laser refractive surgery . Corneal topography also plays a key role in postoperative evaluation of patients with unexpected results after refractive surgery .Laser keratorefractive surgery has numerous contraindications, both systemically and locally, of which the corneal ectatic disorders are the most important to recognize at the preoperative evaluation in order to exclude these cases from surgery . In ophthalmology, the term “corneal topography” is improperly used in order to characterize the shape of the corneal surface and its radii of curvature . First-generation corneal topographers are devices that project a system of concentric light circles (the Placido disk) onto the corneal surface, measure their angle of reflection and thus calculate the corneal curvature at various points, providing information on the corneal shape . There are several generations of corneal topographers, according to the technology they use. The Placido disc method is the most widely used . Second-generation corneal tomographers, Orbscan® type , assesses the corneal elevation through optical sections obtained by combining Placido disc and slit scanning technologies, thus being able to characterize the posterior face of the cornea .The term “topography” is derived from the Greek words “topos” (place) and “graphein” (to write) and refers to the study and representation of forms, being used especially in geography and astronomy . The term “tomography” is also derived from the Greek words “tomos” (section) and “graphein” (to write) . The technology of corneal tomographers uses a Scheimpflug type rotating camera that allows the analysis of both anterior and posterior faces of the cornea, by direct measurements, not only by mathematical assessments as in the case of topographers . In contrast, Scheimpflug technology provides less information on possible distortion of the anterior corneal face compared to the Placido disc .Corneal tomographers are new (third) generation devices, their technology allowing digital reconstruction of the structures of the anterior segment, without using a curvature system .Modern capture and analysis systems that combine the Scheimpflug rotating camera, the Placido disk and the slit scanning, such as the Galilei®  and Sirius® , were created to combine the advantages of topographers and tomographers . Compared to ultrasound pachymetry, the topographical systems provide a thicker value of the corneal thickness, as the anatomical limits for measurements are different . However, even if the topographers measure the cornea from the tear film layer to the Descemet’s membrane, the integrated computer can replicate the ultrasound pachymetry values using an adjustment factor of 0.92, the so-called “acoustic factor” .Corneal topographic assessment frequently involves the analysis of four maps: the pachymetric map, the anterior elevation map, the posterior elevation map and a map of corneal curvature . For both corneal surfaces, the analysis of the elevation maps provides valuable data regarding: the corneal apex, the thinnest corneal point and the center of the corneal central region [13]. The posterior corneal surface is very useful for the diagnosis of the preoperative and postoperative keratectasia as it is not altered by the hyperplastic effect of corneal epithelium nor by the corneal surgical maneuvers as the corneal flap creation or the excimer laser ablation [13].The elevation maps are obtained by comparing the anterior and posterior corneal surface to a best fitted surface, usually to a sphere (“Best-fit Sphere”) and recording every difference between both surfaces . For the axial map, the corneal surface is considered to have spherical geometry, consideration that is erroneous [13]. For this reason, the true shape and the power of the peripheral area of the cornea are not properly assessed [13]. The axial map is useful for qualitative analysis of the anterior cornea  and its interpretation is easy for less-experienced users [13]. The tangential map represents the curvature of the corneal periphery [13] more accurately and provides a more detailed analysis of changes of the anterior corneal surface . The corneal surface is reconstructed based on local curvature radii. However, the interpretation of the tangential map is more difficult and more complex than in the case of the axial map [13]. There are more principles for determining corneal curvature maps: axial map  or tangential map  . Physiologically, in childhood and adolescence an up to 0.75 D with-the-rule astigmatism may be present as a result of the slight pressure of the upper eyelid on the cornea . Towards senescence the astigmatism turns against-the-rule, after a period in which the cornea acquires a more spherical shape . There is a wide range of normal topographical aspects, the cornea not being perfectly regular like the calibration spheres . When this aspect is compared with a best-fit sphere as reference level, the central cornea is steeper and the mid-periphery is flatter . On corneal topographic assessment, cold colors (blue hues) encode flattened corneal areas and warm colors (red hues) encode prominent ones .The normal cornea is prolate, which means that it is steep in the center and flat in the periphery . This aspect will be highlighted on the topographic maps as a steeper slope toward blue hues in the nasal cornea . An asymmetry between the upper and lower halves of the cornea is also found . The steepest point is usually located superotemporal to the visual axis while the thinnest point is located in the inferotemporal quadrant .There are several characteristics of normal corneas. The nasal corneal half is flatter than the temporal one .In the normal eyes, the two corneas of the same patient are very similar, both eyes tending to have mirror topographic images (enantiomorphic symmetry)  .regular astigmatism, corneal topography displays a symmetrical bow-tie pattern, the principal meridians (of greatest and least powers) being located 90 degrees apart . In the with-the-rule astigmatism, which is the more common form of regular astigmatism, the steepest radius and the bow-tie pattern are located in the vertical meridian  . Less frequent types of regular astigmatism are the against-the-rule astigmatism, when the bow-tie pattern and the steepest meridian are located horizontally , and the oblique astigmatism, when both the steepest meridian and the bow-tie pattern are diagonally placed  .In Topography of abnormal corneaIrregular astigmatism occurs when two halves of the cornea are consistently different  . The two sides of the bow-tie may differ in magnitude  or may not be orthogonally aligned to each other  or both . Fig. 6) but its presence requires caution in keratorefractive laser treatment and topographic follow-up over a long period of time . In other cases of irregular astigmatism, the corneal topography may highlight the presence of a subclinical keratoconus , may confirm an obvious clinical keratoconus  or may reveal another type of corneal ectasia as pellucid marginal degeneration  .Corneal asymmetry does not always indicate the presence of a corneal pathology  or a combination of multiple indices  [13]. Some of the main indices assessed are: simulated keratometry, surface asymmetry index, surface regularity index, central keratometry, inferior-superior index, average corneal power, skew of the steepest radial axis, irregular astigmatism index, apex curvature, asphericity coefficient, etc. [13]. Each of these indices is highly specific for the corneal topographer for which it has been developed, the values obtained could not be extrapolated from a corneal topographer to another [13]. In addition to generating corneal topographic maps, another important use of corneal topographers and tomographers in the preoperative evaluation is the assessment of various indices that allows quantification of the level of irregularity of the corneal morphology . Both anterior and posterior corneal faces can be characterized this way . Corneal topography and tomography are also useful for assessing the shape and degree of the astigmatism . Abnormal corneal topography is the most important identifiable risk factor for corneal ectasia [16]. Corneal topography is a mandatory investigation in the preoperative evaluation of the patient candidate for laser keratorefractive surgery, in order to assess the corneal shape, to determine the radii of curvature and the corneal thickness .The corneal topographic and tomographic evaluation is performed for each point of the corneal surface, thus, after being color-coded, the results are transposed into pachymetric, elevation and power maps of the cornea [AcknowledgementsNot applicable.Sources of FundingNone.DisclosuresNone of the authors has any financial or proprietary interests to disclose.