Power generation using waste-gas is an effective and green way to reduce the emission of the harmful blast furnace gas (BFG) in pig-iron producing industry. Condition monitoring of mechanical structures in the BFG power plant is of vital importance to guarantee their safety and efficient operations. In this paper, we describe the detection of crack growth of bladed machinery in the BFG power plant via vibration measurement combined with an enhanced spectral correction technique. This technique enables high-precision identification of amplitude, frequency, and phase information (the harmonic information) belonging to deterministic harmonic components within the vibration signals. Rather than deriving all harmonic information using neighboring spectral bins in the fast Fourier transform spectrum, this proposed active frequency shift spectral correction method makes use of some interpolated Fourier spectral bins and has a better noise-resisting capacity. We demonstrate that the identified harmonic information via the proposed method is of suppressed numerical error when the same level of noises is presented in the vibration signal, even in comparison with a Hanning-window-based correction method. With the proposed method, we investigated vibration signals collected from a centrifugal compressor. Spectral information of harmonic tones, related to the fundamental working frequency of the centrifugal compressor, is corrected. The extracted spectral information indicates the ongoing development of an impeller blade crack that occurred in the centrifugal compressor. This method proves to be a promising alternative to identify blade cracks at early stages. Containing combustible and contaminating gases such as CO and CH4, the emission of BFG without proper processing emerges as a major source of the fog and haze in China. However, owing to its property of low heat value, it is relatively difficult to realize repeat utilization of BFG. In current steel-making operations, the development of furnace gas utilization for power generation is rapidly promoted because it enables alternative disposal of the problem gas while simultaneously harnessing it as an energy source [The iron and steel manufacturing industry is one of the most energy-intensive industries in the world. As a kind of byproduct, blast furnace gas (BFG) is an important type of waste gas generated during blast furnace operations where iron ore is reduced with coke into metallic (pig) iron . It is ry source .The system of a BFG power plant consists of multiple types of rotary mechanical components such as motors, bearings, gears, and other bladed machinery. Normal and safe operation of these mechanical systems is of vital importance to ensure highly efficient reutilization of BFG. However, these mechanical components usually operate at severe working conditions of high temperature, varying working speed, and heavy load. Therefore, failures due to fatigue are likely to occur, resulting in not only economic loss but also catastrophic accidents . To prevCompared with other rotary components , in situ dynamic analysis of bladed machinery is more difficult. Among the various failure types of bladed machinery, a significant problem that has not been properly addressed is the detection of crack growth. Doing work to the input BFG via impeller blades, the bladed machinery can generate high-pressure output gases. Because the working impeller blades must be exposed to extreme environments of high temperatures and corrosive BFG at high speeds, cracks are inevitable after long service. However, it is extremely difficult to detect incipient cracks using online condition monitoring techniques, especially at early stages of crack development. Typically, a fatigue crack becomes conspicuous only after the crack reaches about 80% of the total fatigue life of a structure . This prIn this paper, we attempted to monitor the health state of the bladed machinery in a BFG power plant. Vibration features, induced by a blade crack developed on a centrifugal compressor, were extracted using a novel signal processing approach. This approach derives harmonic information of high precision from the vibration signals. The varying of the health state of rotary machines can lead to a change in deterministic harmonic components, which are located in the lower frequency range of the collected dynamic signals. Traditionally, fast Fourier transform (FFT) is suitable for extracting information about these harmonic components. However, owing to the analyzing characteristics of FFT, severe distortions on information of frequency, amplitude, as well as phase are likely to occur . In the To achieve good spectral correction performance in low signal-to-noise (SNR) conditions, some comparatively classical methodologies turn out to be very effective. These methodologies include zero-padded discrete Fourier transform (DFT) , the chi3/h) in the world in 2011. An accidental event occurred due to a fully developed blade crack on the centrifugal compressor, which is the key part of a booster fan. In the described case study, a range of vibration sensors were mounted on bearing housings of the shaft, on which the centrifugal compressor is installed such that vibration signals were collected. Ten vibration tests were conducted and a few records of velocity signals were collected via a data acquisition system. To investigate the harmonic information of the harmonic tones related to the fundamental working frequency, we propose a novel spectral correction technique with a post-processing step of active frequency shift operations on the FFT spectrum. Essentially, this proposed technique utilizes a conventional ratio-based spectral correction method on the information of interpolated Fourier spectral bins. The enhanced performance of the proposed active frequency-shift spectral correction (AFSSC) method is validated via numerical simulations. Sinusoidal signals corrupted with white Gaussian noise are employed to compare the performance of the proposed AFSSC with three other comparison methods. It is demonstrated that the proposed AFSSC using rectangular window possesses the most superior spectral correction capability. Moreover, a normalized health state indicator is further constructed using the corrected information to measure the energy weight of the harmonic tones. After using the health indicator, the energy weight of the component 2×, the second-order harmonic tone of the fundamental working frequency, successfully reveals the ongoing development of the blade crack.The power generation system studied in this paper is a celebrated power plant that had the highest burning rate , frequency (f), and phase (From Equation (2), it is demonstrated that FFT only provides frequency information at the angular frequency In Equation (4), In actual measurements, In a specific realization of discrete Fourier transform, if the recorded sample length is However, it should be pointed out that the choice of Additionally, we define the following useful concepts of frequency error (FE) with its normalized version when the digital signal is segmented with a specific type of window function. According to the fundamentals of signal processing, the actual single tone harmonic component is associated with the point with the maximal modulus in the spectral domain. We also name such a point a window vertex. Window vertexes may not be available in the fast Fourier Transform (FFT) spectrum because FFT only provides spectral bins evenly spaced at the spectral resolution It is also known that In this subsection, we mainly focus on a review of two classical ratio-based spectrum rectifying methods . That isThe rectangular window of length Its spectral counterpart is expressed asThe approximation in Equation (13) is made on the assumption that the sampling length The normalized frequency width of a main lobe is Assume a simple harmonic component whose harmonic parameters are For simplicity of explanation, a passenger function,In ratio-based spectral correction, Hence, the index for the actual frequency can be obtained as A cosine window is defined using the following mathematical prototype:Correspondingly, its spectral counterpart is expressed asBy setting parameter lue is 1 a can be The plot of modulus function in Equation (23) is shown in Without loss of generality, we illustrate high-precision spectral correction using the Hanning window. Let Substituting the variables Equivalently, the above relation can be expressed asThe normalized frequency error Accordingly, the Hanning-window-based spectral correction for the harmonic information of amplitude, frequency, and phase can be summarized asAlthough ratio-based spectral correction can achieve high precision and adopts different kinds of window functions, a common characteristic is the utilization of complex-valued spectral bins located within the main lobe in the frequency domain. In a high signal-to-noise ratio (SNR) situation, for spectral correction problems of a single tone, the Hanning window is reported to be of higher precision compared with the rectangular window . MagnituFrom For simplicity of comparison, we can set the amplitude –6 of the amplitude at the window vertex.A Hanning window offers better energy concentration in comparison with a rectangular window. However, no matter which type of window is employed, aliasing between the spectrum of the positive-frequency part and that of the negative-frequency part is inevitable. From w vertex c. In othIn a noise-free situation, the above conditions explain why a Hanning window is superior to a rectangular window in rectifying a simple sinusoidal wave. However, problems regarding the correction precision would occur in both types of window functions when other interference components, for example the measurement noise, are presented in the recorded signals. The causes are listed below.(1) As stated in (2) The higher the amplitude of an MLSB is, the greater noise resistibility is obtained. However, it is not possible to ensure that all the MLSBs are high in amplitude in the FFT grids when the normalized frequency error As explained in the above sections, the correctness of MLSBs plays an important role in guaranteeing precision results. It is expected that all MLSBs are large in amplitude such that each of them is equipped with robust noise resistibility. In this subsection, we attempt to investigate the energy distribution of the spectral bins located in the main lobe of the window function. The following concept of condition number for the MLSB set is defined. Let For the Hanning window,In the above definitions, the operator As indicated in L samples, the procedure of the proposed ASFFC algorithm is summarized as below. In the procedure, an index integer Assuming that we have collected a digital signal Step 1. Let Step 2. In the spectrum of the original signal, find some frequency ranges within which the amplitudes are large and are related to the characteristic frequency of the investigated mechanical system. A set of such spectral bins is denoted asThe values of Step 3. Let the normalized integer index associated with the frequency Step 4. Compute the harmonic information of two MLSBs, which are denoted as Step 5. Correct the harmonic information using the two MLSBs in Step 4 via a ratio-based spectral correction method using a rectangular window. Let the corrected information of frequency be Step 6. Compute some interpolated spectral bins using the original definition of DFT. Let the normalized frequency increment Step 7. Find two spectral bins in the set Step 8. Apply ratio-based spectral correction algorithm as we introduced in the rectangular window case using the two selected spectral bins (Equations (18) and (19)). As a result, the corrected harmonic information Step 9. Synthesize a simple sinusoidal signal using the obtained corrected harmonic informationStep 10. Let The above procedure can be divided into two parts. Step 6 refers to the active frequency shifting operation, which is capable of enhancing the noise resistibility of spectral bins in the subsequent spectral correction steps. Step 7 is the classical spectral correction using a rectangular window. It should be mentioned that the information of frequency is corrected prior to the frequency shifting operation and the information of amplitude and phase is estimated after the frequency shifting operation, as illustrated in The interpolated spectral bins around an actual harmonic component are demonstrated in the schematic diagram of Curves of condition numbers belonging to a rectangular window as well as a Hanning window, after the spectral interpolation step, are displayed in In this section, we attempt to verify the enhancement of the proposed AFSSC method in retrieving harmonic information corrupted by strong noises.Assuming that a record of measured simple sinusoidal signal, corrupted by white Gaussian noise Let the frequency of the harmonic component in Equation (41) be set to 20.01 Hz. Owing to the fact that the spectral resolution of In As the analyzing frequency of the simulated signal is 500 Hz, which is half of the sampling frequency according to the Shannon’s information theory of sampling, the parameter In the following tests, instead of selecting a fixed frequency shift, we explore the correcting performance of the proposed AFSSC method when i) RecRB offers the optimal performance in correcting the information of amplitude, especially when ii) RecAFSSC is superior in correcting the frequency no matter what the value of i) for higher values of CN, attained via the active frequency shifting, operation is beneficial for enhancing the correction accuracies of frequency and phase, and (ii) higher values of CN can reduce the correcting accuracy of amplitude.The Y axis in As mentioned in In this section, a case study regarding the structural health monitoring of a BFG power plant is introduced. The project of this power plant was launched by a major manufacturer that supplies high-end steel products in China. As this manufacturer is located in a major metropolis of eastern China, there is high demand regarding environmental protection issues. This power plant was established to promote exhaustive burning of environmentally harmful BFGs such that residual energies can also be recycled for steel product manufacturing. A schematic diagram of the BFG power plant is given in 3/h, the highest burning flow rate in the world in 2011. As indicated in The installed capacity of this power plant is 350 MW. In addition to BFG, alternative input fuels of the power plant can also be coke oven gas (COG), corex gas , natural gas, or light oil. The power plant is world-renowned for its combustion boiler. The maximal flow rate of the boiler is 1131 kmAn accident occurred on the booster fan of this BFG power plant. This accidental event was caused by a broken blade on the centrifugal compressor. There are 11 impeller blades arranged in the radial direction of the centrifugal compressor. The fault source was found to be a fully developed crack on the faulty impeller blade , which rThe faulty blade was thrown away due to strong centrifugal force at a time when the booster fan was operating at the lower speed (744 r/min). The faulty blade knocked against the volute and caused severe structural damage. A piece of volute debris even broke the housing of the booster fan, as shown in The mechanical transmission chain of the booster fan is illustrated in As shown in To investigate the historical measurements of this faulty booster fan, the collected vibration signals stored in the factory database were checked. There are 10 sets of historical data that can be traced. The data, along with the scheme of each measurement, are listed in L are defined asIn order to make an overall analysis of the vibration measurement, a statistical analysis was performed. Two indicators, the root mean square (RMS) value and the kurtosis in the time domain, were used. RMS can detect the energy amplitude of the time series, while kurtosis can measure the peakiness of the potential impact components. The two indicators for discrete series In In According to the results revealed by According to the materials presented above, the rotation frequency of the centrifugal impeller at low working speed can be computed asThe frequency resolution of the vibration measurement isi-X denote the corresponding ith-order harmonic component of the fundamental working frequency.This means that we can obtain frequency bins, related to rotation frequencies form the Fourier spectra, at 12.5 Hz and 16.25 Hz. From the fast Fourier spectra, the estimated working frequencies are In × and 3× f,h.As such, in the following analysis, attention is focused on the information of 1×, 2×, and 3×. Using the proposed AFSSC, corrected information regarding amplitude, frequency, and phase is listed in jth order harmonic component in the ith vibration test be denoted byith test, the velocity signal To give a precise description of the information, we utilize the concept of a simple harmonic phasor to incorporate the corrected spectrum information Because the dimension of the original vibration signal is velocity (mm/s), an operation of time domain derivative is required to transform For the acceleration signal jth-X. As for the fundamental frequency, the normalized energy weight became larger from Test No. 1 to Test No. 3, and this indicator decreased in later tests. Remarkably, values of the energy weight of the second order decreased in early tests and increased significantly in later tests. There is no obvious growth trend regarding the third-order harmonic component.The associated energy weights are shown in From the above results, it is concluded that the growth in the amplitude of higher-order harmonic component successfully indicates the evolution of a dynamic imbalance in the centrifugal impeller. Therefore, the normalized indicator, proposed based on the AFSSC technique, can serve as an effective measure of the crack growth in the blade.To validate the enhancement of the proposed AFSSC in harmonic information recovery, we also employ FFT and ratio-based spectral correction method to process the signals.If we interpret the harmonic information by merely using FFT, there is no correction of amplitude or phase and the spectral bins with maximal modulus around the harmonic tones are investigated. Following the procedure introduced in jth order single-tone component in the ith test be defined asj = 1, 2, 3 and To make a comparison, results in In the early tests, the amplitude of the components 2× and 3× are low in energy, therefore the error between ach test d.In this paper, we propose the AFSSC algorithm to address spectral correction problems in the field of structural health monitoring. AFSSC achieves a good trade-off between efficiency and accuracy. It is suitable for correcting harmonic information from signals of relative short sampling length. To the authors’ knowledge, there have been many studies employing FFT and its related spectral analyzing techniques. Such techniques are often adopted to investigate the physical behaviors of materials. Li and Shen applied FFT as an alternative approach to the finite element method to compute the effective properties of composite materials with a periodic microstructure . Nichola(1)A novel spectral correction method is proposed. Rather than using spectral bins of the FFT spectrum for correcting all harmonic information , the proposed AFSSC method utilizes zoomed-in spectral interpolations around the actual harmonic components. The amplitudes of the two spectral bins, employed for ratio-based spectral correction, are high in value such that robust noise resistibility is ensured. By utilizing the active frequency shifting operation, the values of CN are increased, which is beneficial for enhancing the correction accuracies of frequency and phase.(2)Two numerical simulations are conducted to validate the enhanced effectiveness of the proposed method. Sinusoidal signals are simulated with a small frequency shift compared to the FFT grids. In the presence of strong white Gaussian noise, RecAFSSC is superior to any other comparison technique in correcting information of frequency and phase for all possible normalized frequency error. RecAFSSC is also the most advantageous method for correcting the phase information when (3)A case study on the structural health monitoring of BFG power plant is conducted. There are 10 historical records of vibration signals before the occurrence of an accident involving the centrifugal fan. This event was caused by a crack that developed in the impeller blade. These data were collected from sensors mounted on the bearing housing. In analyzing the data, classical statistical indicators failed to reveal crack features as no explicit evolution trends with respect to time are detected. After applying the proposed AFSSC, the harmonic information related to fundamental working frequency is extracted with high precision. A normalized health state indicator, measuring the energy weight of each harmonic tone, is constructed. The results show that the normalized energy weight of the 2× component, the second-order harmonic tone of the fundamental working frequency, increases as time passes. This indicator successfully indicates the ongoing development of an incipient crack in the fault blade of the machinery.In this paper, we investigate crack detection problem of blade machinery using vibration-based condition monitoring techniques. The major findings are summarized as follows:


The Centrifugal compressor is a piece of key equipment for petrochemical factories. As the core component of a compressor, the blades suffer periodic vibration and flow induced excitation mechanism, which will lead to the occurrence of crack defect. Moreover, the induced blade defect usually has a serious impact on the normal operation of compressors and the safety of operators. Therefore, an effective blade crack identification method is particularly important for the reliable operation of compressors. Conventional non-destructive testing and evaluation (NDT&E) methods can detect the blade defect effectively, however, the compressors should shut down during the testing process which is time-consuming and costly. In addition, it can be known these methods are not suitable for the long-term on-line condition monitoring and cannot identify the blade defect in time. Therefore, the effective on-line condition monitoring and weak defect identification method should be further studied and proposed. Considering the blade vibration information is difficult to measure directly, pressure sensors mounted on the casing are used to sample airflow pressure pulsation signal on-line near the rotating impeller for the purpose of monitoring the blade condition indirectly in this paper. A big problem is that the blade abnormal vibration amplitude induced by the crack is always small and this feature information will be much weaker in the pressure signal. Therefore, it is usually difficult to identify blade defect characteristic frequency embedded in pressure pulsation signal by general signal processing methods due to the weakness of the feature information and the interference of strong noise. In this paper, continuous wavelet transform (CWT) is used to pre-process the sampled signal first. Then, the method of bistable stochastic resonance (SR) based on Woods-Saxon and Gaussian (WSG) potential is applied to enhance the weak characteristic frequency contained in the pressure pulsation signal. Genetic algorithm (GA) is used to obtain optimal parameters for this SR system to improve its feature enhancement performance. The analysis result of experimental signal shows the validity of the proposed method for the enhancement and identification of weak defect characteristic. In the end, strain test is carried out to further verify the accuracy and reliability of the analysis result obtained by pressure pulsation signal. During the operation of a centrifugal compressor, the blades suffer from combined effects of centrifugal force, unsteady flow and vibration and so forth . TherefoMany scholars studied the blade condition monitoring with different ways. For example, Rao et al., proposed a method to extract the characteristic information from vibration signal of the gas turbine for recognizing the blade state . Witek sA big problem should be noted is that the characteristic information representing the blade crack is often submerged in the complex background noise, which will lead to the weak defect characteristic difficult to identify from pressure pulsation signal. Therefore, it is necessary to study the method that can effectively extract the weak crack information from the test signal with strong background noise. Restraining noise seems to be a common and effective method, which can inhibit the noise and increase the signal-to-noise ratio (SNR). However, it is not useful for the weak signal with strong noise. Stochastic resonance (SR) can provide a new idea compared to the traditional noise suppression methods. That is because the stochastic resonance is able to transfer part of noise energy to the weak characteristic through a nonlinear system. As a result, it can improve the quality of weak signal . So, theThis paper uses CWT to filter the sampled signal around blade passing frequency first and then utilizes Gaussian potential (GP) to change Woods-Saxon single stable stochastic resonance system into a bistable stochastic resonance system for extracting the weak characteristic frequency excited by blade crack defect. GA is used to optimize the related parameters. The method is validated with the analysis of simulation signal and actual test signal. The detailed structure is shown as follows: Wavelet transform developed from the traditional Fourier transform is a local transform in time and frequency domain which can extract the details of signal through multi-scale analysis. It has been intensively investigated and applied to extract fault characteristics of mechanical equipment from its vibration signal as it can effectively filter noise and preserve characteristics information. In this paper, continuous wavelet transform is used to pre-process the obtained experimental signal and its definition is shown as Equation (3). Set Then Wherein The inverse transform equation indicates that there is no energy loss for CWT and the energy is conserved. Thus, Equation (5) is valid. Moreover, the modulus of the signal’s CWT coefficient Woods-Saxon potential (WSP) and GP are combined to form the new potential function of bistable stochastic resonance system which will be described in The potential function Wherein, The radial GP well model can be expressed as follows:Wherein, The WSP and GP models mentioned above can be combined into a new bistable potential well model. And the new WSG potential function can be expressed as follows:The shape of the new potential function is shown in As the bistable stochastic resonance can enhance the weak low frequency signal by utilizing the noise energy reasonably, it can be thought as a nonlinear low pass filter in some literature ,23. TherTaking Brown particle’s over damped motion excited by noise and external driving force in a bistable system into consideration, the stochastic resonance system can be described by the following equation:Wherein Equation (12) above represents a classical bistable stochastic resonance system through the nonlinear Langevin equation. If WSG bistable potential well model is used to replace the traditional potential function Wherein, It can be seen from Equation (13) that the system output depends on the potential model and input signals. The difference between WSG stochastic resonance system and the classical system lies in the potential model. In fact, the occurrence of stochastic resonance is determined by the combined effect of the system, input signal and the noise. The stochastic resonance phenomenon can be described by the oscillation of Brown particles under a variety of excitation  in a created special scene. In all forces, potential well force is obtained from the first-order derivative of potential function. In reality, the periodic force and the noise power are fixed, so the effectiveness of the stochastic resonance system is largely determined by the potential well force.In general, if the potential well is too wide, the particle can’t reach the potential well wall of the other end or the barrier. It also means the particle needs greater resilience to help its periodic motion in a cycle. On the other hand, if the well is too narrow, the particle can’t reach the expected position along the right path in a cycle, because the wall of the potential well will force the particle return back to its original position in advance. Similarly, the excessively high barrier will make it difficult for the particles to move back and forth in two potential wells, as a result they will only do the reciprocating motion in one potential well which will have no effect on enhancing the weak signal from strong noise. If the barrier is too low, the motion of particles will have no periodicity and just be controlled by the effect of noise. That is to say, the output signal is disordered. In addition, if the potential well wall is too steep, the produced great resilience will further lead to the rebound of the particles. Conversely, it will not provide sufficient energy for particles to complete the periodic movement if the well wall is very flat. In conclusion, the periodic motion of the particles and the amplification of a weak characteristic signal can be ensured only when the shape of the potential model reaches the best state. Therefore, the corresponding parameter optimization method is necessary.(1)Signal pre-processing based on CWT: As blade passing frequency, a high-frequency component calculated by multiplying blade numbers with rotating frequency, is the main frequency during the operation of the centrifugal compressor, the low-frequency component caused by blade defect may be modulated to the blade passing frequency ,25. In a(2)Selection of parameters to be optimized: As (3)Parameters setting and initialization for GA algorithm: Several required parameters need to be set when using GA algorithm. They are population size (4)Output signal calculation and parameters update: Each produced chromosome is brought into Equation (13) and then this equation is solved by using the Fourth-order Runge-Kutta method to obtain the output signal, where the processed envelope signal is set as the input weak periodic signal. Next, calculate the SNR of output signal with Equation (14) and set it as objective function value. Searching the optimal parameters based on fitness function and updating the generation. Repeat the above procedures until the optimal parameters are obtained.(5)Weak characteristic enhancement and extraction: Based on the obtained optimal potential parameters Based on the CWT filtering and WSG stochastic resonance feature enhancement method, the CWT-WSG method is proposed to extract the weak characteristic information. As the output signal quality of WSG stochastic resonance method is determined by potential parameters, the optimal output signal can be obtained by tuning related potential parameters. Therefore, Genetic algorithm (GA) is studied here for the determination of optimal parameters. Signal-to-noise ratio (SNR) of the output signal shown by Equation (14) is set as the objective function in GA.As the weak characteristic frequency induced by the blade crack defect is usually difficult to be identified, the proposed method is used here to improve this problem. Considering the strain method is not suitable for the long-term condition monitoring of all the blades, pressure pulsation signal is adopted in this paper. Then, the proposed CWT-WSG SR method combined with GA is used to enhance and extract the weak defect characteristic frequency from the pressure signal. Strain data will also be sampled and analysed to evaluate the reliability and effectiveness of the analysis result obtained by using the proposed method. In the condition that the consistent defect characteristic frequency can be extracted from both strain signal and pressure pulsation signal with the proposed method, it can be verified this method is effective and has potential in the application of long-term condition monitoring and weak defect warning for large-scale centrifugal compressor blades. The detailed scheme is shown by the following system diagram seen as In order to illustrate the effectiveness of the proposed CWT-WSG stochastic resonance method, a simulation signal is analysed first. Considering the actual situation that amplitude modulation phenomenon will occur if there is defect in the blade, an amplitude-modulated signal is constructed shown by Equation (15).Wherein the carrier frequency In order to extract the weak modulation frequency 60 Hz, CWT method is used to filter the signal around 1500 Hz first shown in For further extracting the characteristic frequency, WSG stochastic resonance model is constructed to process the envelope signal. Related optimal parameters of the WSG model are set as The centrifugal compressor test rig is shown in In order to accurately collect crack defect information of the blade, appropriate data acquisition system and sensors are needed to collect and read data. NI4472 eight channel dynamic signal acquisition module, a synchronous signal acquisition module with 110 dB dynamic range, 24-bit resolution and 102.4 kHz sampling frequency, is selected as the hardware and it constitutes the test system with the DASP software of East China Institute of Noise and Vibration during the experiment. Dynamic pressure sensors are used to monitor the pressure pulsation signal in the experiment. The principle of the sensor is that sound wave can cause the vibration of air. The pulsant air will then press the diaphragm and the diaphragm will generate an electric signal under the action of the mechanical force. The test results are closely related to the arrangement of measuring points. How to arrange the data points is a key factor to the success of the test and it is also an important guarantee for the validity and accuracy of the data. In addition, measuring points positions play an important role for the effective extraction of the blade crack information in the subsequent data processing step. Therefore, it is necessary to design measuring points reasonably before the experiment. The reasonable positions of the measuring points must meet the following requirements: (1) The layout of measuring points need to meet the requirements of the test; (2) The positions of measuring points need to ensure the authenticity and validity of the test data; (3) The positions also need to meet the requirements of site conditions, which means the test equipment should be placed on the points easily and safely. The pressure sensors are placed on the impeller inlet, the diffuser inlet and exit respectively to monitor the pressure pulsation signal in this experiment and Considering the defect characteristic is weak, the used pressure sensors should be placed near the position of the rotating impeller. According to the actual situation of the Φ800 test rig, radial holes are machined in the selected measuring points so that pressure sensors can be embedded. The probe of sound pressure transducer stretches a certain distance into the pipe inner wall, which is very close to the impeller so that the useful pressure signal containing the crack information can be obtained. PCB 106B is chosen as the sound pressure sensor in the experiment. The maximum dynamic pressure it can withstand is 68.9 KPa.Pressure pulsation signal is generated due to the interaction between the rotor and the stator inside the compressor and the main characteristic frequencies in normal operation conditions are blade passing frequency and its harmonic frequencies. In order to get crack information of the cracked blade comprehensively, a test is carried out on a compressor with a cracked blade at 5000 rpm working condition. Pressure pulsation signal of the three positions mentioned above are collected synchronously with multi-channel acquisition system. Therefore, the feature frequency caused by blade crack needs to be demodulated with further analysis. As the blade passing frequency is 1083.33 Hz, CWT is first used to filter the signal around 1083 Hz and the filtered signal is reconstructed through continuous wavelet inverse transform shown as According to the optimization result of GA, the optimal parameters of the WSG stochastic resonance model are set as It can be concluded from the above analysis that a low vibration frequency 53 Hz is generated for the cracked blade during the rotation of the impeller. This low frequency vibration of the cracked blade will act on the air and can be found from pressure pulsation signal nearby. As blade passing frequency belongs to high frequency component, the 53 Hz will be modulated to it seen as the sideband. CWT-envelope method can’t extract this weak defect frequency effectively because of the strong noise interference. Conversely, this characteristic frequency can be found easily with further processing by CWT-WSG stochastic resonance method. In order to further verify the validity of the result, strain test and analysis are also conducted described as follows.The blade’s strain test acquisition system consists of a SG403/SG404 four-channel wireless strain sensor node, a wireless receiving gateway, BeeDate wireless sensor network software and BF120-2AAGN20-W strain gauges. The installation of the wireless strain node is shown in Analysing the strain signal obtained at 5000 rpm working condition. The acquired spectrums are shown in Condition monitoring and incipient weak defect warning for centrifugal compressor blades is always a big challenge as the impeller works in a closed space, which means the signal that can effectively reflect the blade’s state is difficult to acquire. Traditional non-destructive testing and evaluation (NDT&E) methods can determine the blade defect effectively, however, these methods can only be performed during the shutdown period of the compressors, which are time-consuming and costly. Therefore, they are not suitable for the long-term on-line condition monitoring of blades. Considering the interaction between the rotating impeller and the airflow nearby, the blade’s abnormal vibration induced by the crack defect will affect the around pressure pulsation of the airflow. Therefore, pressure pulsation signal is used for the on-line condition monitoring and defect characteristic extraction in this paper. However, as the characteristic information is usually weak together with strong noise, traditional signal processing methods cannot identify it effectively. CWT is used to pre-process the sampled signal and WSG stochastic resonance is then adopted to enhance the weak characteristic frequency. GA is used to obtain the optimal parameters for improving the enhancement performance of this stochastic resonance system. The accuracy of the proposed method is verified by the strain experiment. In conclusion, this paper provides a new idea for the weak defect characteristic enhancement and the identification of centrifugal compressor blades. For future research, a study on multi-cracks of different sizes should be conducted by using a pressure pulsation signal, which can make this method more helpful in practice. In addition, as the synergistic effect of noise can better improve the characteristic enhancement performance of the stochastic resonance system, multi-scale noise tuning can also be further studied in the future to achieve a better blade defect detection performance for the proposed CWT-WSG stochastic resonance method.


The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important information conveyed by the hand gesture of a user. Although many works have focused on learning the benefits of thermal information from thermal cameras, most have focused on face recognition or human body detection, rather than hand gesture recognition. Additionally, the majority of the works that take advantage of multiple modalities , usually adopting simple fusion approaches between the two modalities. As both thermal sensors and visual sensors have their own shortcomings and strengths, we propose a novel joint filter-based hand gesture recognition method to simultaneously exploit the strengths and compensate the shortcomings of each. Our study is motivated by the investigation of the mutual supplementation between thermal and visual information in low feature level for the consistent representation of a hand in the presence of varying lighting conditions. Accordingly, our proposed method leverages the thermal sensor’s stability against luminance and the visual sensors textural detail, while complementing the low resolution and halo effect of thermal sensors and the weakness against illumination of visual sensors. A conventional region tracking method and a deep convolutional neural network have been leveraged to track the trajectory of a hand gesture and to recognize the hand gesture, respectively. Our experimental results show stability in recognizing a hand gesture against varying lighting conditions based on the contribution of the joint kernels of spatial adjacency and thermal range similarity. Recognizing hand gestures has been extensively studied in various research areas, as hand gestures pertain to the understanding of meaningful expressions of motion by humans . EspeciaStill, most of the conventional thermal-related methods ,15,16,17thermal guided joint filter. Our approach is motivated by the investigation of the mutual supplementation between thermal and visual information for the consistent description of a hand gesture. Our proposed method effectively tracks the trajectory of a hand, and recognizes the gesture under varying lighting conditions.In this paper, we propose a hand gesture recognition method based on a novel feature-based joint filter; namely, a In contrast to the extensive research on visual sensor-based hand gesture recognition, relatively few systems have been reported that take advantage of thermal information. Particularly, few works have proposed the complementary strategy of using thermal and visual information in the feature computing stage. The existing works rather focus on extracting features only from the thermal image, which is robust against illumination and conveys distinguishable clues from those of visual images. Larson et al.  proposedIn this paper, we design a joint filter connoting not only the spatial distance relationship between thermal and visual images, but also the range relationship in order to provide a conjunct feature representation of mutual complement between thermal and visual information. We have built a flexible structure combining thermal and visual information with respect to spatial distance and range at the same time. Based on our method, both visual and thermal sensors can be fully exploited, resulting in hand gesture tracking and recognition which operates robustly under changes in illumination. Our proposed method is applicable to the existing hand gesture tracking and recognition methods, as it directly enriches the visual image with the cooperative role of a thermal image.In this chapter, the thermal guided joint filter is presented in detail, and we describe how to benefit from the cooperative relation between the thermal sensor and the visual sensor.The spatial and temporal correspondence between the thermal image and the visual image are indispensable in order to leverage information from both modalities. First, the adjustment of the spatial difference is obtained by calibration and registration ,25,26. TOur proposed joint filter takes advantage of both the brightness information from the visual image and the temperature information from the thermal image. Moreover, the information entropy  of brighp, p, q, I, and N is a normalizing factor. The joint filtered output at position Equation , where Vr between thermal and visual images and the halo effect h of the thermal camera, per Equation (Equation ). σ1 detEquation . The smoEquation ). σ2 detEquation , where HWhen a temperature value is close to the average temperature or the variation in temperature values is large, the response of the thermal range similarity kernel is large. Additionally, when the Kullback–Leibler divergence of a visual image, Equation ). DKL [3ion  and thermal (T) images and the corresponding results of three different representations in the bright  lighting condition. One is the multiplication result   for the CNN) , Random Forests, and support vector machine (SVM)) based on hand gesture tracking, as we mainly focus on the potential feasibility of our proposed thermal-guided joint filter. As the purpose of adopting ConvNets is to verify the improvement in recognition performance based on our proposed method, the MNIST handwritten digits [CNN training tool of TensorFlow library. The TensorFlow library automatically provides an efficient backpropagation algorithm, and AdamOptimizer is used to modify the variables and minimize the loss as cross-entropy. In the Feature column in Classifying a handwritten digit becomes a more challenging problem in the context of a complex natural situation . Starting from hand-crafted features or template-matching, many recent studies leverage ConvNets  , which n digits   on Windows 10 has been utilized in our experiments.To remedy the deficiency in current works on thermo-visually calibrated data for hand gesture—especially for varying illumination environment—we collected 650 videos (image sequences) of hand gestures including ten different classes by a thermal camera and a visual camera to build a TVC-hand gesture database. The TVC-hand gesture database comprises a set of images with a checkerboard for calibration and a set of image sequences with hand gestures under various lightings. We started collecting data from a bright  lighting environment and elaborated to dark lightings with four different partial lighting directions as shown By adopting the multi-class confusion matrix , the numThe experimental results in the bright lighting condition show that all methods recognize hand gestures fairly well. The recognition using a single thermal image ranks the lowest among the compared methods. A sufficient amount of information has been included in the visual image compared to the thermal image. As this paper is putting emphasis on the experiment of the robustness against varying illumination, this normal lighting condition serves as a starting guideline.On the other hand, the experimental results in the dark lighting using back side lighting shows that the single visual image-based method suffers from the lack of visual information caused by the reduced overall brightness. The multiplication-based fusion method directly uses the visual information, so it behaves in the same way as with the single visual image-based method. The addition-based method consequently makes use of the thermal information in this lighting condition, leading to a similar result to that of the single thermal image-based method.In partial lighting conditions such as left side lighting, the single thermal image-based method and the addition-based method show performance degradations. The physical distance between the right hand and the heat source located on the left side is relatively short (in the left lighting condition), and the heat source influences the temperature of the partial side of the hand. However, even though the partially emitted light source affects the visual appearance of a hand, the initial detection region itself starts with partially received lighting. The difference between the multiplication-based fusion method and the addition-based method comes from how the two methods respond to the horizontal error in registration. Specifically in the regions where the registration errors occur, the multiplication-based fusion method responds as the intersection calculation result between thermal and visual images, while the addition-based method responds as the union calculation result between the two images, which resembles an exaggerated halo effect.For the experiment in the presence of front side lighting condition, meaningful details of texture in the visual image still remain, despite the reduced overall brightness. Our proposed joint filter-based method and the addition-based method show slightly better performances than the single visual image-based method and the multiplication-based fusion method. The single thermal image-based method ranks the last, like in the normal lighting condition. Commonly, the partial lighting from front side causes generally diminished brightness in the visual image, but it provides a concentrated effect on a hand region, resulting in an increase in hand gesture recognition performance.We use AUC and ROC curves for performance comparison as shown in We measured the times for tracking and recognition, when recognition consists of training and testing. The average tracking processing time for a frame was compared, as each gesture has a different duration. Because of the two calculations of Gaussian distributions for the allowances of spatial adjacency and thermal range similarity, our proposed filter-based method consumes additional time, as presented in In order to look into the influence of the comprehensive MNIST database, we repeat the previous experiments with exclusion of the MNIST learning data. As shown in It is possible for all tracking algorithms to fail to track the trajectory of a gesture, especially when the feature of a hand region is not sufficiently extracted because of a challenging lighting condition. That is the main reason why the recognition performances of the algorithms are all different, providing the comparative robustness against lighting conditions. Therefore, the effect on the recognition caused by the tracking failure of a gesture can be explained by the degradation of the recognition performance.Thermal information provides the importance level of a hand region so as to enable selective exploitation of the visual content from informative regions. Visual information, on the other hand, not only takes part in determining the sensitivity of the thermal range kernel, but also compensates the insufficient details of a thermal image. Our experimental results verify that the performance of the hand gesture recognition is improved by adopting our proposed joint filter, compared to those of the single modality-based methods and those of simple fusion-based methods.Compared to single modality-based methods, our method consistently represents a hand better, which leads to the improved performance in recognizing hand gestures. The thermal-guided joint filter makes use of adaptive portions of temperature values, which differ from different regions, based on the contribution of the two joint smoothness terms of spatial adjacency and thermal range similarity. Specifically, the thermal range similarity kernel provides an adaptive way to deal with different temperatures. An important note is that the information entropy of a visual region kernel internally determines the adaptiveness of thermal content. As a result, our proposed method not only performs hand gesture recognition effectively in bright lighting conditions, but also maintains the performance to a certain level in dark lighting conditions by preventing the environmental factor-caused performance degradation.Comparing the methods of simple fusion, the hand gesture recognition using our proposed thermal-guided joint filter outperforms the multiplication-based fusion and addition-based fusion methods in the presence of various illumination conditions. The multiplication-based fusion method can suppress the influence of background which commonly has lower temperature than that of body heat in bright lighting condition. Yet, it shows drawbacks in dark lighting conditions, because the foreground unwillingly gets suppressed. With a partial light source  in dark lighting conditions, only the partial region on which the light is cast remains informative. On the other hand, the addition-based method generally involves the combined characteristics of thermal and visual images at the same time in the presence of bright lighting and the partial light source conditions. In dark lighting, the results of the addition-based method are akin to the thermal image itself. However, our proposed thermal-guided joint filter-based method highlights the detailed texture in the region where body heat emits relatively higher temperature in brightly lit regions, and also more or less conserves the detail in the dark region. Even for the partial lighting condition with left or right side lighting, our proposed method effectively works. In all conditions, it substantially suppresses the regions except for that of a hand. Eventually, our proposed method effectively recognizes hand gestures against varying illumination by its flexible exploit of thermal and visual contents.Comparing the different classification methods, it is worth observing that CNN, Random Forests, and SVM generally show the similar tendency that the thermal guided joint filter provides strength against varying lighting conditions.Another limitation is caused by registration error. Looking closer to the regions which contain the registration error, the multiplication-based fusion method returns the intersection calculation result between thermal and visual images without having much textural information, while the addition-based method returns the union calculation result between the two images, which resembles an exaggerated halo effect. On the other hand, the regions with the registration error tend to be blurred by the spatial adjacency kernel when using our joint filter. Along with further systematically minimizing the registration error between the two modalities, it is also possible for a theoretical development to be studied in our joint filter to alleviate this limitation.We propose a new feature computing-based joint filter between thermal and visual information for hand gesture tracking and recognition. Visual images contain detailed texture, whereas they are vulnerable to varying lighting conditions. On the other hand, thermal images convey the meaningful cue of temperature, yet they have low resolution and a halo effect on edge regions. Thermal information has been studied in the literature to enrich visual information, but the existing approaches do not sufficiently utilize the cooperative relation between thermal and visual information in low feature level. Based on our method, the defects of the two modalities have been substantially complemented, and their merits have been fully exploited. In other words, our proposed method takes advantages of the thermal information’s stability against brightness and the visual information’s detailed texture, while complementing the low resolution and halo effect of thermal information and the visual information’s weakness against illumination. The feature of a hand gesture extracted by the proposed thermal-guided joint filter has been recognized by deep convolutional neural network. Our experimental results show that the performance of hand gesture recognition is stable against varying lighting conditions. Our method effectively recognizes hand gestures by consistently representing a hand in the presence of various lighting conditions. Based on the contribution of the joint smoothness terms of spatial adjacency and thermal range similarity, the thermal-guided joint filter provides flexible portions of temperature values differing from different regions. Still, reducing the computational time and minimizing registration error need to be studied.For future work, we first plan to further investigate the benefit of the joint filter, based on Hidden-Markov Model (HMM) or Dynamic Time Warping (DTW) for the recognition and based on more complicated gestures and complex hand shapes. Then, we plan to improve the computation efficiency of our joint filter. As our method is easy to apply to other thermal-guided tasks, we also plan to apply our method to multiple sensor-based systems for recognition of body gesture or facial gesture, and to adopt the thermal-guided joint filter to different sensors.


One of the main challenges of aerial robots navigation in indoor or GPS-denied environments is position estimation using only the available onboard sensors. This paper presents a Simultaneous Localization and Mapping (SLAM) system that remotely calculates the pose and environment map of different low-cost commercial aerial platforms, whose onboard computing capacity is usually limited. The proposed system adapts to the sensory configuration of the aerial robot, by integrating different state-of-the art SLAM methods based on vision, laser and/or inertial measurements using an Extended Kalman Filter (EKF). To do this, a minimum onboard sensory configuration is supposed, consisting of a monocular camera, an Inertial Measurement Unit (IMU) and an altimeter. It allows to improve the results of well-known monocular visual SLAM methods (LSD-SLAM and ORB-SLAM are tested and compared in this work) by solving scale ambiguity and providing additional information to the EKF. When payload and computational capabilities permit, a 2D laser sensor can be easily incorporated to the SLAM system, obtaining a local 2.5D map and a footprint estimation of the robot position that improves the 6D pose estimation through the EKF. We present some experimental results with two different commercial platforms, and validate the system by applying it to their position control. Research on autonomous aerial robots has advanced considerably in the last decades, especially in outdoor applications. Enabled by MEMS inertial sensors and GPS, Unmanned Aerial Vehicles (UAVs) that show an awesome set of flying capabilities in outdoor environments have been developed, ranging from typical flight manoeuvres , to collAlthough technological progress has made possible the development of small Micro Aerial Vehicles (MAVs) capable of operating in confined spaces, indoor navigation is still an important challenge for a number of reasons: (i) most indoor environments remain without access to external positioning systems such as GPS; (ii) the onboard computational power is restricted and (iii) the low payload capacity limits the type and number of sensors that can be used. However, there is a growing interest in indoor applications such as surveillance, disaster relief or rescue in GPS-denied environments (such as demolished or semi-collapsed buildings). In these stages is often preferable to use low-cost MAVs that can be easily replaced in case of breakage, damage or total loss.From the navigation point of view, state estimation of the six degrees of freedom (6-DoF) of the MAV (attitude and position) is the main challenge that must be tackled to achieve autonomy. The inaccuracy and high drift of MEMS inertial sensors, the limited payload for computation and sensing, and the unstable and fast dynamics of air vehicles are the major difficulties for position estimation. So far, the most robust solutions are based on external sensors, as in , where aThe problem of building a map of an unknown environment from onboard sensor data while simultaneously using the data to estimate the robot’s position is known as Simultaneous Localization and Mapping (SLAM) . Robot sAlthough there have been significant advances in developing accurate and drift-free SLAM algorithms for ground robots using different sensors and algorithms ,19, atteIn aerial navigation, sensors must be carefully chosen due to mobility, design and payload limitations. All aerial robots are endowed with an Inertial Measurement Unit (IMU) that combines accelerometers, gyroscopes and sometimes magnetometers to measure orientation, angular rate and accelerations. However, tracking the MAV position using dead reckoning based on the IMU  produces large errors and drifts , so the Due to their low weight and consumption, most commercial MAVs incorporate at least one monocular camera, so Visual SLAM (VSLAM) methods have been widely used ,23,24,25In this paper, we propose a SLAM system for GPS-denied environments that can be easily configured according to the onboard sensors of low-cost MAVs. This work is motivated by a recent research line of the RobeSafe Research Group  of the UThis paper focuses on the development of the SLAM system for the MAVs based on IMU, vision and laser sensors (depending on their availability) and fusing different state-of-the-art SLAM methods. The system robustly obtains the 6-DoF pose of the MAV within a local map of the environment. We consider a minimum sensory configuration based on a frontal monocular camera, an IMU and an altimeter. Two recent and robust monocular VSLAM techniques , so that drivers of sensors and some algorithms can be used without development.We consider a quadrotor freely moving in any direction in Throughout this document, the following notation is used to distinguish actual, measured and commanded variables: the actual variable is indicated without emphasis, while the corresponding measured variable is emphasized with “–”accent, and the commanded one with “^” accent. For example, One of the main specifications of the proposed SLAM system is that it can be configured to be implemented with different low-cost MAVs, using their onboard sensors. Usually, the software running onboard a commercial MAV is not accessible, and the control software is neither open-source nor documented in any way. Indeed, many low-cost MAVs are primarily sold as high-tech toys and only can be commanded from their own applications running on a Smartphone or a laptop. However, other platforms provide a Software Development Kit (SDK) for programming applications by sending commands through an ad-hoc wireless LAN network set up by the drone, opening the door to research works. Some of these MAVs, whose drivers are available as ROS packages to be easily used, are the Pelican and Hummingbird quadrotors of Ascending Technologies , the CraOn the other hand, most commercial MAVs incorporate an IMU, a sensor for measuring height (ultrasound and/or barometer) and at least one monocular horizontal camera. It conforms the minimal onboard sensor configuration required by our proposed SLAM system. Some drones with a limited onboard computational capacity and a greater payload can easily incorporate a light 2D laser rangefinder, which will be considered as an additional sensor that will greatly improve the results of the SLAM system under certain environment conditions.bebop_autonomy package 52], a sealtimeterh is the direct altimeter measurement, laserdx and laserdy are the laser range measurements in x and y directions, and VSLAMh, VSLAMdx and VSLAMdy are the equivalent estimates of the VSLAM system. As VSLAMh = VSLAMz, it can be seen that real-salez is directly the altimeter measurement, that is very accurate. If laser is not available, we suppose the same scale factor for the three axis, and estimate it using the altimeter. The error committed under this assumption is negligible in narrow environments, as the indoor ones to which our SLAM system has been designed. However, it could be increased in wider outdoor environments. The scale factors are updated at each iteration of the SLAM system, because they may change over time.After a study of these state-of-the-art monocular VSLAM methods , we deciIn the following subsections, we show some partial results of the VSLAM module with scale factor correction using the two chosen algorithms, so that some conclusions can be obtained about their performance when applied to aerial robots.LSD-SLAM performs a highly accurate pose estimation from direct image alignment, and obtains in real-time a 3D reconstruction of the environment as pose-graph of keyframes with associated semi-dense depth maps. We use these maps as local information for VO but, due to their computational and memory requirements, we discard these dense maps for reconstructing the environment. When laser is available, we chose to use the 2.5D laser map for planning and control because its better accuracy and lower computational requirements are more appropriate to control a fast dynamic plant as a MAV.In order to make an initial assessment about the performance of LSD-SLAM in visually adverse scenarios, we have chosen two test environments. The first one, shown in As a coarse evaluation, and using some metrics of the actual MAV trajectory, we obtain the following undesirable effects: (1) in the storage area, the lack of light causes a clear lengthening in the estimation of the second segment of the trajectory, from 4 to 5.5 m approximately; (2) by contrast, in the corridor environment the pose estimation is highly accurate in the initial room, but a clear shortening effect of around 25% is observed in the corridors (from 22 m to 16 m in the first corridor and from 16 to 12 m in the second one), due to the loss of visual features. Furthermore, LSD-SLAM is very sensitive to pure rotational movements as the one at the end of the first corridor, where an error of about 15° is observed.rgbd_dataset_freiburg1_xyz has been used. This dataset contains a video recorded from a monocular camera that describes smooth and rotation free movements that are perfect for an initial comparison of translational errors. In the results section of this paper a more thorough analysis is performed with our own datasets and benchmark.In order to have a clearer comparative of both algorithms in terms of accuracy, an external public benchmark has been applied. The chosen benchmark is “RGB-D SLAM Dataset and Benchmark”  of the CAccording with these results, ORB-SLAM is slightly more accurate than LSD-SLAM in terms of translational errors. In addition, ORB-SLAM is more robust facing pure rotational movements. This conclusion has been reached by means of the trial-and-error approach . However, the dataset used has many visual features to extract, which benefits feature-based methods as ORB-SLAM. LSD-SLAM, as direct method, achieves better results in poorly featured environments. Besides, the initialization of LSD-SLAM is self-acting from initial random values given to the depth map, while ORB-SLAM needs a specific initialization stage to build a points map of the environment before starting the tracking.As the SLAM system has been designed to execute the VSLAM module in a ground station , these execution times (with median speeds of 36.2 ms for LSD-SLAM and 32.2 ms for ORB-SLAM) are both feasible for tracking the fast dynamics of a MAV.When a 2D laser rangefinder is available on the MAV, it can be used to substantially improve the pose and map estimation under certain environment conditions. It is clear that VSLAM methods cannot work in low light conditions and their results are poor in featureless environments such as corridors. By contrast, laser sensors work properly under all light conditions and provide high-frequency and direct range measurements of the environment. So, this sensor not only provides redundancy, but also complementarity, and can be easily included in the bayesian framework of our SLAM system through the Extended Kalman Filter, as it is shown in The Scan Matcher module aligns consecutive scans from a laser to improve the MAV’s motion estimation of the EKF. Although lots of scan matching techniques have been developed and applied to SLAM for ground robots moving on flat surfaces , most ofIn this work, we adapt the HectorSLAM system to our scan matching module. It allows the system to obtain a 2.5D map and a 3-DoF estimation of the MAV’s footprint pose within the map, consisting in the  coordinates and yaw angle ѱ in the world coordinate frame {W}.The 3-DoF pose estimation is based on optimization of the alignment of beam endpoints with the map obtained so far . The endThese drawbacks will be compensated in our whole proposed SLAM system due to the addition of visual and other onboard sensors information, as well as prediction estimates, as will be shown in the results section.In order to fuse all available data, we use an EKF . This EKThe prediction model is obtained from the full motion model of the quadcopter’s flight dynamics and reaction to control commands derived in . This moOn the other hand, the influence of the sent control command 1 to K8 for the different platforms from data collected in a series of test flights. From Equations (7) to (12) we obtain the following overall state transition function:We estimated the proportional coefficients KThis model relates the state vector with the onboard measurements obtained through the navigation channel of the MAV  to the position and roll-pitch-yaw representation.When the VSLAM module successfully tracks a video frame, its 6-DoF pose estimation is transformed from the coordinate system of the front camera to the coordinate system of the drone {D}, leading to a direct observation of the drone’s pose given by:The scan matcher obtains a 3-DoF estimation of the MAV’s footprint pose, that is considered as a direct observation of the corresponding state variables, as it is shown in the following linear observation model:Once the estimated position of the MAV is provided by the EKF, a PID controller has been designed to control de movements of the MAV. A reference It allows the algorithm to drive the MAV along a series of points in the world coordinate frame {W}, so it can follow a specific trajectory, as will be shown in the results section.For controlling a quickly reacting system such as a MAV, an accurate and delay-free state estimation is required. The delays in the estimation lead to a poor control even if the estimation is correct. When using a low-cost MAV, the delay caused by the wireless communication channel must be taken into account, mainly in the transmission of the compressed camera images. The time required between the instant a frame is captured and the instant the respective calculated control signal is applied  lies between 160 ms and 300 ms. Fortunately, these delays can be easily compensated within the EKF framework.SLAM+PIDt < T.Firstly, we chose an execution period for our SLAM system that ensures that a new video frame would be processed on each iteration of the system. The VSLAM algorithm takes a median of 35 ms depending on the selected method, while the execution time of the scan matcher is about 10 ms. The EKF and PID controller execution times are much lower, of only a few milliseconds. So, an execution period of T = 50 ms has been chosen, so that video_transt and (2) a calculated control command needs to reach the MAV and take effect command_transt, depend on the bandwidth used by nearby wireless LAN networks. If the onboard processor of the MAV is open-access (as in the case of the Erle-Copter), these times can be estimated by measuring two latency values using an echo signal to the drone, and they can be updated in regular intervals while the connection is active. In our tests environment, video_transt typically lies between 50 ms and 200 ms and command_transt between 30 ms and 100 ms. In our system, these delays are rounded to an integer multiple of the SLAM execution period T, being:The amount of time that (1) a new frame captured by the frontal camera needs to be transmitted via wireless LAN to the ground station T.As the measurements of the Navdata channel are transmitted each 5 ms, and the readings and processing of the scan matcher are performed in less than 10 ms, these delays are neglected because they are lower than the execution period t = kT. The video frame processed by the VSLAM module at that time was captured by the drone in a time corresponding to n iterations before, and so it must be used to correct the estimation of the state t = kT, using the prediction stage with buffered previously sent commands and the correction stage with laser and navdata buffered previous measurements to estimate the current state In this section, we present the experimental results that validate our proposed SLAM system. Firstly, we show some practical aspects about our test bed that include the software implementation and the ground truth system used for validating the estimation results. After that, we show the experimental results obtained with the minimal configuration of the SLAM system that includes only VSLAM, IMU and an altimeter. Finally, some tests that include laser measurements are shown, comparing the effect of using the different sensors separately and fused in the proposed EKF.Two different low-cost commercial MAVs have been used in our experiments: the Bebop drone a and theOne of the major difficulties of the experimental setup was to design a ground truth system to validate the pose estimation based on an external sensor. Since a motion capture system that could obtain a reliable measurement of the actual six degrees of freedom position of the MAV was not available, we have used a simplified system that allows us to approximately estimate some coordinates of the MAV’s pose under certain assumptions. The value of this ground system is that it has not accumulative error. It is based on a monocular camera on the ceiling of the test area, as it is shown in As a disadvantage of this ground truth system, it only covers an open test area of 5 × 5 m. This is sufficient for testing the SLAM system with the minimal sensor configuration (based on VSLAM and onboard sensors), by commanding some square trajectories to the control system and comparing the estimated trajectories with the ground truthed ones. However, this test environment  is not suitable to validate the SLAM system when using also laser measurements, because walls are so far to be detected with our laser sensor. For this reason, we have used two other different scenarios to validate the complete SLAM system, coinciding with the two environments in which we performed the initial tests of the VSLAM and scan matcher modules. These are the storage area shown in In this section we include some experimental results obtained with the Bebop drone in the ground-truthed test environment. In this case we use the minimal sensor configuration of the proposed SLAM system that includes the horizontal camera, IMU and altimeter. The objective of these experiments is to compare the ability of LSD-SLAM and ORB-SLAM when they are applied to a MAV, by solving the scale ambiguity and improving the state estimation fusing the IMU and altimeter data in the EKF. In these tests, the PID controller receives a series of points that conform a rectangle of 120 cm × 60 cm as reference path, maintaining the reference orientation of the drone constant along this path.Firstly, we show the results of using all the stages of the EKF with the two different VSLAM methods in order to compare them. In order to compare both VSLAM methods with our own sensor and ground truth data, we have calculated the same translational error measures that we obtained with the external benchmark and dataset used in On the other hand, the main contribution of the proposed SLAM system is multi-sensor fusion, exploiting the typical onboard sensors of commercial drones. The usage of IMU and altimeter measurements, all them integrated into an EKF that also takes into account the commanded velocities through its prediction stage, greatly improves the estimation results of either of the VSLAM techniques.In In these cases, only the visual correction model is used in the EKF and we can see that the estimation results are very poor. Black line shows the estimated trajectory when fusing all the information in the EKF. It is easily observable that estimation results are clearly improved. In this section we show the experimental results obtained with the Erle-Copter drone using the complete multi-sensorial SLAM system . As it was stated, these experiments have been performed in confined test environments in which walls can be detected by our URG-04LX sensor  whose detection range is 4 m. In these environments we do not have a ground truth system, so we use some metrics of the followed trajectory on the XY plane to validate the estimations.Firstly, the results obtained in the storage room test environment are shown in Finally, we show in Taking these results into account, the authors of this paper are currently working on a module that automatically detects and classifies the environment using different visual and structural descriptors . This classifier will use the onboard sensors of the MAV (laser and vision) as source information. The result of the classification will be used to automatically adapt the SLAM system parameters to obtain the best results in each situation. For example, in corridor environments could be useful to compensate the shortening effect of both main sensors (laser and vision) by overestimating the movement of the MAV in the prediction stage of the EKF, and/or increasing the covariance matrixes of laser and vision observation models. Indeed, it may be desirable to completely discard VSLAM in poorly lit environments or the scan matcher estimation in wide environments out of the detection range of the laser sensor.Estimating the movement of a MAV relative to its environment is a hard challenge. In indoor and GPS-denied environments, systems that achieve similar results to classical odometric methods for ground robots have not been yet developed.The SLAM system proposed in this paper has to be evaluated in the context of the application to which it is intended. As we stated in the introduction, a small low-cost MAV with very limited onboard computational capacity moves near a ground robot with a powerful remote unit which executes the SLAM algorithms. So, execution time is not a constraint in this case. However, only typical onboard sensors can be used and communication delays must be taken into account.One of the requirements imposed to our system is sensor configurability, that we achieve using an EKF for sensor fusion. A minimal sensor configuration that adds the monocular camera to typical proprioceptive sensors  has been supposed. We have demonstrated that state-of-the-art monocular VSLAM methods can be applied to aerial robots taking advantage of onboard sensors to solve scale ambiguity and to improve the pose estimation through the EKF. So far, PTAM has been the most popular choice for implementing VSLAM in aerial robots. In this work, we have compared two more recent and precise methods as ORB-SLAM and LSD-SLAM, proposing a scale correction method based on altimeter and laser (if available) measurements. ORB-SLAM has demonstrated to be a more accurate method in visually positive environments, such as well-lit and sufficiently featured ones. It also provides better results with rotational movements, while LSD-SLAM could be a better choice in featureless environments. In any case, our SLAM proposal fuses the estimate of VSLAM with the onboard sensor measurements and a prediction model, and this is proven to provide a substantial improvement in the pose estimation. As a future work line, the VSLAM method could be automatically chosen as a function of the characteristics of the environment and the trajectory.When available, a 2D laser rangefinder is a highly suited sensor for MAVs in indoor environments, due to its fast and direct range detection, so a scan matcher module has been added to our EKF-based SLAM framework, based on the Hector Mapping algorithm. It has been demonstrated that in most environments, the fusion of visual and laser information with onboard sensors and prediction models provides a better estimation result. However, some environments as corridors are still a major challenge because both sensors (vision and laser) tend to short their length. Although this is compensated with IMU measurements and the prediction model, estimation results are still poorer than in more featured environments. As a future research, we are working on detecting and classifying the environment into some types to adjust the covariance matrixes of the EKF to minimize these undesired effects.It is important to note that we use SLAM as a framework for robust pose estimation and tracking. Obtaining a global and consistent map is not within the objectives of this work. So, depth maps or point clouds are used only as local maps for localization, and discarded once the MAV obtains a new keyframe. Furthermore, loop detection and closure methods have not been added to our system. As a future research line, we are working on integrating visual 3D maps with laser 2.5D maps to obtain wide global and consistent maps.Therefore, the importance and contributions of this paper are focused on the fusion of several sensors for solving the SLAM problem of a complex robot platform as a MAV. We have applied successful state-of-the-art algorithms in order to advance the research in the fusion and control topics. We have developed specific models for fusing the estimations of all the sensors of the MAV  and a prediction model to improve the estimates with respect to those made independently by any of the sensors. We have demonstrated that the proposed SLAM architecture can be used in real time for controlling the movements of the MAV.This paper shows a multi-sensorial SLAM system that successfully tracks the 6-DoF pose of low-cost MAVs in GPS-denied environments using a remote control station. To do this, several state-of-the art SLAM algorithms have been applied and compared. The fusion of monocular vision with laser measurements (when available) and other typical onboard sensors  by means of an EKF framework simplifies the configurability of the system. Using two commercial platforms, it has been demonstrated with experimental results that the proposed SLAM system improves the results of the baseline techniques when estimating the trajectory of the MAV in different environment conditions.


Purpose. This study examined the influence of anisotropic material for human sclera. Method. First, the individual geometry of patient-specific sclera was reproduced from a laser scan. Then, high quality finite element modeling of individual sclera was performed using a convenient automatic hexahedral mesh generator based on harmonic field and integrated with anisotropic material assignment function. Finally, comparison experiments were designed to investigate the effects of anisotropy on finite element modeling of sclera biomechanics. Results. The experimental results show that the presented approach can generate high quality anisotropic hexahedral mesh for patient-specific sclera. Conclusion. The anisotropy shows significant differences for stresses and strain distribution and careful consideration should be given to its use in biomechanical FE studies. Recently, studies have focused on understanding the properties of sclera biomechanics because the biomechanical properties of the sclera and lamina cribrosa determine the biomechanical changes of the optic papilla , 2, playThe posterior part of eyeball consists of three layers: the sclera, choroid, and retina. The sclera is the toughest of these three tissues and the retina is the softest. The posterior sclera is divided into two layers where it meets the optic nerve. The outer two-thirds form the scleral canal which transitions to the optic sheath, and the inner third forms the lamina cribrosa, an irregular mesh-like structure through which the optic nerves traverse the eyeball. Subjected to the same stress, the tangent modulus of the retina, choroid, and sclera differs by an order of magnitude, with the sclera being the highest. Thus, the sclera plays a crucial role in maintaining the shape of the eyeball. The sclera is the outer coating of the eye and is comprised of fibrous tissue, consisting almost entirely of dense bands of parallel and interlacing collagen that maintains the biomechanical properties of the sclera . StudiesAs a numerical modeling method for structure analysis suitable for complex material properties and complex boundary conditions of biological structures, finite element (FE) analysis has become a popular analytical tool in biomechanical studies.Earlier finite element models were simple linear, elastic, homogeneous, and isotropic materials . Pinsky Other studies showed that the stress borne by ocular tissues was determined by the 3D geometrical shape of the tissue, and IOP-generated stress and the corresponding response occurred in different regions of the sclera , in otheThere are two main classes of primitives that are used for 3D finite element modeling: tetrahedral and hexahedral elements, each with their own advantages and disadvantages. Tetrahedral elements are geometrically versatile and can be reliably generated by many automatic meshing algorithms, even for complex shape models. However, a good mesh of hexahedral elements can vastly reduce the number of elements and consequently reduce analysis and postprocessing times. In addition, hexahedral elements may be more suited for nonlinear analysis or situations where the alignment of elements is important to the physics of the problem, such as in computational fluid dynamics or the simulation of anisotropic materials. Many hexahedral meshing approaches have been proposed : featureSome studies have employed regular mapping hexahedral meshing to generate regular hemisphere sclera with the same shell thickness , a stratThe goal of this study was to investigate a convenient automatic hexahedral mesh generator for patient-specific sclera based on a harmonic field. The basic idea was to smoothly distribute a regular hexahedral mesh on the irregular patient-specific sclera. The harmonic field is one of the most efficient tools for smooth distribution . Its graThe remaining of this paper is organized as follows. The details of our work are explained in The starting point of our investigation was a well-reproduced individual geometry of a patient-specific sclera, produced by a high-precision laser scanner as shown in Figures As demonstrated in Instead, we investigated an ad hoc automatic hexahedral meshing for the posterior sclera, initially without the ONH region for simplicity. The basic idea was to smoothly distribute a regular hexahedral mesh on the irregular patient-specific sclera.One of the most efficient tools for smooth distribution is the harmonic field, which has no local extrema other than at the constrained vertices . If all f such that Δf = 0, where Δ is the Laplacian operator, subject to the Dirichlet boundary conditions. The standard definition of the Laplacian operator on a piecewise linear surface mesh M is the umbrella operatorj ∈ N(i) is the set of vertices adjacent to vertex i and wij is a scalar weight assigned to the edge . The standard choice for the weights wij of surface mesh is the discrete harmonic weights wij = (1/2)(cotαij + cotβij), where αij and βij are the angles opposite the edge. Assembling the vertices' function values fi into an n-vector f, the Laplacian operator can be written in matrix form Lf = 0, where the elements of the n × n matrix L are given byIn more detail, for both the outer surface and inner surface mesh of posterior sclera, we would like to construct a harmonic field Ax = b, consisting of the positive definite sparse matrix A and a right-hand-side vector b. Iterative methods such as the preconditioned conjugate gradient method can efficiently obtain the solution of the linear system.Eliminating the rows and columns corresponding to the constraint vertices by bringing them to the right-hand side yields a linear system of the form The resulting harmonic field on the outer surface of sclera is illustrated in For both the outer and inner surface of the posterior sclera, a group of even sampled seed points were placed on the end circles around the equator and were traced by the directions of the gradient vector field derived from the harmonic scalar field. These streamlines of gradient flow smoothly and converge precisely at the end circles around the ONH. At the same time, a group of isocontours for the harmonic scalar field were sampled by even scalar values. These gradient streamlines and the isocontours formed a perfect quadrilateral meshing for the outer and inner surfaces of the posterior sclera. We additionally insert a middle interpolation layer of quadrilateral mesh between the outer and inner surface. These 3 layers of quadrilateral meshes provide the sample topology and could be connected automatically to generate the complete hexahedral mesh of the sclera, as illustrated in It should be noted that although the shape of patient-specific posterior sclera is not a regular hemisphere and the ONH is not located in the center pole of the hemisphere, the resulting hexahedral mesh still shows very regular grid cell arrangement. In addition, the mesh element is adaptively distributed with higher density near the ONH region and lower density near the equator.According to previous studies , the sclx-axis and the gradient direction as the y-axis, such as the red and green axis vectors shown in Figures z-axis, indicating the sclera thickness direction.To address this challenge, we used the already existing harmonic field. It is obvious that the tangent direction of these isocontours is compatible with the trajectory of the circumferential sclera fibers and the direction of the gradient vectors is compatible with the trajectory of the meridional fibers. Specifically, we set a local material coordinate for each element cell in the model, using the circumferential direction as the Ex = 8.6 MPa, Ey = 6 MPa, and Ez = 2.5 MPa, based on previous studies [For the anisotropic elastic material constants of the sclera, we set the  studies , 26. TisE = 3.8 MPa, compatible with previous studies [To determine to what extent anisotropic characteristics affect the stress and strain distribution of sclera, we also built an isotropic sclera FE model and calculated the effective isotropic properties to take the spatial average of the elastic constants in all directions. This was accomplished using two averaging methods that provided upper and lower bounds for the transformed isotropic constants. We averaged the upper and lower bounds to obtain the constants using the isotropic elastic  studies . In addi studies , 27. We For each model, the boundary conditions constrained all three degrees of freedom at each of the nodes located at the scleral equator. Loading was simulated by applying an intraocular pressure of 30 mmHg on the inner surface of the posterior sclera.For the patient-specific sclera model in As discussed before, some studies employed regular mapping hexahedral meshing to generate regular hemisphere sclera with uniform shell thickness , which iStatistical analysis of the dihedral angle distortion from perfect hexahedral element shows that our approach produced more high quality mesh elements, as demonstrated in In the IA-FEMesh generator, the closest-point projection algorithm results in some mesh vertices that deviate from the input model, especially on the sharp feature edges. In contrast, our harmonic mesh generator requires that all mesh vertices are precisely on the input model and thus better preserves the original geometric shape of the patient-specific sclera model.In addition, an effective anisotropic material assignment procedure can be easily integrated with our mesh generator based on the same harmonic field. The IA-FEMesh generator does not allow consideration of anisotropic material.Figures Figures The important pathological feature of glaucoma is the atrophy of the optic nerve and the thinning of retinal nerve fiber layer. The maximum change of stress occurred in the posterior part of the eyeball around ONH, which means that the posterior part of sclera and ONH suffer the most stress. From a biomechanical perspective, the eyeball is a pressure vessel. IOP can produce tissue deformation, strain, and stress. ONH is closely related to visual function, but it is difficult to detect the effect of IOP on the scleral lamina cribrosa due to complicated effects of extension, compression, and shearing . The sclThe sclera is considered rigid material with high stress with low strain. The scleral tissue is complex and exhibits anisotropy , nonlineThis paper presents a novel method to generate a patient-specific hexahedral mesh of a sclera model based on harmonic field that can effectively handle the regular shape of patient-specific sclera and the significant intraregion variation in sclera thickness. The experimental results demonstrated that the method is computationally efficient. This approach can generate high quality hexahedral meshes for patient-specific sclera models and avoids element flipping and mesh angle distortions to a large extent, eliminating the need for further postsmoothing procedures to detect irregular elements and improve element quality. In addition, an anisotropic material assignment procedure can be easily integrated based on the same harmonic field.Experimental results showed that the maximum displacement of scleral tissue closest to the ONH is larger than the tissue farther away from the ONH. Additionally, the scleral tissue around the ONH is the most concentrated area of stress as intraocular pressure increases. These results are similar to previous findings in animal experiments . The bio