In the case of space-based space surveillance (SBSS), images of the target space objects captured by space-based imaging sensors usually suffer from low spatial resolution due to the extremely long distance between the target and the imaging sensor. Image super-resolution is an effective data processing operation to get informative high resolution images. In this paper, we comparably study four recent popular models for single image super-resolution based on convolutional neural networks (CNNs) with the purpose of space applications. We specially fine-tune the super-resolution models designed for natural images using simulated images of space objects, and test the performance of different CNN-based models in different conditions that are mainly considered for SBSS. Experimental results show the advantages and drawbacks of these models, which could be helpful for the choice of proper CNN-based super-resolution method to deal with image data of space objects. The Space-Based Space Surveillance (SBSS) satellite , launcheThe goal of image super-resolution (SR) is to restore a visually pleasing high-resolution (HR) image from a low-resolution (LR) input image or video sequence. HR images have higher pixel densities and finer details than LR images. Image SR has been proved to be of great significance in many applications, such as video surveillance ,22,23, uTherefore, this paper emphasizes the important role of CNN for single image SR with the purpose of space application. We comparably study four recently popular models including SRCNN   namely the external rectangle of the space object. Particularly, taking the probable impact of noise into account, we extract the external rectangle of all pixels whose gray value is above ten instead, and increase the length and width of the rectangle by 30 pixels without exceeding the image boundary. Since the four CNN-based SR models in In addition, for better and comprehensive research and comparison, 91 images proposed in Yang et al.  which weWe use peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM)  as the mL) and the mean squared error (MSE) between images. Given the ground truth X with a total of N pixels and its corresponding constructed image Peak signal-to-noise ratio is widely used in the field of image quality assessment. It is defined by the maximum possible pixel value   focuses We first train SRCNN, FSRCNN, VDSR and DRCN using T91 dataset to train three models for each network fixing the scale as 2, 3 and 4, respectively. Scale 2 means the spatial resolution of reconstructed image is twice of the input image. The larger the scale factor is, the worse the reconstructed image is, because the input image has relatively less information.The experimental results are shown in In order to meet different requirements, we often need to train multiple networks according to the reconstruction scale, because the network trained by fixing the scale as a certain number is only adapted to reconstruct this certain scale, i.e., fixed scale super-resolution. When the testing scale is different from the training scale, the reconstruction result will be worse. In addition, training several networks means multiplied number of parameters and time consuming process of training. This is a problem that cannot be ignored in practical application.In response to the problem mentioned in Because of the existence of deconvolution layer in FSRCNN, the structure of network will be different if the training HR/LR patches are in different scales. So FSRCNN cannot be trained to reconstruct different scale images using this strategy. Multiple scale super-resolution results of the other three networks trained on T91 dataset are shown in The experimental results prove that it is feasible to reconstruct the image at any scale by using this training strategy. The performance of VDSR and DRCN is relatively good. Compared with the fixed scale super-resolution results in For further comparison and analysis in the field of space objects, we perform more comprehensive experiments using BUAA-SID 1.0 dataset. We design experiments for each network to explore the performance of models trained by fixing scale or mixing scales when testing at a certain scale. That is to say, we test the reconstruction ability at three scales of every model we trained, not just the scale it is trained for. The experimental results of SRCNN, VDSR and DRCN are shown in By analyzing the experimental results of the above three methods trained by multiple scale and single scale image pairs, we can get a consistent conclusion. Using multiple scale image pairs to train the network can achieve the purpose of reconstructing HR images at any scale and the model performs well. It is only a little worse than the model whose training and testing scale is perfectly matched. While models trained by fixed scale cannot fit well when the testing scale does not match the training scale. Besides, the bigger the gap between them is, the worse the effect appears. In practical application, it is often necessary to reconstruct the space object image at any scale, but not just a fixed scale. Therefore, getting a single model which is universal to reconstruct HR images at any scale is a better choice. As for the performance of each individual network, it can be can easily see in How to train our networks is also an important factor that may affect the final results. Direct training and transfer training are two common choices. Direct training means training a randomly initialized network directly using space object training set, while transfer training in our experiments is pre-training the network parameters firstly with T91 training set, and then using space object data to fine-tune the pre-trained network. We compare the effect of these two training methods on the task of reconstruct HR images of space objects.We can see the final results of four networks in The computational complexity of the methods is also an important factor to measure their time efficiency and memory cost. We compare the times of multiplication calculation and the number of parameters of the four CNN-based networks, in order to theoretically analyze their computational complexity. Results in In practice, the space object images to be reconstructed may have different levels of noise, and the addition of noise will have a certain impact on the reconstruction effect. So it is necessary to experimentally test the anti-noise performance of the four CNN-based networks. Gaussian noise with a standard deviation (std) of 1–10 is added to the LR images of the testing set, as well as salt and pepper noise and Poisson noise. The super-resolution reconstruction results are compared with the noise-free HR image to obtain the PSNR/SSIM between the them. We can see from The analysis of the advantages and disadvantages of these four deep learning models can help choose the most suitable model for single image super-resolution of space objects.In the circumstance that we do not have enough space object images to train a deep learning model, we take a model trained by natural images as shown in Mostly previous work using single scale LR/HR trainset to train the network. According to We also design the experiments about direct learning and transfer learning in Furthermore, we analyze the computational complexity of these four deep learning models. According to At last, we analyze noise robustness of four networks. All the four methods trained by noise free data cannot process images with noise effectively. Generally, SRCNN has better noise robustness than the other three network. If the image to be reconstructed contains strong noise, a feasible approach is to first denoise the image and then construct it.Overall, SRCNN has the simplest structure, but the main body and edge of the space target are not well reconstructed by SRCNN since only three layers of SRCNN limit its ability to express and reconstruct space target image features. FSRCNN contains eight layers and uses a deconvolution layer to raise image resolution, because the first seven convolutional layers are calculated on low resolution images, FSRCNN runs faster than SRCNN while its SR performance is unremarkable. VDSR reconstructs the residual image, making it easier to study the difference between LR and HR. The edge and texture of the space target reconstructed by VDSR are clearer. DRCN uses recursive convolution networks. Its output layer takes the advantages of the information of the 3rd to 7th layers, thus, the main structure and edge details of the space target can be super-resolved best among the four CNN-based models, in both fixed scale and multiple scales. As a result, we suggest using DRCN fine-tuned from pretrained model on natural dataset as CNN-based SR model for space-based imaging sensors.To meet the needs of image super-resolution in space applications, we have comparably studied four recent popular models for single image super-resolution based on convolutional neural networks. We not only explore the difference in the performance of these models, but also find some common properties which may be more important to inspire further research. Firstly, a multiple scale training strategy has been proven as an efficient way to obtain a single model to reconstruct HR images at any scale. Solving multiple scale SR tasks with one model is more valuable in practice. Secondly, transfer training makes the network easier to converge, and has slightly better results than training the initialized network using space object data directly. Thirdly, testing results will be better if the consistency between the training set and testing set is high. It is the key to success on a particular mission, but it is also an obstacle to expansion on other tasks. Finally, noise is a killer for image super-resolution because it is also amplified during reconstruction. In general, DRCN is the best model of the four models in this paper, since DRCN performs best in super-resolution of space object images in fixed scale and multiple scale. According to this work, researchers may see the advantages and disadvantages of CNN-based super-resolution methods more clearly and then promote the development of image super-resolution in space applications.


Reliable fault diagnosis of rolling bearings is an important issue for the normal operation of many rotating machines. Information about the structure dynamics is always hidden in the vibration response of the bearings, and it is often very difficult to extract them correctly due to the nonlinear/chaotic nature of the vibration signal. This paper proposes a new feature extraction model of vibration signals for bearing fault diagnosis by employing a recently-developed concept in graph theory, the visibility graph (VG). The VG approach is used to convert the vibration signals into a binary matrix. We extract 15 VG features from the binary matrix by using the network analysis and image processing methods. The three global VG features are proposed based on the complex network theory to describe the global characteristics of the binary matrix. The 12 local VG features are proposed based on the texture analysis method of images, Gaussian Markov random fields, to describe the local characteristics of the binary matrix. The feature selection algorithm is applied to select the VG feature subsets with the best performance. Experimental results are shown for the Case Western Reserve University Bearing Data. The efficiency of the visibility graph feature model is verified by the higher diagnosis accuracy compared to the statistical and wavelet package feature model. The VG features can be used to recognize the fault of rolling bearings under variable working conditions. Nowadays, rotating machines are used widely in industrial systems, and they have become the most critical equipment in many industrial systems. In the main components of rotating machines, the fault of rolling bearings is the most frequent reason for unexpected machine breakdown and results in economic loss . TherefoSeveral vibration and acoustic measurement-based methods have been developed for the detection of defects in rolling element bearings. To the best of our knowledge, the vibration-based diagnosis method is the most widely employed because vibration signals contain a wealth of information about the structure dynamics ,5,6. TheResearchers often overlook the feature classification and arbitrarily choose the number of features . HoweverArtificial intelligence (AI) has been developed and applied to reduce the breakdowns of manufacturing systems ,21. ReseIn sum, feature extraction and selection are very important for fault diagnosis of rolling bearings because the classification performance depends on the quality of the dataset and largely the features used. Although many feature models have been developed, it is still very difficult to extract them correctly due to the nonlinear/chaotic nature of the vibration signal, so the number of features is growing now in order to recognize the fault or estimate the health of rolling bearings more precisely and efficiently .It is well known that the vibration signals are one kind of time series data acquired by sensors. Recently, a new and simple method was introduced to convert a time series to a network, called the visibility graph (VG). It has been proven that the network structure resulting from the VG approach inherits some of the properties of the time series data ; for exaThe graph analysis method has been applied in many areas including traffic systems , biologyHowever, there is little research to bridge graph analysis and signal-based fault diagnosis, and the graph analysis method is also limited. In this paper, the VG approach is used successfully to transform the vibration signals into a graph or a binary matrix. The global VG features are proposed based on the graph analysis, and the local VG features are proposed based on the texture analysis method of images. Therefore, a new feature model of vibration signals is formulated in this paper.In this paper, we present a novel nonlinear feature model that can extract the visibility graph (VG) features from the time series vibration signals. A novel signal processing method is proposed. The vibration signals of rolling bearings are converted into a binary matrix. The vibration signals are firstly analyzed from the viewpoint of graph and image processing. A new set of VG features is extracted from the vibration signals. The sequential feature selection (SFS) algorithm is applied to select the VG feature subset that can realize the best diagnosis performance. The artificial neural graph (ANN), K-nearest neighbor (KNN) ,49 and sTo motivate this work, the Case Western Reserve University (CWRU) Bearing Data  are usedData segmentation: The signals are segmented according to the sample rate and rough shaft speed to ensure each obtained sample covers several circles of signals.Feature extraction: The visibility graph method is used to convert the acceleration signals into a binary matrix. From the matrix obtained, considering the feature model of the complex graph and image, we extract the feature vector within this model. We do this for all available VG feature models and produce the VG feature pool.Feature selection: We select the optimal number of VG features based on the diagnosis performance. Multiple classifiers can be used to estimate the performance of features.Results analysis: The advantage of the VG feature model is validated by comparing the performance of the VG feature model, the statistical feature model and the wavelet package feature model.The framework to perform the bearing fault diagnosis in this paper is illustrated in i, j, k, We denote the vibration signals by The visibility graph can reflect the characteristics of the signal envelope. If the two signals can be connected according to Equation , the envi, j, Therefore, we can use Jensen’s inequality  to descrM (symmetric matrix), whose elements The obtained visibility graph can be represented by its adjacent matrix Based on the obtained adjacent matrix of the visibility graph (VGAM), firstly, we can extract visibility graph features from VGAM, for example graph density and graph index complexity; however, the graph features prefer to describe the global characteristics of VGAM rather than local properties. Meanwhile, the VGAM can also be seen as binary images , which oIn this section, the candidate VG features and the feature selection algorithm are described. We can construct VG associated with the temporal acceleration signal of rolling bearing vibration by using the method mentioned above. However, the graph cannot be directly analyzed numerically. Therefore, we gleaned the VG features from previous studies as much as possible. The graph density , degree The candidate VG feature set is composed of the global VG feature subset and the local VG feature subset. The global VG features come from graph theory. The three global VG features used in this paper include the graph density, standard deviation of node degree and graph index complexity, and the 12 local VG features used in this paper refer to the 12 GMRF parameters from the five-order traditional GMRF analysis of VGAM.(1)VG density:G denote the visibility graph transformed from vibration signals. The density G is defined as a ratio of the number of edges to the number of possible edges in the graph. Let Ddenote the VGAM, the VG density can also be represented by the VGAM, that is:The VG density (VGD) reflects the size of graph or its adjacent matrix. Let If the signal envelope is composed of much longer concave EC than convex EC, more signals can be connected, and thus, the VG density is larger, otherwise, the VG density is lower.(2)VG complexity:The VG complexity (VGC) is to signify the global complexity of the complex graph structure and adjacent matrix. Let  follows :(4)VGC=4λmax≤n−1 . It has λmax≤n−1 . Furtherλmax≤n−1 . Therefo(3)VG degree:i has to the other nodes. For the vibration signals, a smaller or larger value (compared with other points) makes them visible or invisible by many other points, which in turn causes their corresponding nodes to be or not to be the hot spot of the graph with many connections. The degree of signal i can be formulated as:The VG degree is the number of connections or edges the signal However, the VG degree is array data, so feature preprocessing is thus needed , and theD is represented by D. The Markov random field models are described by the conditional probability n-th-order symmetric neighbor set of site s. The first- to fifth-order neighbor MRF relationships are depicted in The local VG features are extracted by considering the binary matrix as the gray-level matrix of binary images. Gaussian Markov random fields (GMRF) have been shown to perform better than other models in both the classification and segmentation of textured images ,58; therr and characterizes the local properties of VGAM. The parameter set The GMRF model assumes that the Equation  can be aL denotes the length of Then, the vector In summary, three global and 12 local VG features are extracted. However, it is not certain that all the VG features are necessary for fault diagnosis because some VG features may reflect similar properties of vibration signals; therefore, we should select the necessary features that greatly contribute to the fault diagnosis.In this section, our goal is to select the necessary features that greatly contribute to the fault class diagnosis. The SFS algorithm  is appliE denote the number of testing samples and A multiple-classifier including support vector machine (SVM), k-nearest neighbor (KNN) and artificial neural network (ANN) classifier are applied to detect the bearing fault classes and produce the significance of each VG feature with regard to the updated feature subset. The diagnosis accuracy (ACC) is selected as the criteria to judge the classification performance. The accuracy (ACC) or its complement, i.e., the error rate, is the most easily understandable index. Let The significance of each VG feature http://data-acoustics.com/measurements/bearing-faults/bearing-4/).The two datasets used in this paper come from the Case Western Reserve University (CWRU) Bearing Data Centre and Intelligent Maintenance Systems (IMS) bearing run-to-failure dataset because the two datasets have been widely used in previous studies ,60,61,62http://csegroups.case.edu/bearingdatacenterpages/download-data-file). The dataset used in the experiment is acquired by the sensor at the DE when the bearing at the DE has faults because the sensor at the FE can detect the faults at the DE with less confidence, and the sampling frequency is 12 kHz.The CWRU dataset consists of time series of the vibration acceleration value  measured from a sensor mounted on the bearing housing at the drive end (DE), fan end (FE) and base of the induction motor. The vibration signals of the rolling bearings were obtained from different states—(1) normal operating conditions (NM); (2) inner race fault (IR); (3) rolling ball fault (RB); (4) outer race fault (OR)—and recorded for motor loads of 0–3 horsepower (motor speeds of 1797–1720 RPM). Fault levels ranging from 0.007 inches in diameter to 0.0210 inches in diameter were introduced separately for each fault class. More information about the data structure can be found on the website  method is applied to select the optimal VG feature subset. The classifiers including LSVM, 1-NN and MLP are applied to validate the VG feature model based on the CWRU data. The results show that the VG feature model can be used to analyze the vibration signals because the selected VG features can be taken as the input of the classifiers in order to recognize the fault precisely. The VG feature model can be used to recognize the fault class of rolling bearings under the variable working conditions, which means that our model can predict the localization of the fault and its severity without knowing the load and rotation speed clearly.Finally, the advantage of the VG feature model is verified by comparison with the statistical feature model and the wavelet package feature model, and the VG feature model performs better than the other two feature models. While the results generated using the proposed method were based on limited test data, they offer useful insights into fault diagnosis. The VG features are novel and can be selected by engineers to detect bearing faults.In the future study, we will validate the VG features by using vibration signals of other rolling bearings. The proposed method may also serve as an analytical method to detect the faults of other machines such as gear boxes using the combination of the VG feature model and other features in the future. Considering the real-time condition monitoring of rolling bearings, a resampling method of vibration signals will be proposed to reduce the computing complexity of the visibility graph in our future works.


Navigation is a precondition for ocean space vehicles to work safely in polar regions. The traditional polar algorithms employ the grid strapdown inertial navigation system (SINS) as the backbone and Doppler velocity log (DVL) output velocity as measurements to constitute the integrated navigation system, of which, however, the position errors still accumulate with time. The ultra-short baseline (USBL) position system can provide position information that can be used to improve the performance of the SINS/DVL integrated system. Therefore, a grid SINS/DVL/USBL integrated algorithm for polar navigation is proposed in this paper. In order to extend the availability of the USBL and improve integration accuracy in polar regions, the USBL observation model is established based on the relative position measurement firstly. Then, a grid SINS/DVL/USBL integrated algorithm is proposed to fuse the information of these sensors with a modified Kalman filter (MKF) dealing with the sparse USBL output. Finally, a vector fault detection method, which takes the measurements as detection objects instead of the filter, is designed to locate the measurement fault and can be employed by the centralized filter to improve the fault-tolerant. Simulation and experiment results show that the proposed grid SINS/DVL/USBL integrated navigation system can further restrain SINS errors especially the position errors effectively. Meanwhile, the vector fault detection method can detect and isolate the fault measurements of centralized filter immediately and accurately. Therefore, the proposed fault-tolerant grid SINS/DVL/USBL integrated navigation algorithm can improve the reliability and accuracy of polar navigation for ocean space application. More and more significant scientific research is carried out in the ocean of the polar regions ,2. The hThe traditional integrated navigation algorithms are based on a north-oriented geographic framework and will lose efficacy in polar regions because of the meridian convergence . A grid R, altitude angle It is known that the Doppler velocity log (DVL) is widely used to provide velocities , and theositions . InfluenThe acoustic sensors may be failed due to the operation environment , and witThe wavelet filter, sequential probability ratio test and residual n filter . But theIn this paper, a fault-tolerant grid SINS/DVL/USBL integrated navigation algorithm is proposed to improve the navigation reliability and accuracy in polar regions for ocean space application. The DVL and USBL outputs are introduced as measurements, and the USBL observation model is designed based on the grid frame with relative positions as measurements to overcome the position task. Then the grid SINS/DVL/USBL centralized filter model is proposed for polar navigation. Besides, to deal with the sparse USBL output measurements, an MKF is employed to maintain the high and stable filter update frequency and accuracy. Finally, a vector fault detection method is designed to improve the fault-tolerant of integration. In the vector method, the fault detection vector is designed based on the classified measurements. Each element of the fault detection vector represents the quality of the corresponding measurement group. Therefore, the measurement is taken as the object to be detected and isolated instead of the filter, which can improve the utilization of effective measurements and fault-tolerant of the filter, especially the centralized filter. All of the design in the above will improve the accuracy and reliability of the integrated navigation system for vehicles in polar regions, and some experiments are conducted to validate the performance of the proposed integrated navigation algorithm.The remainder sections are organized as follows. The grid SINS, which is chosen as a backbone navigation system, is employed with the relative position from the USBL and velocity from the DVL to compose an integrated system for the polar navigation. Besides, some different frames are reviewed here, i.e., the inertial frame As shown in A grid north axis , (2) and (3) can refer to .As shown in It is assumed that the sound velocity and wavelength under water are known. Then, the sound signals transmit between the USBL array and the transponder to measurement the relative position between them. As shown in As discussed above, the single navigation device cannot maintain the accuracy of polar navigation. Therefore, a polar grid SINS/DVL/USBL integrated navigation algorithm is proposed to enhance the navigation performance for vehicles. In this section, the grid SINS/DVL/USBL filter model is designed for the proposed integrated algorithm. The DVL provides the absolute velocity while the USBL provides the relative position The dynamic models of the integrated navigation system include the grid SINS, DVL and USBL models, which can be described by state equations as follows:The detail of the dynamic models will be described in the follow sub-sections, where the dynamic model of the USBL based on relative position measurements will be discussed in particular. According to , the starefer to .Besides, the DVL state is random velocity error, i.e., Therefore, the grid SINS and DVL dynamic models can be described as (7) and (8), respectively.The USBL output direction errors The range Considering of the range error After the calibration, the USBL installation error The USBL dynamic model is described as:The observation models of the grid SINS/DVL/USBL integrated navigation system include the DVL model and the USBL model, which can be described as follows:The detail of observation models will be described in the following sub-sections where the observation model of the USBL based on relative position measurements will be discussed in particular. Both the grid SINS and DVL can output vehicle velocity, which is chosen as one of the system observations.G frame can be obtained as follows [After calibration, the DVL installation error and scale factor error are compensated and neglected in this paper. The DVL output in  follows :(14)V˜DVBesides, the grid SINS output velocity can be described as:Therefore, the DVL observation model of the integrated system can be given as:The USBL position system can provide the direction and range between the USBL array and transponder expressed as The r relative to USBL array in u frame can be calculated and obtained from the grid SINS output, which is defined as e frame as a known quantity. b frame expressed in e frame and provided by the grid SINS. The position coordinate of transponder The first order total differential of (20) is written as:The error of Then refer to , and equFinally, the USBL observation model of the integrated system can be given as:The Kalman filter (KF) is a linear recursive filter algorithm based on the linear minimum variance. The KF filter runs in real time so it has numerous applications in navigation.It is supposed that there are N measurements with different update frequencies involved in the KF, and their update periods are The employed integrated filter model is an approximate linear model and the KF is a linear algorithm. If the interval between two adjacent filter updates is too long, the filter may lose its precision especially for the high dynamic range. However, the output of the USBL is one kind of sparse measurement signals which will decrease the filter update frequency. Therefore, the conventional KF is not the optimum filter algorithm for the grid SINS/DVL/USBL integrated navigation. In this section, an MKF is employed to improve the update frequency of filter in this paper. The MKF update period If the USBL output has not been updated, the filter works in the time update mode. In this case, the filter steps are expressed as:When the USBL output has been updated, the filter works in the time and measurement update mode. In this case, the filter steps are expressed as:The update frequency of the MKF will not be influenced by the uncertain and low update frequency of the USBL. The MKF can also be employed by other integrated navigation systems to effectively deal with the sparse measurement signal. To improve the fault tolerance of the polar grid SINS/DVL/USBL integrated navigation system, the fault detection vector is designed and the vector fault detection method is proposed in this paper. With the vector, the fault detection method can take the measurements as objects to detect and isolate the fault, and simultaneously improve the utilization of effective measurements, which can also improve the integration performance. In this section, a vector fault detection method based on the residual th time-step, and Firstly, the measurements as the detection object are classified based on the independence of fault’s occurrence, and can be rewritten as Then, the residual of the measurement estimation If no fault occurs, When the measurement fault occurs, the statistical characteristics of measurement noise will change, and the residual The fault detection vector is designed as:A detection flag vector According to the detection flag vector Taking the grid SINS/DVL/USBL integration as an example, By designing and employing the detection vector, the vector detection method can locate the fault measurement instead of the fault filter, and fault measurement will be isolated by the reconstruction of observation matrix but the effective measurements will be kept. Therefore, the vector detection method can also be employed by the centralized filter. Except for the residual To validate the performance of the proposed grid SINS/DVL/USBL integrated algorithm, the simulations and experiments are conducted in this section. Besides, the sea state and vehicle motion status are considered. The sea state is set as moderate condition, and the motion status includes static, motion with constant velocity, motion with constant acceleration, diving and turning a corner. The main inertial measurement unit (IMU) and motion parameters of the simulation experiment are described as follows.−5 g and the random errors of accelerometers are set up as zero-mean Gaussian white noises.Firstly, the gyroscopes drifts are set as 0.05 degrees per hour and the random errors of gyroscopes are set up as zero-mean Gaussian white noises. The accelerometer biases are set as 6 × 10Secondly, the attitude of the vehicle is set as a sine function to simulate the influence of the moderate sea state. The amplitude and period (denoted as amplitude/period) of pitch angle, roll angle and yaw angle are set as 2 and the vehicle has the maximum speed 5 m/s. The latitude and longitude of the initial position P is set as Thirdly, the acceleration of the vehicle is set as 0.2 m/sFinally, the DVL output frequency is 1 Hz, and the USBL output frequency is 0.2 Hz. The DVL velocity error is less than 0.3 m/s. The USBL range measurement error is less than 20 m, and the angle error is less than 1.5°.In this section, the simulation experiments are conducted to assess the performance of the proposed grid SINS/DVL/USBL integrated navigation algorithm, including the integrated navigation algorithm and the vector fault detection method. There are three polar integrated navigation algorithms discussed in this section, i.e., the traditional grid SINS/DVL integrated algorithm, the traditional grid SINS/DVL/USBL integrated algorithm 1 and the proposed grid SINS/DVL/USBL integrated algorithm 2. The traditional grid SINS/DVL/USBL 1 employs the absolute position as measurements, and the proposed grid SINS/DVL/USBL 2 employs the relative position as measurements. All the three integrated navigation algorithms choose the MKF as the filter algorithm.The navigation errors of the grid SINS and three integrated algorithms are depicted in As shown in In this section, the performance of the vector fault detection method and the fault-tolerant grid SINS/DVL/USBL integration are discussed. The centralized MKF is the filter algorithm and the residual vector R from the USBL, the direction angles R. The angle flag is the fault detection flag of the USBL measurement As description in the R contains mutant fault but As the flag lines showing in The navigation errors in this experiment are shown in As shown by the blue curves in As a whole, the proposed vector fault detection method chooses the measurement as objects to locate the fault by employing the detection vector, so it also can detect and locate the fault measurement effectively when employed by the centralized filter. With the help of the vector fault detection method, the grid SINS/DVL/USBL integrated navigation algorithm can improve the fault tolerance and maintain the accuracy when some measurement mutant fault occurs.In this section, the semi-physical experiments are conducted to further validate the performance of the proposed algorithm due to the geographic restriction of the polar experiment. The semi-physical experiments are designed based on two principal factors which mainly affect the performance of the standard polar and ocean navigation strategies, i.e., the amplification of navigation errors caused by the meridian convergence and the navigation precision mainly influenced by the IMU measurement errors. Therefore, the semi-physical experiment, which can simulate the polar meridian convergence and simultaneously contain the actual IMU output errors, will be more realistic to verify the performance of the proposed navigation algorithm. Moreover, compared to the simulation experiments, more standard navigation strategy and experiment condition are involved in the semi-physical experiment . The GNSThe semi-physical experiment contains three parts, i.e., the extraction of sensor errors and generation of the semi-physical experiment data, the performance validation of the integrated algorithms, and the performance validation of the fault detection method, which will be discussed in the following subsections.In this subsection, a set of polar semi-physical experiment data that contain the actual IMU output errors and ideal IMU outputs is generated and will be employed to verify the performance of the algorithm. In this experiment, the ideal IMU outputs in polar regions can be generated by the trajectory generator, and the key point to get the semi-physical experiment data is to get the actual IMU output errors.The signal flow to get the actual IMU output errors, i.e., In the turntable experiment, an IMU is installed on a high-precision three-axis turntable as shown in During the turntable experiment, the IMU output, i.e., Then the actual IMU output errors are obtained as:The Meanwhile, the trajectory generator is designed to simulate the polar region geographic environment which leads to the meridian convergence. The ocean space vehicle trajectory in polar regions and the corresponding ideal IMU outputs, i.e., Therefore, by adding the actual IMU errors, i.e., The standard navigation strategies for ocean vehicles include the SINS/DVL integration, SINS/DVL/GNSS integration and SINS/DVL/USBL integration, in which the DVL provides the velocity and the GNSS and USBL provide the absolute position. The grid SINS/DVL/USBL integrated navigation algorithm proposed in this paper employs the DVL to provide velocity and the USBL to provide the relative position as measurements. There are four integrated navigation algorithms discussed in this section including the standard strategies and the proposed novel strategy:Algorithm 1: the standard grid SINS/DVL integrated navigation algorithm;Algorithm 2: the standard grid SINS/DVL/GNSS (absolute position measurement) integrated navigation algorithm;Algorithm 3: the standard grid SINS/DVL/USBL1 (absolute position measurement) integrated navigation algorithm;Algorithm 4: the proposed grid SINS/DVL/USBL2 (relative position measurement) integrated navigation algorithm.As shown in The corresponding semi-physical sensor output data obtained in The navigation errors of different algorithms in the semi-physical experiment are depicted in As shown in The proposed vector fault detection method and the fault-tolerant grid SINS/DVL/USBL integrated algorithm are discussed in this section. To validate the performance of the vector fault detection method, some mutant constant errors are added into the measurements. Compared to the simulation experiments, the ocean space vehicle trajectory and the corresponding semi-physical sensor output data obtained in The measurement errors and the fault detection flags are shown in As shown in Owing to the detection vector, the fault measurements can be isolated by the reconstruction of filter, and as the red lines showing in Though the R measurements and angle measurements, i.e., Hence, the proposed vector fault detection method can be employed by the centralized filter to detect and locate the fault measurements and improve the grid SINS/DVL/USBL integrated algorithm’s reliability and fault-tolerant to maintain the requirement of ocean space vehicles.A fault-tolerant grid SINS/DVL/USBL integrated navigation algorithm based on the centralized filter and relative position information is proposed to improve the performance of ocean space navigation systems in polar regions. Firstly, the USBL and DVL output are employed as the measurements and the filter observation model of the USBL is proposed based on the relative position measurements to improve the USBL availability. Then, the grid SINS/DVL/USBL filter model is designed to improve the navigation accuracy. Moreover, considering of the sparse measurement from the USBL, the MKF is employed to improve the filter update frequency and estimate accuracy. Finally, the vector fault detection method, which takes the measurements as detection object instead of the traditional filter, is proposed to detect and isolate the measurement fault of centralized filter. Simulations and semi-physical experiment results show that the vector fault detection method can detect and isolate the measurement fault of centralized filter effectively, and together with the novel fault detection method, the grid SINS/DVL/USBL integrated algorithm can provide high-precision and high-reliable navigation information for ocean space vehicles to operate normally and safely in polar regions. Some important further developments are planned for the future. Firstly, if the conditions allow, a polar experiment is expected, and more kinds of external measurements will be employed to improve the integrated navigation accuracy and reliability.


The next generation of Intelligent Transportation Systems (ITS) will strongly rely on a high level of detail and coverage in traffic data acquisition. Beyond aggregated traffic parameters like the flux, mean speed, and density used in macroscopic traffic analysis, a continuous location estimation of individual vehicles on a microscopic scale will be required. On the infrastructure side, several sensor techniques exist today that are able to record the data of individual vehicles at a cross-section, such as static radar detectors, laser scanners, or computer vision systems. In order to record the position data of individual vehicles over longer sections, the use of multiple sensors along the road with suitable synchronization and data fusion methods could be adopted. This paper presents appropriate methods considering realistic scale and accuracy conditions of the original data acquisition. Datasets consisting of a timestamp and a speed for each individual vehicle are used as input data. As a first step, a closed formulation for a sensor offset estimation algorithm with simultaneous vehicle registration is presented. Based on this initial step, the datasets are fused to reconstruct microscopic traffic data using quintic Beziér curves. With the derived trajectories, the dependency of the results on the accuracy of the individual sensors is thoroughly investigated. This method enhances the usability of common cross-section-based sensors by enabling the deriving of non-linear vehicle trajectories without the necessity of precise prior synchronization. As the future of road transportation is being shaped around the idea of autonomous mobility, new methods of data acquisition and processing are being developed. Especially in the field of driving assistance systems, there is much current research on detecting and localizing vehicles with many different sensors like radar, LiDAR, computer vision, and acoustics. On the other side, infrastructure-based Intelligent Transportation Systems (ITS) still need further development to exploit the full potential of the already existing sensors. This especially means increasing the level of detail reached with current ITS techniques, which often only deliver aggregated traffic data consisting of datasets with a time resolution of minutes covering traffic parameters like traffic flux (vehicles/hour), density (vehicles/km), and speed (km/hour). This kind of data is a limiting factor for comprehensive analysis on the interaction between individual vehicles. For such detailed analysis, traffic data are required on a microscopic scale, which includes the quasi-continuous trajectory of every vehicle on the road.The analysis of vehicle-to-vehicle interactions is indispensable for a detailed traffic safety analysis also covering risky situations between vehicles beyond only counting traffic accidents . Many ofSeveral alternative techniques exist today, capable of acquiring real microscopic traffic data. Of course, one of the most straightforward methods is equipping a large number of vehicles with high-precision Global Navigation Satellite System (GNSS) sensors and recording data from each vehicle . Such meA better approach to recording traffic data with a large coverage is the use of infrastructure-based sensors. A very common technique is based on inductive loops ,12. TheyWhile there are a few studies showing how radar-  and laseAn additional difficulty in using already deployed CCTV cameras for computer vision methods is measuring the exact sensor distance or accurately synchronizing the video streams. In our research project “Highly automated tunnel surveillance for catastrophe management and regular operation” (AUTUKAR) regarding automatic video analysis in tunnel surveillance systems, we can only rely on the existing planning documents of the tunnel to determine distances between cameras, which often do not have the desired accuracy. On the other hand, when using real-time video analysis based on video streams, network bandwidth limitation means that a very accurate time synchronization between the streams cannot be guaranteed.Finally, using cameras is also often restricted by data privacy, a very delicate subject especially in European countries, which often makes application of these sensors impossible. Using sensors that are 100% compliant with privacy laws makes a huge difference in European countries.Spatio-temporal offset estimation when measuring the distances between sensors or an exact clock synchronization is not possibleVehicle registration without any specific identification like number platesReconstruction of vehicle trajectories with acceleration/deceleration maneuvers only based on cross-section recordings.In this paper, we present a method of spatio-temporal synchronization (offset estimation) of sensors and an appropriate fusion technique to reconstruct microscopic traffic data. The sensors record limited traffic data, which are covered by common detection sensors at fixed cross-sections. Firstly, this results in an enhancement that enables radar and loop detectors to generate area-based microscopic traffic data, even when a precise sensor synchronization is not possible. Secondly, it allows the filling of gaps of data already covering a limited road section, thus reducing the sensor density requirement of computer vision-based systems. The method solves the following problems simultaneously:Thus, the presented work has a great impact by enhancing the usability of cross-section-based vehicle detectors by enabling them to acquire microscopic traffic data based only on unsynchronized records of detected timestamps and respective speeds. As many of the cross-section-based sensors do not require recording images or videos, using this method enables the acquisition of vehicle trajectories without any privacy issues. The method also greatly reduces the financial, organizational, and maintenance effort for camera-based acquisition with the ability of filling interim gaps between covered road sections, thus leading to less sensors required to cover a given length. With the proposed method, simple and cost-effective sensors can be used for a detailed safety analysis. If such sensors are deployed along highway exit lanes, this method can be adopted to derive parameters such as individual deceleration rates along the exit lane and to support decision making in regards to appropriate traffic harmonization methods. Furthermore, surrogate safety indicators such as space headways, time-to-collision, or minimal deceleration rates to avoid a crash can be derived, to better understand the evolution of these potential conflicts over time. In addition to presenting the basic principles of the method, we will also analyze the limitations of the method with respect to traffic volume, cross-section distance and detection accuracy.The remainder of the paper is organized as follows: As formulated in the previous section, we have three different problems to solve: the coordinate systems of the sensor data are not completely synchronized; there is no known matching between individual vehicles; and the dynamic behavior of the vehicles between the two measured cross-sections, which we call gap sections, are unknown. We solved the first two problems simultaneously by making an assumption about the vehicle dynamics, namely that the sensors are close enough and the traffic is fluent enough, so that on average, the drivers make limited deceleration or acceleration maneuvers. This assumption is feasible for the considered orders of magnitude for the distances considered in this work of 100 m–200 m, because in fluent traffic, such sections are covered within several seconds, which restricts the amount of maneuvers. We must also note that it is of great interest to reconstruct microscopic data of fluent traffic as this state leads to high-risk interactions between vehicles due to large speed differences. For non-fluent traffic, analytically determining more complex dynamic behaviors between cross-sections is impossible without the means of driving behavior modeling. Additionally, low traffic speeds also lead to less safety-critical interaction, so from the point of view of traffic data analysis on a microscopic scale, this state is less of an interest than fluent traffic.i at time t, We formulate the assumption by describing the vehicles’ movement along the street axis (in one lane) as:t.From Equation , it can Estimation step (E-step): Find an estimate for the complete data sufficient statistics.Likelihood Maximization step (M-step): Determine the parameters of the distributions based on the estimated data.Based on the assumption formulated above, we treated the vehicle matching as a probability density estimation problem. To solve it, we applied the expectation maximization algorithm described in . The algM vehicles with their respective speeds from the second sensor form linear Gaussian kernels, while the N vehicles and speeds from the first sensor form the data upon which we are trying to find the distribution parameters. The mean of the individual Gaussian models is defined by the mean speed between a vehicle pair as a linear trajectory, as defined in . If the number of zeros, which are unique in the respective row and column, is equal to the smallest of the two matrix dimensions (number of vehicles in first or second sensor), an optimal matching has been found.Additional reduction: Reduce all non-covered elements by the minimal non-covered value and add that value to all elements covered by both horizontal and vertical lines. Continue with line covering again.Now, after we found the sensor offsets and the soft matching probabilities lgorithm  to find The result of the assignment problem is a set of pairs, where for each vehicle of the first sensor, a vehicle of the second sensor is matched. The assignment is only an interim step towards reconstructing the trajectories as it still assigns the vehicle pairs based on a linear trajectory based on mean speed between the vehicle pairs. However, the assignment is very robust, because the real trajectories of the vehicles are similar enough to the linear ones in order for the matching to be successful.We also have to note at this point that the resulting assignments also hold assignments of outliers, as they have also been translated according to the EM-algorithm. Thus, both false positives and false negatives in the data of both sensors need to be considered. We applied a threshold of After finding appropriate assignments, we reconstructed the trajectory of the vehicles between the two sensors using curves, which can be parametrized to connect the two points while also following the detected speeds profile. In other words, the connecting curves r curves . They han. u increases within the interval n can be deduced to:The general definition of Bézier curves is:B(u)=∑i=0u to zero and one using the derivatives and equating it with the values defined by the input data. In our case, we had six conditions in total  in order to analyze specifically their effect on the results. Finally, the underlying microscopic vehicle data can also be generated and used as a perfect reference. In comparison, real reference data measured in vehicles can never be completely errorless.d. Then, for each vehicle, we generated a normally-distributed acceleration value with mean We based the synthetic data generation on the measurements of the first radar device, as it recorded all passing vehicles on three lanes and thus delivered the largest and densest data volume. We extracted the data of the morning rush-hour and overcame the limitation of time resolution by adding a uniformly-distributed number of milliseconds between zero and 1000 to each recorded entry. We kept the recorded speed values and chose a second cross-section at a specified distance Offset reconstruction error: the deviation between the sensor offset reference value and the reconstructed one. Depending on spatial or temporal optimization, this involves the location or time offset in meters or seconds.Trajectory reconstruction error: the deviation between reference vehicle trajectories and the reconstructed ones measured as a root-mean-squared error in meters.Matching sensitivity: the recall value between the correctly-matched pairs after reconstruction and the pair of the reference dataset.Matching relevance: the positive predictive value between the correctly-matched pairs after reconstruction and all the matched pairs.Number of iterations: the number of iterations the EM-algorithm requires to converge.The considered performance metrics are defined as follows:In this first validation step, we considered the case where the timestamp and speed of the vehicles can be recorded without error at the exact cross-section of the sensor. We varied the number of vehicles from 5–400 using a sensor distance of 100 m. Additionally, we varied the distance between the sensors from 50–150 m using 200 vehicles. We used 50 runs for each setting, resulting in a total of 4000 runs for the analysis of the vehicle count and 500 runs for the analysis of sensor distance. We first used only spatial optimization. The results showed that for all the settings, our algorithm was able to reconstruct both the sensor offsets and the vehicle trajectories perfectly, meaning that all appropriate metrics had zero mean. The only metric varying was the number of iterations for convergence, which seemed to depend on the number of vehicles used, but not on the distance between the sensors . We alsoNext, we analyzed the effect of varying the detection location. This constituted a type of error resulting from the fact that a sensor will not detect a vehicle at the exact same cross-section as it is installed, although we treated the data as such. In most cases, each vehicle will be detected at an individual cross-section depending on the physical characteristics, which influence the detection. An example would be the reflectance of the vehicle chassis in the case of a radar sensor, which will influence how early a vehicle can be detected. We used a total number of 200 simulated vehicle trajectories for this step, and 75 m, 100 m, and 125 m were used for the sensor distance. Note that we were still assuming that the detector made no error in speed measurement and started with the sensors being perfectly time-synchronized, so we only applied optimization in the space dimension.From In contrast to the sensor offset estimation, the mean RMS error of the vehicle trajectories only depended on the variance of the detection location and remained constant with increasing number of vehicles see . The linWe go on to examine the performance of the spatio-temporal offset estimation. Obtaining a time offset of zero is of course expected as we did not specifically add any time shift to the dataset. When looking at the results, the error values showed that while the time offset estimation mean was indeed at zero, the location estimation error significantly increased. Because the performance showed very limited dependence on the sensor distance used, we will limit our further analysis to a distance of 100 m.One other important source of error when recording traffic data consists of measuring the wrong speed of the vehicles. Although many of the vehicle detectors used in enforcement applications have a very high-quality standard and measurement accuracy, this error can never be completely neglected. Moreover, detectors developed for simpler applications like gathering traffic statistics most probably have lower accuracy standards. Thus, it is important to examine the influence of this error. In order to reduce the complexity of the results, we will neglect variations in the detection location. When not explicitly stated, the number of vehicles used for the optimization will be 200.In the left plot of Another limitation of the sensors we used was the level of quantization of the recorded values. Especially, the coarse resolution of time was a limiting factor when using our radar detectors, as they were only able to record timestamps up to a resolution of one second. This led to the necessity of examining the influence of the quantization on the results using our synthetic dataset. We conducted this experiment by again using 200 vehicles and three sensor distance values of 50 m, 100 m, and 150 m, as we expected that the distance could change the outcome. We neglected all previously-discussed errors by setting them to zero mean and variation. We applied a quantization to the time values of both sensors. We implemented the quantization by means of a factor value iterating from 1–10, which defined the partition of a second to which the recorded values were rounded. With a factor of 10, the values were rounded to a tenth of a second, while with a factor of one, the values were discretized to seconds just as the case in our real-world measurement.As we can see from We have already discussed many possible measurement errors given a vehicle has correctly been detected. Nevertheless, depending on the circumstances of the recording and on the traffic density, the detectors will also record to some extent false negative and positive detections. For the presented algorithm, the false negatives of one sensor led to outliers in the dataset of the other sensor. Thus, in this validation step, we first examined how these outliers reduced the resulting accuracies. For the analysis, we varied a percentage of false negative detections for both sensors. We simply removed detections from both sensors at random accordingly to the false negative rate chosen. For each value, we again applied many runs with our algorithm, in each run generating new vehicle trajectories and deleting some of them as described. In A very interesting effect can be seen in Similar to the false negative detections, we also simulated false positives. From the recorded radar data, it can be observed that false positive detections occurred mostly in the presence of vehicles. It did for example occasionally happen that a large vehicle like a bus or a truck was wrongfully detected as two small vehicles. In the absence of traffic, false positive detections were extremely rare. To simulate this fact, we specifically generated false positive detections at both sensors around the already existing detections. Thus, we extracted a number of vehicles according to the chosen false positive rate and copied the data of those vehicles also adding a random time to the original timestamp with a minimum and maximum absolute difference of 2 s and 3 s, respectively. In order to examine the combination between both false negatives and positives, we additionally set the false negative rate to 0, 0.1 and 0.2. The sensor distance was fixed to 100 m. In addition to validation with synthetic data, in this subsection, we want to demonstrate the capabilities of the proposed methods using real measurements at two cross-sections. In order to be able to validate device offset reconstruction, we used high-precision RTK-GNSS sensors for two purposes: to record the exact location of the radar devices with a precision of a few centimeters and to record the vehicles’ paths along the curve. We were able to record the device cross-sections with a high precision with static GNSS data recording at the exact location of the radar devices. The GNSS data recorded when the sensor was mounted on the vehicle showed a high amount of jitter due to driving beneath the bridge seen in For the validation of the reconstruction of microscopic traffic data, we drove with the equipped vehicle 33 times within three hours. For this period of time, we recorded both radar data and vehicle paths from the distance measurement device. As we saw in the previous subsection, our proposed algorithm was unable to converge to a good result when there were more than 25% outliers in the recorded data. When looking at We used the EM algorithm with both spatial and temporal optimization, as we could not guarantee a very precise time synchronization between the radar devices. The calculated offset after convergence was −0.63 s and 69.7 m between Radar 2 and 4, which matched very well with our measured reference. This indeed was in accordance with the results from the synthetic data validation. As the vehicles appeared randomly within the time windows of 1 s, the spatio-temporal offset error balanced out over many matched vehicle pairs. In other words, for a number of vehicle pairs that would correct the temporal offset towards the lower second value, their was a similar number of pairs that corrected the offset towards the higher second value. The results also delivered 274 matches between the two cross-sections. Moreover, from the 33 passes recorded with our equipped vehicle, we could successfully find 25 passes in the resulting matches. This paper presented a method for spatio-temporal synchronization and microscopic traffic data reconstruction from cross-section-based detection devices. The required input data consisted of the timestamp and speed of individual vehicles. The three main problems that were been overcome were: unknown offset of the recording devices, unknown vehicle matches, and unknown vehicle trajectories. The proposed methods were based on the assumption of limited vehicle dynamics between the recording devices. It has been shown how under these assumptions, the iterative EM-algorithm can be used to register the individual detections of the devices and estimate the previously unknown offset simultaneously. In a further step, the registered datasets were used to reconstruct trajectory data between the devices by means of quintic Beziér curves.After presenting the basic methodology, a comprehensive validation was presented to demonstrate the capabilities of the method. Using synthetically-generated datasets, different error possibilities, like location variance, speed measurement error, and outliers in the input data, were discussed, and their effect on the outcome of the method was thoroughly investigated. It was shown that errors in the offset estimation can be reduced by increasing the number of vehicles in the datasets, which led to a higher computational load. Thus, one must find a trade-off between these two parameters. Additionally, on the synthetic datasets, a validation based on empirical data was also conducted, where was shown that the algorithm not only precisely estimated the offsets of the used devices, but also accurately reconstructed vehicle trajectories.Based on this work, further validation of the matching performance could be conducted by precisely recording individual vehicle matches. This could be done by using detectors based on number plate recognition. Furthermore, the applicability of the proposed methods should also be demonstrated by using other types of cross-section detectors like inductive loops and computer vision systems. In order to provide more information of the vehicle passes, the method could be applied in a multidimensional space to also reconstruct the lateral movements of vehicles rather than only longitudinal trajectories. Additionally, an iterative version of the method could be developed to be able to optimize continuously the parameters of the cross-sections with a continuously-growing data volume. Finally, a very interesting and promising extension of this work would be a thorough analysis on the possibility of overcoming the limitations regarding dynamic behavior. If the propagation of shock waves in congested traffic is of interest, many dynamic maneuvers between cross-sections need to be taken into account. This could be done by incorporating driver modeling and microscopic traffic simulation in combination with data on the vehicle lengths in order to derive complex vehicle interactions.


The Mediterranean region is affected by considerable daily and seasonal temperature variations due to intense solar radiation. In mid-seasons, thermal excursions can exceed tens of degrees thus influencing the long-term behaviour of jointed rock masses acting as a preparatory factor for rock slope instabilities. In order to evaluate the thermal response of a densely jointed rock-block, monitoring has been in operation since 2016 by direct and remote sensing techniques in an abandoned quarry in Acuto . Monthly InfraRed Thermographic (IRT) surveys were carried out on its exposed faces and along sections of interest across monitored main joints. The results highlight the daily and seasonal cyclical behaviour, constraining amplitudes and rates of heating and cooling phases. The temperature time-series revealed the effect of sun radiation and exposure on thermal response of the rock-block, which mainly depends on the seasonal conditions. The influence of opened joints in the heat propagation is revealed by the differential heating experienced across it, which was verified under 1D and 2D analysis. IRT has proved to be a valid monitoring technique in supporting traditional approaches, for the definition of the surficial temperature distribution on rock masses or stone building materials. Small-scale landslides as falls, slides or topplings are the most common mechanism involving jointed rock masses in correspondence with cliff or steep-slopes ,2,3,4. TMost of the InfraRed Thermographic (IRT) applications in Earth Sciences concern the monitoring of volcanic eruption  and of vThermal forcing which continuously acts on jointed rock masses can induce permanent deformations. Thermo-mechanical effects are indeed referred in literature to the results of cycles of expansion and contraction in the rock matrix, which can operate over long time, inducing cumulative stress concentration along joints and promoting irreversible strains . CyclicaThe comprehension of thermo-mechanical effects in jointed rock masses requires the definition of both the amplitude of thermal action and distribution of near-surface temperatures , which aIn order to quantify the effect of intense solar radiation on discontinuous rock mass and evaluate the predisposition of the latter to thermally-induced strains, a two-year thermal monitoring was monthly carried out on a selected test-site where an intensely jointed rock-block is exposed to direct solar radiation , throughFor the considered case studies, the control of rock face orientation (aspect) and the role of joints in heat propagation within the rock mass have been studied by analysing daily and seasonal temperatures variations in the rock mass on both its surface and in depth, adopting 1D and 2D analyses. A conceptual model for the development and progression of thermally induced strains has been also proposed. Basing on this model, an experimental approach for estimating the predisposition of a rock-wall in a quarry area to thermo-mechanically driven rock-blocks instability have been tested.The Acuto test-site is located NE of Acuto municipality on Mt. Ernici ridge , where Mesozoic limestones (mudstone or floatstone with rudists) widely outcrops . An abanThe N20E trending quarry wall is exposed to intense solar radiation and predisposed to evolve through gravitational instabilities by rock fall, sliding or toppling mechanisms. Moreover, the quarry rock wall is characterised by heights ranging from few meters up to 50 m and a length of hundreds of meters. The rock mass is also intensely jointed: geomechanical scanlines and remote surveys using total station have been performed on the quarry wall and five joint sets were distinguished , a S0  showed a thermal conductivity value of 2.09 ± 0.01 W/mK.3 sized was selected as main target for the installation of a monitoring system in order to understand its thermal response under direct solar radiation and the heat propagation in 2D conditions , installed at the centre of the monitored rock-block, in the front face at a depth of 8 cm from the surface, to measure the inner rock mass temperature and of a weather station, installed at the top of the slope, equipped with air-thermometer, hygrometer, rain-gage and anemometer for wind speed and wind direction. All sensors are set to acquire data every minute with storage in a local data-logger CR1000 Campbell Scientific, equipped with 24 acquisition channels. The data are transmitted to a local server, every 4 h, by a GPRS wireless connection system that enables remote control of datasets.The multi-parametric monitoring system has been implemented with seasonal remote surveying via InfraRed Thermography (IRT) in order to evaluate the role of both solar radiation and rock-block exposure on surficial rock temperature. IR-imaging represents a reliable, rapid and non-destructive technique that allows to derive temperature distribution at the rock-air interface, quantifying the effect of solar radiation on heat propagation over a discontinuous medium ,58.T) profile in depth (x) for a semi-infinite solid:Rock masses are exposed to daily and seasonal temperature variations at its surface, which propagate in depth according to the Carslaw and Jaeger  model, wdepth x, . This fa® 885-1 thermal camera . With the aim to validate the IRT technique and verify the accuracy of the obtained results, a comparison between rock temperature in depth (derived by rock thermometer) and on its surface (derived by IRT) was performed.In order to understand the amplitude of thermal perturbation and assess the role of cyclical forcing in the stability of rock masses, a combined use of direct and indirect sensing techniques has been adopted availing of the type-K thermocouple (hereafter named rock thermometer) and a TESTOThermal monitoring was carried out with a seasonal cadence over two years, trying to constrain the impact of sunlight radiation and the impact of opened joints on heat propagation across a jointed heterogeneous medium. The thermal monitoring was achieved selecting days presenting the same weather conditions and choosing preferably sunny days without cloud cover, to minimise the daily variability and highlight the seasonal signals. It worth note that all the timing hereafter discussed are referred to solar times.The instrumented rock-block, protruded with respect to the rock wall behind, was selected as the target of the IRT analyses , which wThe monitoring was performed by means of standard acquisition of thermal images over the diurnal hours (from 8:00 to 17:00) with a regular time-interval of 30 min, framing the block from a distance of about 5 m apart from both faces of the monitored rock-block. Such a distance is small enough to guarantee a spatial resolution of about 8.5 mm and to separately detect the largest set of joints. The total acquisition was designed to avoid perturbative effects related to distance and angularity of the target from the point of view ,61 as weEmissivity values of rock mass was evaluated a priori adopting a standard reference method with the use of reference material with a known emissivity (ISO 18434-1). Field measurements of reflected apparent temperature were carried out using a reflector method, according to the ISO standard, availing of aluminium foil and setting an emissivity equals to one and a distance between the camera and the monitored object to zero.®, which was derived for each IR-measure by weather station acquisition.For each thermal image, environmental controlling factor such as air temperature and humidity  were alsA quantitative analysis of thermal images was carried out through the extraction of thermal time-series from some rock mass zones selected on both rock-block faces. To this aim, thermal images were converted to thermal matrixes from .csv to .ASCII file format by the proprietary software and a Phyton compiled script. Thermal time-series were extracted by selected control points on the georeferenced images for different zones of the rock mass. To guarantee a statistical significance of the recorded thermal values and filter local temperature difference resulting by surface irregularities and asperities, the derived temperatures were averaged by measuring on the nine neighbour control points in the two rosettes . The IRT approach was also tested for the monitoring of temperature distribution across main open joints located on the back face of the target rock-block , several isothermal bands are identified describing an inhomogeneous distribution of rock temperature on both faces. The presence of these isothermal bands is attributable to differential heating affecting the rock-block because of an irregular geometric shape of both rock faces and angle of incidence of the sunrays, which changes according to different positions of sun during its daily solar path a. When tThe timing of the temperature maximum appears to be shifted in the daytime with respect to seasons, evidencing anticipated timing of the temperature peak on front face and a delayed thermal maximum on the back face, moving from Winter to Summer  and inner rock temperature at a depth of 8 cm was achieved for every monitored season during the period 2016–2018.As shown in F on The IRT monitoring campaigns during the local temperate Winter seasons a, when tDuring the Summer c, when dA comparison between temperatures obtained on surface and in depth by the two techniques (IRT and Thermometry) can be performed only for the frontal face. In Summer, the intense radiation warms the block for all the afternoon, delaying the thermal inversion to the night hours, as evidenced by the position of the IRT temperature curves . After this peak, in fact, the rock-block is shadowed and the surficial temperature gradually decreases.The thermal excursion in cooling derived for the front face confirms the higher values in the half seasons, which reached values of 3.75 °C in the Spring and of 5.26 °C in the Autumn .Thermal excursion over Summer season is just 1.95 °C, while in Winter season it assumes maximum values of 1.60 °C. Also at the front face, the cooling rate of the surficial rock surface was calculated monthly and then averaged to return a seasonal value, taking into account a different time interval between 12:00 and 15:00 , using the same equation presented before.The cooling rates for the mid-seasons are higher than ones derived in Summer and Winter (0.65 °C/h and of 0.54 °C/h respectively), as previously observed for the back face. Nevertheless, their absolute values are sensibly lower than ones obtained for the back face, due to the different timing during the cooling rate has been calculated. Because of the occurrence of the cooling of the front face in a time-lapse between 12:00–15:00, when a contemporary increase of air temperature takes place, a direct comparison between the rates derived by the two faces cannot be carried out.Specific seasonal values of surficial rock cooling rates and thermal excursions in the cooling ramp for the front face of the rock-block are summarised in The IRT monitoring is also valuable to derive the annual thermal behaviour over the four seasons, highlighting the difference between warm and cold periods and inferring long-term thermal cycles.At this aim, the temperatures at rock surface over the two years were evaluated for the front and the back faces of the rock-block, taking into account three different hours : (i) 11:The annual temperature distribution for each hour , allows The comparison with the annual air temperature revealed how maximum differences between the surficial rock temperature and the air temperature are reached on both faces, at time intervals in which the incident solar radiation is higher . On this side, the temperature distribution across two main opened joints was evaluated , testingDuring the heating phase of Spring and Summer, the air temperature within the joint is almost 5–6 °C lower than the surrounding rock temperature, because of the difference in thermal conductivities and specific heat capacities between the two media. This temperature difference can reach 8–10 °C in Autumn and Winter seasons, due to the lower temperature of the air filling the joint. During the cooling phase of all the four seasons, the air temperature within the joints and the rock surficial temperatures gradually decreases towards the same value, reaching isothermal conditions. The air temperature is consistently lower than the one founded within the opened joints, so revealing an influence of the warmer rock on the air temperature within the joints . More inNo significant temperature decay was observed across the two joint-walls of the opened joints , the temperature rise immediately decreases with an exponential function of the distance. Likewise, during the cooling phase (starts at 14:00) the temperature begins to decrease starting from the exposed front face, while, in depth, rock mass cool slower with a decay proportional to the magnitude applied thermal input and its position respect the heat source TPD of . A sharpThe presence of the open joint causes the fragmentation of the heat front propagation and a different amplitude of thermal cycles between the two portions of the rock mass, with temperature contrasts enhanced by the presence of joint in both heating and cooling phase . At the The IRT survey technique was here applied to deduce a thermal response of a double exposed rock-block to solar radiation in daily and seasonal cycles, adopting 1D and 2D analysis. The thermal conductivity obtained by laboratory tests, allowed to define the heat propagation through a 1D conductive model, deriving information on thermal gradients in the rock matrix through the comparison among IRT-based temperatures  and data from the rock thermometer (that monitors the rock temperature at depth of 8 cm). These parameters also allow to validate the temperature distribution resulting from IRT technique, comparing the latter with the theoretical temperature values deconvolved from depth to surface applying the Carslaw and Jaeger law , thus deThe IRT monitoring results allow a quantitative analysis on thermal rate of rock surface for both the back face (heating and cooling rates) and the front face (only cooling rate) of the monitored rock-block was carried out allowing to detect seasonal difference in the heat exchange velocity between environmental air and rock mass at its interface. This rate appears to be mainly influenced by intensity of direct solar radiation, by the surrounding environmental air temperature conditions, and, lastly, by the timing of the thermal maximum (function of the exposure of the rock-block). For this reason, both heating and cooling rates are higher in Autumn when the difference between the air temperature and the rock mass temperature is higher and heating and cooling of the rock mass are faster. As reported by other authors , thermalDaily mean temperatures with relative spatial variability, under 2D conditions, were calculated for the four seasons, evidencing a direct correlation with the same seasons . Daily thermal excursion derived by IRT, instead, underlined the importance of solar radiation in the thermalisation of the rock surface, showing a maximum excursion in Autumn with respect to the other seasons, when moderately high solar radiation but low air temperatures coexist. On the contrary, the temperature recorded by rock thermometer in depth appears to be primarily related to air temperature, with smaller disturbances deriving by direct solar radiation.The seasonal trend of the rock temperature at the rock surface was derived by averaging the monthly values referred to different seasons. For these reasons, the IRT technique is also suitable to study the long-period thermal signal that involves the entire monitoring period, evidencing the relations between surficial rock temperature and air temperature for both rock faces at different time-intervals. The long-term monitoring proved for the Winter season the occurrence of thermal inversion with the air, already starting from the daytime (14:00 for the front-face and from 17:00 for the back face). The thermal inversion is ascribable to lower radiative power of the sun in Winter season with respect the air temperature and to the exposure of the block. Possible night-time thermal inversions during the thermal monitoring carried out were not here analysed.The IRT monitoring performed showed an accurate spatial and time resolution for the description of transient thermal input over the different seasons , as wellThe observation of the temperature distributions resulting by multi-temporal 2D analysis  allows tThe results of IRT monitoring performed on the selected rock-block adopting 1D and 2D analysis, allow to infer the role of persistent opened joints in the propagation of the heat front in depth, that is, inside the rock-block. Open joints falling within the thermal active layer can interrupt the propagation of the heat flux inside the rock, playing a strong role in the 3D-thermal behaviour of the jointed medium. This fragmentation of the thermal front which can results in a different thermalisation of the rock-block . In partIn addition, because of this orientation respect to the back face , can be regarded as a possible mechanism responsible of thermo-mechanical inelastic strains along the joints and microcracks pervading the rock-block, which can cumulate over a long-period time window, by the repetitive accumulation of irreversible strains.This thermally-driven mechanism, together with other elasto-plastic or viscous deformations, can represent a mechanism, even without the presence of a wedge-driving block ,48.Rock masses are exposed to daily and seasonal thermal cyclical inputs deriving by intense solar radiation whose action, repeated on wide time-windows, can be regarded as a preparatory action for slope deformations. In this frame, an exhaustive comprehension of the intensity and variability of thermal inputs acting on a rock mass is mandatory for the validation of cause-to-effect relation between thermal forcing and possible stress-strain response.With this purpose, a multi-sensor thermal monitoring was carried out for two years across the four seasons by means of direct  and passive InfraRed Thermographic (IRT) techniques. The multi-sensing monitoring allows to derive the amplitude of thermal input at the rock surface detailing the effect of solar paths and exposure in the thermalisation of a rock-block. Daily and seasonal monitoring of the rock-air interface allows to: (i) constrain the role of thermal radiation in the heating of a slope mass, (ii) evaluating the magnitude of heating and cooling rates as well as daily thermal excursion.The obtained results confirmed the validity of the IRT technique, which was supported by analytic calculation through the deconvolution of the thermal signal derived by direct sensing considering a 1D heat propagation law in a semi-infinite space. The main outcomes also testify the suitability of the IRT to the characterisation of jointed rock masses with an appropriate time and space resolution: a combination of 1D and preliminar-2D analysis from IR-images was adopted, evidencing the suitability of the technique to the monitoring of jointed rock masses at large scale. The adopted analyses stressed the role of joints in heat propagation, highlighting possible implications for thermally-induced rock mass instabilities due to planar or wedge block failure mechanisms.In addition, the IR-Thermography revealed to be a useful tool for the characterisation of the thermal effects on discontinuous rock mass along wide surfaces, whose understanding is fundamental and preparatory to the study of thermo-mechanical processes at rock-block scale, encouraging further applications of this technique in the studies of thermal weathering of natural rock aggregates and to the conservation of cultural heritage, as well as for investigating precursors of rockfalls.These results represent reliable constraints for evaluating the surficial temperature distribution of jointed rock masses and suitable to validate numerical models devoted to establishing cause-effect relations between thermal stress and induced strains for assessing predisposing conditions for slope instability and mitigate the related risk.


Posture recognition has been widely applied in fields such as physical training, environmental awareness, human-computer-interaction, surveillance system and elderly health care. The traditional methods consist of two main variations: machine vision methods and acceleration sensor methods. The former has the disadvantages of privacy invasion, high cost and complex implementation processes, while the latter has low recognition rate for still postures. A new body posture recognition scheme based on indoor positioning technology is presented in this paper. A single deployed indoor positioning system is constructed by installing wearable receiving tags at key points of the human body. The distance measurement method with ultra-wide band (UWB) radio is applied to position the key points of human body. Posture recognition is implemented by positioning. In the posture recognition algorithm, least square estimation (LSE) method and the improved extended Kalman filtering (iEKF) algorithm are respectively adopted to suppress the noise of the distances measurement and to improve the accuracy of positioning and recognition. The comparison of simulation results with the two methods shows that the improved extended Kalman filtering algorithm is more effective in error performance. Human posture recognition is an attractive and challenging topic due to its wide range of applications, e.g., smart home environments for the monitoring of physical activity levels, assessment of recovery phases of living independently, and detection of accidental falls in elderly people . Among tTraditional human posture detection methods can mainly be divided into two categories: computer vision ,10,11,12On the other hand, human posture detection technology based on acceleration sensor data analysis has been proposed in ,14,15,16Methods based on acceleration sensors can achieve high recognition accuracy in motion states. Since acceleration sensors cannot acquire static information, they have difficulties in the recognition of positions, shapes, and so on. In recent years, the indoor positioning systems have seen increasing development ,22,23,24However, it is doubtful whether the indoor position system can also offer high-accuracy position estimation and posture recognition in dynamic activities. To answer this question, a new posture recognition method with the application of UWB indoor positioning technology is proposed in this paper. In the scheme, a positioning umbrella is designed and constructed first, the UWB distance measuring technology is applied. Receiving tags will be pasted onto key points on the clothes corresponding to human body joints of such as wrists and ankles. Least squares and Kalman filtering algorithms are adopted to reduce the noise interference, thus further improving the positioning and recognition accuracy. Simulation results reveal that the scheme can effectively estimate positions and recognize human postures.UWB is one of the hotspots in indoor positioning research. It can achieve centimeter-level accuracy in positioning, and has good multipath resistance performance. The radio can transmit a long distance with low power consumption. Hence, the hardware of the positioning system proposed in the paper is designed based on UWB. The positioning system consists of two parts: a positioning umbrella and receiving tags, as is shown in There are several arms in a positioning umbrella system. However, positioning in two-dimensional surface needs at least three arms, and positioning in three-dimensional space requires at least four non-coplanar arms. The more and longer the arms are, the higher the positioning accuracy is . Due to Powered by a button battery, the receiving tags attached to coats and caps are designed to be very small. In order to ensure low energy consumption, the positioning algorithm should be concise and effective. Positioning algorithms with large computation burdens or long operation time  do not meet the requirement. In order to make effective use of resources, tags attached to clothes, as is shown in The Microsoft Kinect technology collects more than 10 key joints of the human body to achieve posture recognition tasks. Similarly, we construct a human body model with 14 segments and 15 joints, as shown in Feature vectors and classifiers are always constructed to implement pattern recognition. Because receiving tags attached to clothes may differ in position for different persons, it is difficult to express the posture characteristics by the absolute position of receiving tags. Therefore, we introduce structure vectors to reproduce the postures of the human body. Due to the different characteristics of the human body, the structure vectors are constructed by the receiving tags which are attached to different positions of clothes to represent trunk, limbs and motion information. Since motion information is represented by a combination of receiving tags, behavior features can be obtained by calculating the vector modulus and vector angle. Compared to the method with one tag, the combination is a more effective method for posture recognition.structure vector. Suppose the position of the receiving tag attached to the left elbow is By analyzing the characteristics of human body, the vector by connecting receiving tags attached to the key joints is called Other body structure vectors are similarly constructed. The combination of 14 key joints of θ is changing synchronously. Meanwhile, angles can eliminate the structure vector differences originated from shapes and positions of various people.Besides structure vectors, angle relations between some vectors can effectively reflect the motion information. The angle of the two structure vectors, If Now, we make the naming rule of vector angles. The angle of vector i-th  anchor has its position labeled as j-th  tag is j-th tag and the i-th anchor node can be described in Equation (5), where v is the measurement noise which conforms to Gaussian distribution . The Since there are seven arms in the positioning umbrella in v, which has great influence on positioning accuracy. Therefore, an effective algorithm to suppress the measurement noise must be introduced. Since the algorithm needs to be transplanted to a microprocessor, the data processing methods in the receiving tags can’t be too complicated for saving energy. Therefore, concise and effective methods such as least square algorithms and extended Kalman filtering algorithms are adopted in our application.In Equation (5), UWB distance measurement is affected by noise W is white noise with mean value of 0 and the variance of Q  State Prediction:(2) Covariance Prediction:(3) Kalman Gain Calculation:The Jacobian matrix derived from Equation (11) is shown in Equation (12):(4) Status updating:(5) Covariance updating:e is the Kalman gain, which is calculated from historical data and the latest observation. Too much historical data will lead to cumulative errors. In order to avoid the accumulation of errors, many algorithms such as those described in [N is the length of window function with its range N latest observation The initial filtering value ribed in ,29 are iA in Equation (16). Introduction of the window function in the improved algorithm aims at dropping historical data. The smoothed distance obtained by least square polynomial fitting with the latest N observation can help to reduce cumulative errors, and finally to improve the performance of Kalman filter.Equation (15) is easy to solve with the coefficient matrix x-y plane, and one is deployed in the z-direction. When the character moves under the umbrella, we capture the position of the receiving tag of the left foot  and draw the trajectory in A sketch map of indoor environment is drawn in In the simulation experiment, the character walks several meters in a straight line at a constant speed. We set the sampling frequency as 119 fps, which means the positions of 14 receiving tags, such as head, shoulders, hands, knees, feet, and so on, should be recorded 119 times per second. Due to the high sampling frequency, we can see the details in the curve from While in For further comparison, the mean deviation is an effective metric. The algorithm designed in (a) Positioning comparisonAs described in The disadvantage of Kinect detection results is that they may be sheltered by something or some joints are missed. Moreover, Kinect cannot output the positions of the 14 key joints. However, these disadvantages are overcome by the indoor position system. The joints are missed only when receiving tags are out of power or broken in the proposal system. Unlike the Kinect method, the receiving tags can calculate the positions.The positions of 14 joints of human body can be estimated by the iEKF and LSE methods. Taking the head tag for example, (b) Struction vector comparisonWe now perform further analysis to examine the deviation of structure vectors for body postures. Taking vector (c) Vector angle comparisonVector angle reflects well the postures of the human body. Posture angles analysis play an important role in human posture recognition. Vector angle (d) Influence discussion for window length NN for the window function. When N is set from 0 to 40, we carry out the experiments and calculate the mean deviation of vector N is between 20 and 25. When N is equal to 23, the deviation is lowest. This doesn’t mean that it’s globally optimal when N is 23. The optimum N will float slightly for other vectors. Therefore, the ideal choice is We refer to the improvement of iEKF in (e) Future work(1)We have done basic posture recognition work for a single person. In order to improve algorithm robustness, multi-person postures need to be tested.(2)The experiment is only at the stage of algorithm simulation. Nonetheless, our system hardware is completed, and testing work of transplanted algorithms will be done in the following stage.(3)The final target is to deliver this wearable device to the elderly. More and more postures will be tested such as walking, sitting, sleeping, crawling, calling, falling down, and so on. Evaluating the weaknesses of the entire system and optimizing tasks need to be tested many times.The outputs of the experiment will be two import products: one is the indoor position umbrella which can provide location services, and the other is the specific clothes with receiving tags which can recognize the postures of the human body. In the near future, we need to carry out the following three tasks before we can convert the experimental results into products:N for window function has also been discussed, and the suggestion of reasonable range of N is given. The paper provided an overview of a posture recognition method for elderly care. The technologies available can be divided into two categories: vision-based recognition and sensor-based recognition. To avoid invasion of privacy, sensor-based recognition is a better choice. We proposed a sensor-based scheme for posture recognition with the indoor positioning system and receiving tags. The positioning umbrella with seven arms can provide location services. Meanwhile, the receiving tags are pasted on the surface of special clothes to measure the distance from umbrella by UWB radio. In this solution, we carried out simulation experiments to verify the usability of the scheme. The LSE method and iEKF algorithm are introduced to estimate the positions of receiving tags. We also present posture recognition algorithms with structure vectors and posture angles which combined a couple of tags. Experimental results reveal that iEKF algorithm offers more accuracy than the LSE method, e.g., by calculating the coordinates of the head tag, the mean deviation of iEKF is 7.16 cm, lower than that of LSE (11.70 cm). In the improved extend Kalman filter, the influence of parameter It has to be pointed out that there are also some disadvantages in our solution. We can achieve good posture recognition performance, but many tags are needed to keep working, and their energy is an important concern. In future work, we will transplant the algorithm to the processor, and focus on the further improvement of wearable technologies coupled with different kinds of postures, such as walking, sitting, sleeping, and so on. The test of the robustness and stability of the system also need to be carried out. We believe the prospect of applications for elderly care is vast.