In order to improve the accuracy of predicting the remaining electrical life of AC circuit breakers, ensure the safe operation of electrical equipment, and reduce economic losses caused by equipment failures, this paper studies a method based on the Savitzky–Golay convolution smoothing long short-term memory neural network for predicting the electrical life of AC circuit breakers. First, a full lifespan test is conducted to obtain degradation data throughout the entire life cycle of the AC circuit breaker, from which feature parameters that effectively reflect its operational state are extracted. Next, principal component analysis and the maximum information coefficient are used to remove redundancy in the feature parameters and choose the best subset of features. Subsequently, the Savitzky–Golay convolutional smoothing algorithm is employed to smooth the feature sequence, reducing the impact of noise and outliers on the feature sequence while preserving its main trends. Then, a secondary feature extraction is performed on the smoothed feature subset to obtain the optimal secondary feature subset. Finally, the remaining electrical lifespan of the AC circuit breaker is treated as a long-term sequence problem and the long short-term memory neural network method is used for precise time-series forecasting. The proposed model outperforms backpropagation neural networks and the gate recurrent unit in terms of prediction precision, achieving an impressive 97.4% accuracy. This demonstrates the feasibility of using time-series forecasting for predicting the residual electrical lifespan of electrical equipment and provides a reference for optimizing the method of predicting remaining electrical life. When it comes to the safety and reliability of power systems, low-voltage AC circuit breakers are considered essential protective devices. They are widely used in transmission, distribution, and generation fields. Their primary function is to rapidly disconnect faulty circuits in the event of a fault, ensuring the safe operation of the power system . The remTherefore, analyzing the aging mechanism of circuit breakers and studying the changing trends of characteristic parameters are of significant importance for evaluating the electrical lifespan of circuit breakers, as well as assessing the performance and reliability of the entire power system . By combFor the prediction of the remaining electrical lifespan of low-voltage circuit breakers, commonly used approaches include those based on physical models, experimental data, and intelligent algorithms ,4. PhysiF. Xing et al. improved the prediction model by considering the influence of arc duration on contact erosion, resulting in increased accuracy . Liu Z eOverall, these studies have made valuable contributions to the prediction of AC circuit breaker remaining electrical lifespan. However, there are still challenges to address, such as the need for a comprehensive representation of degradation information using a larger set of feature parameters, dealing with the nonlinear and non-smooth characteristics of electrical monitoring signals, and effectively utilizing the relationship between current and historical states to improve prediction accuracy.With the advancement of artificial intelligence technology, the use of deep learning  and machZ.HE and colleagues used the arc voltage and arc current signals gathered during the life testing process to extract features that reflect the degradation process of miniature circuit breakers. These extracted features were used as input parameters to construct a life assessment model for miniature circuit breakers using the backpropagation neural network (BPNN). The research results indicated that the extracted relative closing time and drop-off time could significantly reflect the deterioration process of miniature circuit breakers. The experimental data testing showed that the life assessment model constructed using trend characteristic quantities achieved good evaluation results . S.Sun eThis study examined a residual electrical life prediction model for low-voltage AC circuit breakers using the SG-LSTM method. The model considers the circuit breaker’s life status as a long-term degradation sequence and takes into account the correlation between multiple feature parameters over time. First, to effectively capture the degradation information of AC circuit breakers, PCA and MIC methods were employed to select an optimal feature subset. Second, the SG convolution smoothing algorithm was applied to smooth the feature sequence and reduce the influence of noise and outliers. Finally, the LSTM model was utilized to consider the correlation between previous and subsequent states, as well as to memorize long-term historical feature information, enabling the prediction of the time series for circuit breakers. Through case studies, the proposed prediction method showed high precision and offered a novel approach for predicting the residual electrical life of switching devices.The theoretical approaches used for this study include principal component analysis (PCA), maximum information coefficient (MIC), Savitzky–Golay convolution smoothing computational method, and LSTM.PCA  is a feaX be the sample matrix consisting of all samples in the dataset, represented as X = [(1) Let The covariance matrix of the sample matrix can be obtained from the mean values as follows:(2) Calculate the eigenvalues (3) Arrange the feature vectors in a matrix by arranging them in rows from top to bottom according to their corresponding eigenvalues. Set the desired dimensionality reduction to k and take the top k rows to form the matrix P.(4) Y represents the reduced-dimensional dataset, and the formula for obtaining it isThe maximum information coefficient (MIC), as introduced by D.N. Reshef and his team in 2011, is a measure of correlation between variables based on the theory of mutual information (MI) . Mutual The mutual information I between two variables U and V is defined as follows:The MIC calculation involves partitioning the scatter plot of a dataset D into several sub-regions and computing the mutual information for each sub-region between the two variables U and V. The highest mutual information value among these sub-regions is then chosen as the MIC value for the variables. The calculation formula for MIC is as follows:Here, “n” represents the sample size, and “B(n)” represents the maximum number of bins or grid cells. Typically, B(n) is chosen as The Savitzky–Golay filter (SG) is a time-domain filtering method that effectively removes noise from waveform signals while preserving their shape and width. It achieves this by employing local polynomial least squares fitting. Due to its ability to smooth and denoise data, the Savitzky–Golay filter has found extensive applications in various fields .The effectiveness of data smoothing depends on the chosen window width, which determines the number of data points used to calculate the average value during the smoothing process. The smoothing process can be represented by the following equation:To minimize the overall impact of the algorithm on the waveform, each point’s value is multiplied by a smoothing coefficient Suppose the smoothing window has a size of n = 2m + 1, and each point’s value is represented by z = . The points within the window are fitted using a k − 1 degree polynomial, as follows:By applying the least squares method, we can obtain a system of k linear equations. When the number of data points (n) is greater than the number of unknowns (k), the equations will have a solution. Consequently, the fitting parameters A can be determined. Hence, the following relationship holds:It can be described using the following matrix:The least squares solution for matrix A, denoted as The smoothing matrix for matrix D isThe “•” in formulas 12 to 14 represents the dot product operation between two matrices.The long short-term memory (LSTM) is a type of recurrent neural network used to address the vanishing gradient problem in traditional recurrent neural networks. It is commonly employed for handling time-series data with long time intervals and delays . The intIn the LSTM network, the forget gate determines if the quantity of information from the previous time step’s cell state By the aforementioned calculation, the forget gate can determine which information from the previous time step’s cell state should be forgotten based on the previous hidden state and the current input. This mechanism enables LSTM to flexibly retain and forget relevant information when processing time sequences, thus capturing the dependencies between sequences more effectively.The input gate is responsible for determining how much new information should be added to the current state. By using the tanh function, candidate information By the aforementioned calculation, it is possible to determine the importance of new information based on the input gate. The candidate information is then multiplied by the decision vector to obtain the updated memory state The output gate decides the output of the current cell and the hidden state that will be conveyed to the next cell. The input of the output gate consists of the previous hidden state By the aforementioned calculation, the output gate determines the output of the current cell state and also determines the hidden state that will be passed to the next cell. This mechanism allows LSTM to control the output and transmission of information based on the importance of the current state, enabling it to adapt better to different time-series tasks.Both the input gate and the forget gate contribute to the cell state In the equation, The recurrent structure of LSTM, as shown in In this structure, each LSTM module consists of an input gate, a forget gate, an output gate, and a cell state. Each module performs calculations based on the current input and the previous hidden state, and then outputs the current hidden state and cell state. By connecting multiple LSTM modules in layers, effective modeling and prediction of long-term sequences can be achieved.The recurrent structure of LSTM allows for the transmission and interaction of information in the temporal dimension. Through gate mechanisms, the flow of information is controlled, enabling LSTM to effectively capture long-term dependencies in time-series data.This study investigates a residual life prediction model based on an AC circuit breaker with SG-LSTM. The first step involves extracting feature parameters from the voltage and current data of the AC circuit breaker, which reflect its performance degradation state. Subsequently, the importance of these feature parameters for residual life prediction is determined using the principal component analysis (PCA) method. Additionally, the maximum information coefficient (MIC) is used to measure the correlation between feature parameters, eliminating those with low correlation and redundancy with residual life. This results in an optimal set of feature parameters. Next, the SG convolutional smoothing algorithm is employed to smooth the feature sequences, obtaining more stable feature data. Then, secondary feature extraction is performed on the optimal set of feature parameters to capture higher-order relationships and nonlinear patterns among features, yielding the optimal set of secondary feature parameters. Subsequently, the optimal set of secondary feature parameters is subjected to dimensionless normalization to eliminate dimensional differences among different features, ensuring they have the same scale for better application in model training. The preprocessed multidimensional time series is fed into an LSTM neural network, where the structure of the LSTM model is defined based on the specific application, including the number of layers, hidden units, activation functions, etc. The LSTM network extracts temporal information from the sequential data. The remaining life of the AC circuit breaker serves as the prediction label, and the time step and data partitioning are set according to requirements. The LSTM neural network is trained by updating weights and biases iteratively to learn and capture the correlations between the AC circuit breaker’s life cycle and the corresponding data. The model generates predictions by feeding the outputs through a densely connected layer. The resulting predicted values are then transformed through an inverse normalization process to produce the final estimate of the residual life. Finally, evaluation metrics are used to assess and analyze the prediction results, measuring the predictive performance and accuracy of the model.In this study, the Adam optimizer is used during the training process to fine-tune the prediction model’s parameters. This optimization technique helps improve the effectiveness of the loss function optimization and enhances the convergence of the model . In thisIn the given equation, n denotes the total number of data points. The symbol In addition to the MSE loss function, this study also employs several other evaluation metrics to assess the quality of the predictions, including the root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination , where “None” indicates the batch size, “1” represents the time step, and “5” corresponds to the number of features in each time step.To capture complex temporal dependencies and patterns in the data, we employ multiple LSTM layers in the architecture. The first LSTM layer (LSTM_1) takes the input time series and outputs a tensor with a shape of . The choice of 10 units in the output dimension is based on the characteristics of the problem, aiming to capture essential information from the time-series data while avoiding overfitting.To introduce regularization and prevent overfitting, we utilize Dropout layers after LSTM_1 (Dropout_1) and LSTM_2 (Dropout_2). The Dropout layer with an output shape of  acts as a regularizer by randomly setting a fraction of the input units to zero during training, promoting model generalization and reducing the risk of overfitting.Subsequently, we employ another LSTM layer (LSTM_3) to further refine the feature representation. The output shape of LSTM_3 is set to  to strike a balance between capturing relevant temporal patterns and reducing the model complexity.LSTM_4 is the final LSTM layer in the architecture, with an input shape of  and an output shape of . The 32 units in the output dimension are chosen based on an empirical trade-off between complexity and representation power.The following Dense layer takes the output of LSTM_4 and outputs a tensor with a shape of , which represents the final prediction for the remaining electrical life of the AC circuit breaker.Lastly, the OutputLayer is used to match the desired output shape of , ensuring compatibility with the model’s prediction and the actual target labels.In summary, the network parameter settings are carefully chosen to accommodate the characteristics of the problem and the experimental dataset. The inclusion of multiple LSTM layers and Dropout layers aims to capture relevant temporal patterns, prevent overfitting, and strike an appropriate balance between model complexity and performance.Given that failures of AC circuit breakers predominantly happen toward the end of their electrical life in practical scenarios, the dataset was split into training (80%), validation (15%), and testing (5%) sets. During the model training process, we employed the mean square error (MSE) as the loss function, and the loss function curve is shown in In the graph, the blue line represents the training set, and the orange line represents the validation set. During the first few iterations (less than 40), both the training and validation sets exhibit a downward trend in the loss function. With an increase in the number of iterations, the loss function gradually reaches convergence. Notably, after 40 iterations, the loss function of the validation set remains stable without any significant increase, indicating that the model successfully avoids overfitting. Therefore, it can be concluded that the model performs well during the training process and has a good training effect.Based on the comparison results in Based on the data presented in To highlight the superiority of the proposed prediction method and compare the differences between different methods more clearly, the authors of this study conducted a comparative examination of the prediction error rates of each method on the test set after feature sequence convolutional smoothing, as shown in The results of the comparison clearly show that the approach used in this research produces predictions that are closest to the true values and displays the greatest level of accuracy. In summary, the SG-LSTM-based residual electrical life prediction model for AC circuit breakers not only exhibits high levels of accuracy and robustness but also satisfies the demands of practical engineering applications. As such, it has considerable potential for use in predicting the residual electrical lifespan of low-voltage electrical equipment.Based on the findings presented in this study, our proposed algorithm demonstrates commendable accuracy, achieving an encouraging 97.4% prediction rate for the remaining electrical life of AC circuit breakers. Through a thorough case analysis, we have observed relative advantages of our method compared to widely used prediction models, such as BP and GRU.Moreover, our research indicates that the application of this algorithm extends beyond traditional planned maintenance, potentially enabling more efficient state-based maintenance practices in power systems. Additionally, our method shows promising potential for implementation in various scenarios, facilitating predictive maintenance for electrical equipment and optimizing resource allocation.Looking to the future, we recognize certain challenges and limitations in the practical implementation of our proposed method. Collaboration with industry experts and continuous research will be essential in addressing specific operational and environmental factors that may impact prediction accuracy. Ensuring the long-term viability of our predictive models in real-world applications relies on continuous data collection and model refinement.In conclusion, this study provides an analytical framework for enhancing residual life prediction algorithms and investigating power supply reliability. Moving forward, we will conduct further research to explore the model’s robustness and scalability, ensuring its effectiveness across various operational environments and load conditions in real-world power systems.


Facilitating the navigation of visually impaired people in indoor environments requires detecting indicating signs and informing them. In this paper, we proposed an indoor sign detection based on a lightweight anchor-free object detection model called FAM-centerNet. The baseline model of this work is the centerNet, which is an anchor-free object detection model with high performance and low computation complexity. A Foreground Attention Module (FAM) was introduced to extract target objects in real scenes with complex backgrounds. This module segments the foreground to extract relevant features of the target object using midground proposal and boxes-induced segmentation. In addition, the foreground module provides scale information to improve the regression performance. Extensive experiments on two datasets prove the efficiency of the proposed model for detecting general objects and custom indoor signs. The Pascal VOC dataset was used to test the performance of the proposed model for detecting general objects, and a custom dataset was used for evaluating the performance in detecting indoor signs. The reported results have proved the efficiency of the proposed FAM in enhancing the performance of the baseline model. According to the World Health Organization, there are more than 250 million visually impaired persons in the world . Until tRecently, autonomous systems were widely deployed in many fields such as automotive ,3, aeriaIndoor navigation presents a serious challenge for visually impaired people. The design of image processing applications for assisting the navigation of visually impaired people is still far away due to different challenges. External challenges such as complex backgrounds, illumination variation, occlusion, and deformation are the most to handle. Internal challenges such as the limit of computation resources and the limit of power must be solved to achieve the needed performances. The proposed application must provide accurate and real-time feedback while being suitable for implementation on low-power devices such as smartphones or dedicated platforms.The main goal of this work is to design an indoor sign detection application to be used for the assistance of visually impaired people’s navigation. Building a powerful indoor sign detection application requires great effort and a precise balance between computation complexity and performance. Most of the existing object detection models were improved to achieve high detection precision and fast inference time that meet real-time constraints without considering computation complexity. Object detection methods can be categorized into two main categories. One-stage methods such as you look only once (YOLO)  and singTo avoid the process of anchor hyperparameters calibration, anchor-free object detection methods  were proThis work proposed FAM-CenterNet, which used foreground information to reduce the influence of complicated background features in indoor navigation scenarios. The semantic segmentation that underpins the foreground region proposal network is supervised with the segmentation label obtained from the bounding box label. The transition between the foreground and background is introduced as the midground, which can give extensive edge information about the objects. For the incorporation of both the foreground position and scale information, detection accuracy is effectively improved. To balance the performance of the model and its computation complexity, the DLA  backboneIn summary, the main contribution of this work is using the centerNet model with a light backbone to guarantee low computation complexity. The proposed backbone was modified to achieve high detection accuracy. The original DLA model has a low computation complexity and acceptable accuracy. To improve the accuracy, we proposed to replace regular convolution with deformable convolution. This modification resulted in increased accuracy with a slight increase in computation complexity. Furthermore, an attention module was used to improve the localization performance of a small object in a complex background. The proposed attention module was inspired by the RPN in two-stage detectors. The RPN was very effective in detecting an object under challenging conditions such as a complex background, small object, and occlusion. The proposed FAM was designed to perform the same function as the RPN with lower computations.The proposed approach has been evaluated on two datasets. First, the Pascal VOC dataset  was usedBuilding an indoor sign recognition system for the navigation assistance of visually impaired people.Proposing a foreground attention module to enhance the performance of an anchor-free object detection model.Introducing the use of midground for transition between the background and foreground to collect relevant edge features of the target object.Introducing the use of foreground scale features for the regression to enhance the performance of the localization process.Proposing the use of a lightweight backbone to balance the performance and the computation complexity.Evaluating the proposed FAM-centerNet on two datasets. Extensive experimentations confirmed the efficiency of the proposed model and achieved top performances in both precision and inference speed while obtaining comparable computation complexity.The main contributions of this work are:The remainder of this paper is arranged as follows: For a long time, object detection has been an active research field that attracts industrial and academic research. Thus, important achievements were reported, and a wide range of methods was proposed. Classical machine learning methods were based on handcrafted features techniques, which are time-consuming with low performance. Recent methods based on deep learning techniques have achieved great success, especially using CNN models. There are two main categories in object detection models, which are: anchor-based models and anchor-free models.Anchor-based models rely on anchor information for both classification and regression. Those models can be classified into two sub-categories which are one-stage detectors and two-stages detectors. Two-stage detectors such as faster RCNN  and maskAnchor-based models eliminated the need for predefined anchors with the estimation of keypoints for the detection ,18,22. TModels with center point estimation only estimate the center point for each target object and use it directly for the final prediction without any grouping stage. Densebox  is basedThere is a particular object detection model that takes advantage of semantic features. Mask RCNN  is an obAn indoor object detection system was proposed in . The proIndoor sign detection was not widely investigated. Afif et al.  proposedAfter a deep analysis of the literature, it is obvious that anchor-free models are more suitable for implementation on embedded devices due to their high performance and low computation complexity. On the other hand, two-stage models are more accurate in detection and more trusted. Furthermore, exploring semantic features can improve the performance without affecting the computation complexity. In this work, we proposed the use of an anchor-free object detection model enhanced with a foreground attention module inspired by the RPN of two-stage models. In addition, semantic features were explored to extract foreground features, and their location was used to enhance the regression output. A detailed description of the proposed model is presented in the next section.The proposed FAM-centerNet is based on an improved anchor-free object detection model. CenterNet was the baseline, and an additional foreground attention module (FAM) was introduced to collect the foreground features. As a backbone, an improved deep layer aggregation (DLA) model  was usedThe integration of the FAM presented a serious problem. The use of the DLA structure has facilitated the integration of the self- and up-branches of the FAM. In addition, the up-branch was designed using deformable convolution and transpose convolution instead of regular convolution. The self-branch was directly injected into the fusion operation from the top of the DLA structure.The FAM was inspired by the RPN  to estimW and height H, the produced center point heatmap in the classification is R is the output stride and C is the class of the detected object (indoor sign in this work). So, a detected center point corresponds to a prediction C and Considering an input image with width d object . In the For the regression task, a scale prediction map After presenting the main concept of the proposed FAM-centerNet model, we will move to present the proposed foreground attention module (FAM) in The proposed FAM is based on semantic segmentation with the fusion of two methods. The first is based on the up-branch and the second is based on the self-branch. As shown in E is the output feature maps of the encoder and D is the convolution layers applied to those feature maps, the output F of the up-branch method can be computed as Equation (3).For the up-branch, the feature maps of the decoder were upsampled based on consecutive convolution layers. The resulting feature maps were used to obtain the foreground features maps. In effect, deformable convolution layers were used to change the channels, and the transpose convolution layers were used to upsample the feature maps. This method of the encoder–decoder strategy was widely used for semantic segmentation tasks, and it was adopted in this work to take advantage of the semantic features. Assuming that FM is the output feature map of the top of the DLA-34, the foreground feature map F can be computed as Equation (4).The self-branch directly applied regular convolution layers to obtain the foreground feature maps. In this method, the feature maps were collected from the top of the modified DLA-34 and then further processing was performed. Considering that Since the proposed FAM is based on object categories, the scale information for each object can be provided for each category. Using scale information for the regression part will improve the performance. So, region proposals from the foreground feature maps were concatenated with the scale information to be fed to the regression process. This technique will enhance the regression and adapt the output bounding boxes to the target objects.For two-stage detectors, each proposal region is classified as foreground or background. In RPN-based models, a region with an overlap of less than 0.3 is a background, a region with an overlap of more than 0.7 is a foreground, and the region between them is ignored. The ignored regions are considered midground in this work. To perfectly localize an object, the generated bounding boxes are too close to the edges of the object. So, the generated segmentation labels are strict foreground regions, and the surrounding area of the object was not considered. Since the surrounding area contains edge information, collecting those features is very important for both regression and classification. The midground region is the surrounding area of the foreground area. C is the categories of the target dataset, R is the output stride, and H and W are the height and width of the input image, respectively. An illustration of the semantic label generation is presented in The semantic labels were generated to be used for training the proposed FAM. Following the strategy in , the annThe training of the proposed FAM-centerNet is based on bounding box labels and the generated semantic labels of foreground and midground. The training workflow is presented in fL, a loss for the center point prediction noted pL, a loss for the scale prediction SL, and a loss for the offset prediction. The loss of the proposed FAM can be computed as Equation (5):To train the model, a custom loss function was proposed. The loss function is composed of four loss parts. A loss for the proposed FAM noted The loss for the center point prediction can be computed as Equation (6):The loss for the scale prediction can be computed as Equation (7):The loss for the offset prediction can be computed as Equation (8):L is the weighted sum of all losses. The total loss can be computed as Equation (9):In summary, the total loss The evaluation of the proposed FAM-CenterNet was performed on two datasets, the Pascal VOC dataset for general object detection and a custom dataset for indoor sign detection. All experiments were conducted on a desktop equipped with an Intel i7 CPU , 32 GB of RAM, and Nvidia GTX 960 GPU. The model was developed based on a Pytorch deep learning framework with GPU support based on CUDA 10 , and cuDNN v7 (NVIDIA), acceleration.−4 with a drop at the 90th epoch to 10−5 and the 120th epoch to 10−6. A batch size of 8 was used and the model was trained for 150 epochs. Many data augmentation techniques were applied such as flipping, rotation, and translation.First, we started with evaluating the proposed model using the Pascal VOC dataset. The Pascal VOC 2012 dataset contains a total of 11,530 images with 27,450 annotated objects for training and validation and 5585 images were used for testing. The Pascal VOC 2012 does not provide annotations for the testing data. The main objective of this dataset is the detection of a set of objects, 20 categories considered, in a real scene. A wide variety of annotated objects were provided with different sizes, distances, orientations, illumination, and occlusion. For training and evaluation of the proposed model, the original resolution of images was resized to 384 × 384. The Adam optimizer was used as a learning algorithm with an initial learning rate of 10−4 with a drop at the 70th epoch to 1.5 × 10−5 and a drop at the 100th epoch to 1.5 × 10−6. The model was trained for a total of 120 epochs with a batch size of 8. The model was initialized with the pre-trained weights on the Pascal VOC dataset. For the evaluation using both datasets, the mean average precision (mAP) was used as an evaluation metric.Second, an indoor sign detection dataset was used to evaluate the model for our target task, which is detecting indoor signs for navigation assistance for visually impaired people. This dataset was collected for different conditions by considering many challenges such as occlusion, the difference in point of view, lighting conditions, and orientation. A total of 4000 images were collected and divided into training and testing sets where 2600 images were used for training and 1400 were used for testing. The dataset provides annotations for four classes which are WC, exit, disabled exit, and confidence zone. Those signs are the most important for regular navigation in indoor environments. The input images were resized to 416 × 416 for both training and testing. To train the proposed model, the Adam optimizer was used with an initial learning rate of 1.5 × 10The evaluation of the proposed model was performed at different configurations of the FAM. As explained earlier, two methods for feature collection with two fusion techniques were proposed for the FAM. The first method is based on the up-branch by upsampling the features and the second method is based on the self-branch that directly collects features from the backbone. For fusion, we proposed summation and concatenation to merge features.The baseline for this work is the centerNet model, and the proposed FAM with different configurations was applied separately to evaluate the influence of each one. The achieved results using the Pascal VOC 2012 are presented in After that, we moved to evaluate the proposed model with different FAM configurations using the indoor sign detection dataset. Similar to the Pascal VOC dataset, we evaluate each configuration separately to observe its influence on the detection rate. The achieved results are presented in As shown in The fusion with the concatenation technique has the most balanced performance with an average improvement of 2% compared to the baseline. Different fusion techniques have achieved variable performances in different categories. The proposed fusion with the concatenation technique saves information generated by earlier layers, while the fusion with the summation technique accumulates the output of previous layers, which leads to losing some relevant features and decreases performance. Therefore, the fusion with the concatenation technique is more powerful and useful for the detection of small objects in complex background scenes. Hence, this configuration has achieved the best performance.While the proposed enhancement improves the detection rate, the inference speed decreases. After a deep analysis of the achieved results, the configuration with the self-branch was identified as the most suitable for indoor sign detection for navigation assistance for visually impaired people. The main reasons for this choice are the following:The FAM with self-branch configuration has the lowest computation complexity. In effect, it does not require any additional layers or operations since it is directly connected to the output feature maps from the backbone. Correspondingly, it will be suitable for implementation on embedded devices and can meet the real-time constraints of the target application.The detection rate of the model is comparable to other configurations. The generated information of the FAM is based directly on the feature maps from the backbone, which may lead to paying more attention to foreground regions. This enhances the detection performances for both classification and localization.Experimental results have proved the effectiveness of the proposed FAM for enhancing the performance of an anchor object detection model by eliminating the interference of a complex background and making the model pay more attention to the foreground regions. Considering that the exit and WC signs are most frequent and useful for navigation assistance of visually impaired people compared to other signs, and considering the importance of real-time processing, the self-branch method is the best configuration for this application.In this section, we will investigate the impact of the scale information and midground. Considering the proposed FAM-centerNet as a baseline model, the introduction of scale information and midground was evaluated on the indoor sign detection dataset. The achieved results of the baseline and the proposed improvement were reported in The reported results proved the efficiency of the scale information introduction for improving performance. The improvements for the exit sign are more obvious. The detection rate for the exit was improved by 1.49%. In addition, the detection rate for other signs has been slightly improved. The impact of the midground introduction was demonstrated by achieving slight improvements compared to the FAM-centerNet. An improvement in mAP by 0.36% was achieved. For the disabled exit sign, no improvements were achieved. The combination of scale information and midground was very effective, and great results were achieved. The model achieved the top detection rate of 94.52% mAP. The proposed improvements were very effective, but the midground introduction presents a lower impact on the performance.To prove the achieved results, a comparison study against state-of-the-art models was performed. Considering the difference in the experimental environment, the existing model was reproduced in our experimental environment for a fair comparison. As shown in Finally, the performance of the proposed model was compared to state of the-art object detection models including SSD, centerNet, RetinaNet and YOLO v3. The comparison was performed using the same dataset and experiment environment. The DLA 34 backbone was considered for all object detection models for a fair comparison. Navigation assistance for visually impaired people in indoor spaces requires high performances detection system. Considering the enhancement of an anchor-free model for indoor sign detection, a FAM-centerNet model was proposed. It was based on the centerNet model with a foreground attention module to eliminate the interference of complex backgrounds in indoor images. The FAM takes advantage of semantic segmentation features generated based on the bounding box labels. In addition, the midground was introduced to make the transition between the foreground and background. The proposed improvements have enhanced the detection rate compared to the centerNet model. Experimental results using the Pascal VOC dataset and indoor sign detection dataset demonstrated the high performance of the proposed model. In future work, the model can be compressed to be implemented on embedded devices and tested under realistic conditions.


An autonomous place recognition system is essential for scenarios where GPS is useless, such as underground tunnels. However, it is difficult to use existing algorithms to fully utilize the small number of effective features in underground tunnel data, and recognition accuracy is difficult to guarantee. In order to solve this challenge, an efficient point cloud position recognition algorithm, named Dual-Attention Transformer Network (DAT-Net), is proposed in this paper. The algorithm firstly adopts the farthest point downsampling module to eliminate the invalid redundant points in the point cloud data and retain the basic shape of the point cloud, which reduces the size of the point cloud and, at the same time, reduces the influence of the invalid point cloud on the data analysis. After that, this paper proposes the dual-attention Transformer module to facilitate local information exchange by utilizing the multi-head self-attention mechanism. It extracts local descriptors and integrates highly discriminative global descriptors based on global context with the help of a feature fusion layer to obtain a more accurate and robust global feature representation. Experimental results show that the method proposed in this paper achieves an average  Place recognition is a fundamental capability for robot state estimation and is widely applied in robotic systems such as autonomous cars. In restricted environments like underground tunnels where GPS ,2 is notUnderground tunnels exhibit significant large-scale geometric repetition due to their unique scene configuration. Sensor data acquired in such environments inevitably lack effective geometric features that can be easily distinguished. Existing algorithms, which are mainly experimented with in usual environments such as school campuses and highways, generally rely on recognizable descriptive features in the data for place recognition. However, when faced with underground tunnel scenes with a large number of repeating 3D structures and similar textures, it is difficult to guarantee the original recognition accuracy. It is even more difficult to deal with the tricky situations that often occur in underground tunnels, such as obstacles blocking the sensors and changes in sensor viewpoints.A few researchers have designed place recognition algorithms based on semantic information, focusing more on the influence of point cloud categories on the recognition effect. SG-PR , on the K key V value pairs related to the Q query vector, capturing local features in the data and adaptively enhancing local information in the point cloud. Additionally, the proposed sinc attention is used for local feature filtering, improving fine-grained quality prediction performance in underground tunnel scenes. This dual-attention strategy effectively enhances the extraction of local descriptors and integrates highly discriminative global descriptors based on the global context using the feature fusion layer, resulting in more accurate and robust global feature representation. Here are the contributions of this article:The use of the farthest point subsampling module significantly reduces the point cloud size, decreasing the computational complexity of the model while preserving point cloud features.A point cloud analysis module based on the dual-attention layer Transformer has been developed, enhancing the accuracy and robustness of place recognition.Experiments in place recognition and robustness testing on an underground track dataset demonstrate that our approach performs exceptionally well, achieving an average dispersion of 0.841.In summary, some advanced algorithms rely on the accuracy of semantic segmentation algorithms, and it is difficult to independently and stably realize the autonomous recognition of underground tunnels. Meanwhile, the advanced place recognition methods based on descriptors make it difficult to ensure recognition accuracy due to the difficulty of fully utilizing the small number of effective features in the underground tunnel data. To address these issues, this paper introduces an efficient point cloud place recognition algorithm, DAT-Net. This method first employs a farthest point subsampling module to remove invalid redundant points from the point cloud data, reducing the point cloud scale while preserving its basic shape and geometric features to improve algorithm efficiency and convergence speed. Next, this paper proposes a dual-attention Transformer module, which utilizes dot-product attention and sinc attention to adaptively extract important features to enhance distortion sensitivity. Specifically, by using the simple and efficient parallelizable dot-product attention, attention is focused on the In order to improve the accuracy of the place recognition algorithm in underground tunnels, this paper proposes an efficient point cloud place recognition algorithm (DAT-Net), as shown in Input: initial point cloud data {Select start point: randomly select a point as S, and then add S = {Initialize the subsampled point set: Begin by creating an empty subsampled point set For each point S.Sort the calculated Euclidean distance values and select the point furthest from point Update the selected point to Loop until the termination condition is met:N is reached.Termination condition: the preset number of downsampled points XOutput: downsampled point cloud data Compared to images, a single frame of 3D LiDAR captures a significantly larger amount of data, and using this directly as input for a Transformer would lead to a substantial increase in model size. Refs ,8,15 havN is set to 10,240 or 5120, it can be seen that the points in some areas are still very concentrated. After a series of attempts, it is found that when N is set to 1024, the downsampled point cloud can still express the shape of the original point cloud well, and will not be as sparse as the point cloud when N is set to 256. Therefore, this paper finally sets the number of downsampled points N to 1024. N = 1024 is the optimal value for this method. The specifications of the sensors and the environmental conditions during data collection influence the shape, density, and features of the original point cloud. The value of N for farthest point subsampling should be adjusted based on the specific environment and sensors.As shown in Setting the step size of convolutional neural networks (CNNs) to 1 is capable of achieving translational isotropy in image analysis. However, in point clouds, 0 degrees and 359 degrees are adjacent, which means limiting the step size of CNNs to 1 does not fully realize rotational isotropy for point cloud analysis. Therefore, a convolutional neural network suitable for point cloud analysis is required to better advance the performance of place recognition. An intuitive solution is to use a recurrent convolutional neural network with a step size of 1. Its greatest property is rotational invariance, which is well-suited for point cloud analysis. First, the downsampled point cloud data R is gathered and computed with the help of a linear connection layer. After flipping the feature As shown in K key–value pairs related to the Q query vector using the simple, efficient, and parallelizable dot-product attention. This helps capture local features in the data, adaptively reinforcing local information within the point cloud. Additionally, the proposed sinc attention is employed to perform local feature filtering and establish long-term dependencies, enhancing the algorithm’s fine-grained quality prediction performance in underground tunnel scenes. This dual-attention strategy effectively improves the extraction of local descriptors. Subsequently, with the help of the feature fusion layer, it integrates highly distinguishable global descriptors based on the global context to obtain a more accurate and robust global feature representation. More specifically, the feature fusion layer autonomously learns and assigns larger weight values to high-quality local descriptors, ensuring that the aggregated global descriptors can better describe the entire point cloud.Transformer and self-attention mechanisms have made a number of contributions in the field of machine translation and NLP. This has inspired the development of self-attention models for images, the most influential of which is ViT . Refs. ,19 triedF is first equated into local descriptors B is the matrix of learnable parameters. The multi-head self-attention layer is shown in Q, K, and V into the Sinc function to obtain a smoother numerical representation. After that, the parameters are exchanged and shared through Multilayer Perceptron (MLP). Based on this, feature multiplication by Q and K is performed to calculate the assigned weights of local features. The formula for the sinusoidal attention mechanism is expressed as follows:The second attention layer is proposed to parse the point cloud features in more depth and is named as the Sine Attention Mechanism. It first substitutes features The size of the features remains constant after multiple attention layers and the quality of the local descriptors can be improved by stacking the network layers L times. The final output is a splice of N D-dimensional local descriptors Inspired by SimGNN  and SG-PJ is multiplied with the average of all local descriptors to construct a global context to provide global structure and feature information. After that, a sigmoid function is used to ensure that the weights are in  to assign a weight value to each local descriptor. Finally, the final global descriptor As shown in In this paper, two linear connected layers are used to reduce the dimensionality of the matching vectors, and a sigmoid function is used to obtain the similarity ® CoreTM i7-6950X CPU processor with four NVIDIA GeForce GTX 1080Ti, each with 11G of memory. The experiments are implemented on a Pytorch [In order to evaluate the effectiveness of the method proposed in this paper, the experiments are conducted using the SubT-Tunnel  undergro Pytorch  deep lea Pytorch  optimizeP denotes precision and R denotes recall.In the experiments, point cloud pairs with timestamps differing by more than 30 s and Euclidean distances less than 3 m will be defined as positive samples, and point cloud pairs with Euclidean distances more than 20 m will be defined as negative samples. During training, sample pairs with timestamps less than 30 s are also included in the training set to increase the interference term. During the evaluation process, positive and negative samples with a time difference of less than 30 s are eliminated. This means that the performance of the algorithm is not evaluated based on simple pairs of positive samples (neighboring scenarios), allowing for a more accurate assessment of its capabilities in real-world environments. In this paper, DAT-Net is compared with state-of-the-art algorithms such as M2DP , LPD-NETF1 score  value isQualitative Analysis: Quantitative Analysis: According to the results in Obstacle occlusion and sensor view angle changes are issues that need to be addressed in the daily tasks of autonomous driving.As shown in Occlusion test: Sensor occlusion is a common phenomenon in automobile and robot operation that can easily lead to incomplete data acquired by sensors. For place recognition, using data with missing important information features can significantly affect the algorithm performance. In order to evaluate the processing ability of different methods for defective data, in this paper, points in a region are randomly removed from the point cloud. The results shown in Rotation test: Sensor viewpoint changes are also a common phenomenon in autonomous driving. For the human eye, it is not difficult to recognize a scene that is at a different view angle when passing by the same place. However, for position recognition, the change in view angle brings a completely different form of data. In order to evaluate the ability of the different methods to handle changes in sensor view angle, this paper randomly rotates each point in the point cloud. The results shown in The obstacle occlusion and viewpoint change tests demonstrate the stability and robustness of the method proposed in this paper for underground tunnel scenarios.As shown in Using the existing algorithms, it is difficult to fully utilize the small number of effective features in the underground tunnel data, and the recognition accuracy is difficult to guarantee for other problems. This paper proposes an efficient underground tunnel location identification algorithm (DAT-Net) that combines point cloud preprocessing and a dual-attention Transformer. The algorithm firstly adopts the farthest point downsampling module to eliminate the invalid redundant points in the point cloud data and retain the basic shape of the point cloud so as to reduce the size of the point cloud and at the same time reduce the influence of the invalid point cloud on the data parsing. After that, the dual-attention Transformer module is proposed to utilize the multi-head self-attention mechanism to promote local information exchange and extract local descriptors. With the help of the feature fusion layer, the highly recognizable global descriptors are integrated according to the global context, obtaining a more accurate and robust global feature representation. The experimental results show that the method proposed in this paper achieves an average


Anomaly detection and failure prediction of gas turbines is of great importance for ensuring reliable operation. This work presents a novel approach for anomaly detection based on a data-driven performance digital twin of gas turbine engines. The developed digital twin consists of two parts: uncertain performance digital twin (UPDT) and fault detection capability. UPDT is a probabilistic digital representation of the expected performance behavior of real-world gas turbine engines operating under various conditions. Fault detection capability is developed based on detecting UPDT outputs that have low probability under the training distribution. A novel anomaly measure based on the first Wasserstein distance is proposed to characterize the entire flight data, and a threshold can be applied to this measure to detect anomaly flights. The proposed UPDT with uncertainty quantification is trained with the sensor data from an individual physical reality and the outcome of the UPDT is intended to deliver the health assessment and fault detection results to support operation and maintenance decision-making. The proposed method is demonstrated on a real-world dataset from a typical type of commercial turbofan engine and the result shows that the F1 score reaches a maximum of 0.99 with a threshold of 0.45. The case study demonstrated that the proposed novel anomaly detection method can effectively identify the abnormal samples, and it is also possible to isolate anomalous behavior in a single performance signal, which is helpful for further fault diagnosis once an anomaly is detected. Critical engineering systems, such as aircraft gas turbines, must run safely and economically for their entire lifetimes. The anomaly detection and failure prediction of the gas turbines is of great importance for ensuring the reliable operation of safety-critical systems, which requires an accurate assessment of gas turbine conditions. Gas turbine condition assessment tracks measurable parameters during flight to derive insights into its current health state and trends for effective operation and maintenance decision-making .Traditionally the condition of a gas turbine is assessed only based on discrete data points gathered during take-off and cruise, typically called engine performance snapshots . Gas turIn the field of gas turbine conditions assessment, the general fault detection schemes use thermodynamic engine models for computing reference values representing the nominal performance of the aircraft engine. Fault detection performs a comparison between these reference values and in-flight measurements. Significant deviations between the measurements and model predictions indicate an underlying fault. Such gas turbine performance models are traditionally derived from physical principles by domain experts . The phyCurrently, most airlines adopted Quick Access Recorders (QAR) for data acquisition, providing flight data continuously sampled at frequencies of 1 Hz and more, which is also referred to as full-flight data covering the whole flight. The availability of these data obtained from a large variety of sensors enables the introduction of new methodologies to assess engine condition, which offers the chance to detect engine faults within one flight more reliably to support more efficient in-service operations and maintenance decisions ,7. An apIn recent years, the availability of big data from engineering systems has opened the era of digital twins, “defined as a virtual representation of a physical asset enabled through data and simulators for real-time prediction, monitoring, control and optimization of the asset for improved decision making through the life cycle of the asset and beyond” . In the Another important application of the DT for gas turbines is to construct a performance digital twin for real-time control, performance monitoring, and fault detection. In the context of performance monitoring and anomaly detection, a well-known solution is to build an anomaly detector in which an underlying digital twin is constructed based on an adaptive physics-based thermodynamic model of the engine provided by manufacturers. Zaccaria et al. adopted an adaptive physics-based model as a performance digital twin for aircraft engine fault detection and isolation . Panov eAs an important enabling technology of data-driven Digital Twin, machine learning, especially deep learning, has recently gained attention due to its ability to learn fault patterns directly from raw sensor data and its capacity to handle non-linearity in complex temporal correlation ,16. In tCurrently, most airlines adopted Quick Access Recorders (QAR) for data acquisition, and the availability of these big data sets enables the introduction of new PDT-based approaches in engine condition monitoring. Data-driven Performance Digital Twin (PDT) is an important method to accomplish real-time condition monitoring covering the full flight of an engine under various operating and environmental conditions. However, the research on the PDT modeling of the aircraft engine covering the whole flight conditions is very limited. This study aims to develop a novel approach to anomaly detection based on the digital twin paradigm that is an accurate simulation of an individual ’as-operated’ gas turbine. The developed digital twin consists of two parts: the Digital Twin model and fault detection capability. The Digital Twin model is a digital representation of the expected behavior of a real-world gas turbine, named Performance Digital Twin (PDT), which capitalizes on multivariate time series data obtained from the physical asset in operation to enable gas turbine performance tracking. Depending on the context, a digital twin can provide various capabilities; in this article, we focus on the anomaly detection capability of a digital twin. The main contributions of this paper is to propose a novel data-driven performance Digital Twin with uncertainty quantification, denoted as uncertain performance digital twin (UPDT), just based on the real-world gas turbine operational data rather than the physics-based model. The proposed UPDT produces a probabilistic digital representation of the expected performance behavior of a real-world gas turbine. Then based on the UPDT, the fault detection capability is developed and a novel anomaly measure based on the first Wasserstein distance is proposed to characterize the full flight data to detect anomaly.The remainder of the article is organized as follows. The developed fault detection framework based on the performance digital twin with uncertainty quantification is presented in Fault detection capability is developed based on detecting UPDT outputs that have low probability under the training distribution, i.e., the Out-of-Distribution (OOD) detection mechanisms which have received remarkable attention in recent years . An anomTraditionally, the gas turbine performance digital twins are based on an adaptive physics-based performance model provided by the engine manufacturers. Since these models are proprietary and usually unavailable to the asset operators, to circumvent the practical constraints of implementing physics-based PDT for airline companies, an alternative is given by data-driven models to develop data-driven PDT directly from the operational data of an engine. One of the key characteristics of a Digital Twin is the interconnection of information between the digital entity and the physical reality. In this paper, this is achieved through the use of sensors of the gas turbine which can directly measure the engine operating, control and performance parameters. The information exchange from the virtual representation to the physical reality is through informed decision-making, such as engine performance degradation assessment and fault detection to support operation and maintenance planning.Constructing such a PDT covering various stationary and non-stationary operating conditions requires correctly assessing the temporal correlations in full-flight data. Currently, most airlines adopted Quick Access Recorders (QAR) for data acquisition, providing flight data continuously sampled at frequencies of 1 Hz and more during the whole flight, which is also referred to as full-flight data. The availability of these data sets enables the introduction of data-driven PDT–based approaches in engine condition monitoring.The PDT aims to produce a digital representation of the expected performance behavior of a real-world gas turbine operating under various conditions during a flight. To correctly assess the transient performance of gas turbine engines requires the previous data points to be considered resulting in an auto-correlation. It is necessary to build models to capture time sequence information in the data. In the following, LSTM based deep learning network ,23, which inside the algorithm can describe a certain code for the input’s representation. It consists of an encoder and a decoder and is mainly applied for dimensionless feature extraction. The main function of the encoder is shown below:f contains a linear change W and a nonlinear activation b. The decoder converts the hidden representation h to the initial input in a similar manner, as shown below:Autoencoder is an unsupervised learning algorithm that attempts to replicate its input to its output. The hidden layer h should carry the interrelationship of different sensor parameters. The aim of the learning process is to minimize a loss function L as far as possible. The specific loss function for the proposed LSTM-AE based scheme will be discussed in Here the parameters T and width S, where T and S are time steps and the number of selected signals. Each input sample is recorded as tX, and tX is a matrix of j × 1, while each column j represents a signal at time t. The matrix tX here can be regarded as a “window” that moves on data series, and the elements inside the window represent the condition of the aero-engine within a specific time interval. This technique allows the trend information to be preserved, which is suitable for processing dynamic and variable time series data.The combined framework of LSTM-AE is suitable for constructing the data-driven PDT due to its advantage in processing time-series data. The scheme of the proposed LSTM-AE based PDT is shown in The digital twin incorporates as-operated data of the physical product to assist in the predictive and decision-making process. The goal of the PDT training is to learn accurate reconstruction of the normal performance behavior of the engine under continuously varying flight conditions. However, the lack of knowledge about the uncertainty of data captured from the physical domain, and consequently of models created from them, has a great impact on how much a PDT conforms to its physical product. In general, uncertainty is classified as epistemic or aleatoric. Epistemic uncertainty relates to the lack of knowledge, caused by poor assumptions, poor models and missing data. On the other hand, the aleatoric uncertainty relates to the variability of physical processes, which is inherent to the non-deterministic nature of measurement processes . CalculaTo improve the reliability and robustness of fault detection, the data-driven PDT should produce a probabilistic digital representation of the expected performance behavior of a real-world gas turbine. In this paper, a novel Uncertain Performance Digital Twin (UPDT)is proposed to take into account various uncertainty, such as operating condition disturbances, engine dynamics as well as measurement uncertainty. In the following, the performance prediction uncertainty quantification is taken into account in the deep neural networks to achieve a more reliable and robust fault detection.tX. The estimation of the distribution of the output data For modeling the uncertainty, the gas turbine performance measurements are assumed to be sampled from a given probability density function diag AnomDensity-based methods attempt to model the distribution of normal data, with an assumption that the anomaly sample has a low likelihood whereas the normal sample has a higher likelihood under the estimated density model. In this study, a density-based method is used to explicitly model the historical nominal data covering expected operating conditions with a multivariate Gaussian distribution, and flag test data in low-density regions as anomaly samples based on their likelihoods.U comes from the training samples of the historical nominal data, and V comes from the test sample of one flight. If the outlier score exceeds a predefined threshold Data scatter will always be present in the full flight data because of measurement system accuracy, recording accuracy, and actual stability of the aircraft engine during the data acquisition. Since there will always be a certain number of statistical outliers during one flight, an anomaly score, distance :(2)DfV=lThe case study is based on a real word dataset from a typical type of commercial turbofan engine, which is collected from a twin-engine aircraft fleet operating mainly for domestic routes over about two years. The engines are operating under various conditions, each of which has served around 3000 flights during this period. The average duration of each flight is about 2 h. Full-flight operation data consisting of continuous and discrete parameters defining the environmental conditions, power settings, and controller settings are acquired and stored in QAR. This data set contains engine dynamics and real word disturbances under actual operating conditions. For the engine type studied in this paper, the example operating and performance parameters related to engine operation recorded in the QAR are listed in i-th sensor parameter, and iS is the standard deviation.The example parameters of the entire flight are shown in The real QAR data of the turbofan engine is used to train and test the UPDT model. For modeling and training, seven parameters  characterizing the engine operating conditions and thrust setting are selected as the input data, and three parameters  characterizing the engine performance are used as the target output. A fixed-size sliding window technique is applied to create the input samples for the UPDT to account for the temporal correlation of the data.The network layer structure of the proposed data-driven UPDT in this article is shown in The training data set is constructed based on 300 flights randomly selected from around 3000 flights covering two years of operation of one turbofan engine, with a total of 1,180,000+ training samples. Then the data from 29 flights randomly selected from the remaining data are used for the test, including 26 normal flights and three known abnormal flights. For a comprehensive investigation of the proposed fault detection framework underlying various fault modes, 120 synthetic datasets were generated by the superimposition of the three key performance measurements with theoretical parameter shifts generated using the engine technique manual provided by the manufacturer. The dataset used for model training and test are listed in With the well-trained UPDT, we can get the reconstructed key performance parameters with uncertainty estimation  from the historical normal flights is presented in The proposed anomaly score derived from the first Wasserstein distance is then calculated to characterize the full flight data. The anomaly score indicates the difference between two 1D distributions of the log-likelihood computed based on the training and test dataset, respectively. If the anomaly score exceeds a predefined threshold, i.e., This work presents a novel anomaly detection approach for gas turbine engines based on a data-driven performance digital twin. The digital twin consists of an uncertain performance digital twin (UPDT) and fault detection capability. UPDT is a probabilistic digital representation of the expected performance behavior of real-world gas turbines operating under various conditions. Fault detection capability is developed based on detecting UPDT outputs that have low probability under the training distribution and flag test data in low-density regions as outliers based on their observation likelihoods.The proposed method is demonstrated on a real-world dataset from a typical type of commercial turbofan engine. The case study result shows the proposed data-driven UPDT can well characterize both the steady and transient performance behaviors of the real-world turbofan engines under various operating conditions. A novel anomaly measure based on the first Wasserstein distance is proposed to characterize the full flight data, and a threshold to this measure is determined in order to detect anomaly flight. The case study result shows that the F1 score reaches a maximum of 0.99 with a threshold of 0.45. One of the limitations of this work is that only three key parameters  are used to characterize the engine performance. Therefore, a potential future research direction is developing a UPDT that can take more performance parameters into account and the cross-correlations among the parameters need to be further explored.


The signal space separation (SSS) method is routinely employed in the analysis of multichannel magnetic field recordings  data). In the SSS method, signal vectors are posed as a multipole expansion of the magnetic field, allowing contributions from sources internal and external to a sensor array to be separated via computation of the pseudo-inverse of a matrix of the basis vectors. Although powerful, the standard implementation of the SSS method on MEG systems based on optically pumped magnetometers (OPMs) is unstable due to the approximate parity of the required number of dimensions of the SSS basis and the number of channels in the data. Here we exploit the hierarchical nature of the multipole expansion to perform a stable, iterative implementation of the SSS method. We describe the method and investigate its performance via a simulation study on a 192-channel OPM-MEG helmet. We assess performance for different levels of truncation of the SSS basis and a varying number of iterations. Results show that the iterative method provides stable performance, with a clear separation of internal and external sources.  Magnetoencephalography ,2 producHowever, the MSR will not completely attenuate all external signals, and some sources of interference may exist within the MSR itself (including other biomagnetic sources such as the heart and electronic devices such as cameras required for patient monitoring). These interference components can be many orders of magnitude larger than the signals of interest, obfuscating neuromagnetic data. The signal space separation (SSS) method ,6 is rouRecently, MEG systems based on optically pumped magnetometers (OPMs) have been developed . UnliThe performance of SSS relies on several parameters, including sensor calibration and accurate knowledge of the array geometry, but also the total number of channels available compared to the dimensions of the SSS basis . At presHere, we describe an iterative approach to the SSS method, which exploits the hierarchical nature of the multipole expansion to enable stable implementation on OPM-MEG devices with a low channel count. We begin with an overview of SSS and an illustration of the issues encountered when applied directly to a simulated OPM-MEG sensor array with 192 channels. We then describe the iterative approach and conduct a simulation study to investigate its performance and the optimal number of iterations required, followed by a discussion of the results obtained.r, Briefly, the SSS method uses a multipole expansion to describe the range of magnetic fields which can be measured by an array of sensors. Assuming the region containing the sensors is free from sources of magnetic field, a scalar potential By constructing two matrices To estimate the internal signal of interest  and require higher truncation orders due to an increased proximity of sensors to the scalp. For example, Tierney et al.  suggesteTo illustrate this point, we estimated the relative reconstruction noise of an array of 64 triaxial (192-channels) OPMs  as shown in To address this and enable the exploration of SSS on OPM-MEG data, we implemented an iterative method for estimating the weights. We exploit the assumption that the SSS vectors represent MEG data in a hierarchical manner, i.e., we assume that lower-order components always explain a larger amount of signal energy than higher-order components . We first compute a subset of weights for a subset of the vectors of First, we separate the  inner subspace vectors corresponding to orders We then apply the same approach to the weights vectorStarting with zero values for all weights, For The process is repeated up to Once the process is completed, a final weights vector To assess the performance of the iterative approach, we returned to the reconstruction noise simulation. We used the same array geometry, signal vectors and range of We used the same simulated signal vectors from We simulated a series of magnetic dipoles placed inside and outside the sensor array. External dipoles were randomly positioned and oriented within a spherical shell with an inner radius of 2 m and outer radius of 3 m. Internal dipoles were randomly positioned and oriented on a spherical shell with an inner radius of 0.005 m and outer radius of 0.05 m; the minimum distance between the internal sources and the sensors was 21.5 mm. Both shells were centred at the centre of mass of the OPM-MEG helmet, as shown in To assess the impact of the number of iterations, we applied the iterative SSS method using Application of the iterative implementation of the SSS method produces stable solutions with low relative reconstruction noise, high explained variance and low RMSE with a stable weights vector estimate ( signals a,b does Although our iterative approach was developed to overcome issues associated with low channel counts, application of the SSS method to the rapidly developing field of on-scalp MEG (including high-Tc SQUIDs  as well Application of our iterative implementation of the SSS method to OPM-MEG systems with low channel count substantially reduces the relative reconstruction noise compared to the standard implementation. For our study, using a simulated 192-channel system, we found that a stable estimate was obtained after just five iterations, with little dependency on model complexity. Further study is needed to investigate the many array geometries afforded by OPM-MEG and apply the method to experimental data.


The assignment of an individual to the true population of origin using a low-panel of discriminant SNP markers is one of the most important applications of genomic data for practical use. The aim of this study was to evaluate the potential of different Artificial Neural Networks (ANNs) approaches consisting Deep Neural Networks (DNN), Garson and Olden methods for feature selection of informative SNP markers from high-throughput genotyping data, that would be able to trace the true breed of unknown samples. The total of 795 animals from 37 breeds, genotyped by using the Illumina SNP 50k Bead chip were used in the current study and principal component analysis (PCA), log-likelihood ratios (LLR) and Neighbor-Joining (NJ) were applied to assess the performance of different assignment methods. The results revealed that the DNN, Garson, and Olden methods are able to assign individuals to true populations with 4270, 4937, and 7999 SNP markers, respectively. The PCA was used to determine how the animals allocated to the groups using all genotyped markers available on 50k Bead chip and the subset of SNP markers identified with different methods. The results indicated that all SNP panels are able to assign individuals into their true breeds. The success percentage of genetic assignment for different methods assessed by different levels of LLR showed that the success rate of 70% in the analysis was obtained by three methods with the number of markers of 110, 208, and 178 tags for DNN, Garson, and Olden methods, respectively. Also the results showed that DNN performed better than other two approaches by achieving 93% accuracy at the most stringent threshold. Finally, the identified SNPs were successfully used in independent out-group breeds consisting 120 individuals from eight breeds and the results indicated that these markers are able to correctly allocate all unknown samples to true population of origin. Furthermore, the NJ tree of allele-sharing distances on the validation dataset showed that the DNN has a high potential for feature selection. In general, the results of this study indicated that the DNN technique represents an efficient strategy for selecting a reduced pool of highly discriminant markers for assigning individuals to the true population of origin. Single nucleotide polymorphism (SNP) is a new type of marker that includes many important characteristics for evaluating animals2, crops3, and human population structure4. At present, genomic data plays a critical role in a variety of biological contexts due to its numerous advantages. However, the curse of dimensionality  is a major limitation to their ability for practical applications. The lack of complete pedigrees and misidentification of parents affects the accuracy of genetic evaluations, and consequently, the efficiency of breeding programs. Identification of the discriminant SNP(s) process is one of the most appealing opportunities to exploit genomic data, for practical use, including determining the population of origin for unknown individuals2. Many researchers have widely investigated discriminant SNP(s) and genetic diversity8. Researchers can use such SNP markers for developing a cheap customized panel to trace the breeds. Furthermore, the SNP(s) can provide a reliable solution for the traceability of breed-specific branded products9.DNA probes and sequences are two important indices in gaining a deep understanding of the evolution process, and the amount of DNA sequence data is rapidly increasing10. The same condition is true for biological data11, especially SNP markers. In various areas of breeding, we are always looking for SNP markers with enormous effects. Now, we import the issue to machine learning, especially the neural network approach. In genetics, this process is also known as Tag SNP Selection Problem (TSSP)12.In feature selection, researchers seek to identify key variables and eliminate annoying (or noisy) variables13. Researchers have argued the shortcomings of ANN, including the complexity of analysis, computational cost, and time consumption. However, we must mention that ANN’s high prediction accuracy compensates its drawbacks to a great extent. Deep Neural Networks (DNN) have been employed to analyze biological data15. They have many applications in feature abstraction and selection17. DNNs were able to construct many biological prediction models18, but their power of feature selection had been ignored for individual discrimination.Mimicking the behavior of the biological brain in the nerve system is the base of Artificial Neural Networks (ANNs), which are the information processing tools20. Fernández, et al.21 have indicated that ANNs were suitable to be used in fields of time series data for weekly milk prediction and clustering individuals in goat flocks. Ince and Sofu22 modeled data with ANN for the prediction of the sheep milk yield by using the back-propagation algorithm.The ANNs have recently been applied as a powerful statistical modeling technique for many areas of different biological data, especially in the animal sciencesFor feature selection (FS) based on ANN, a comparison was made in this study to discriminate among different horse breeds as well as to assign new individuals to their breed. Statistically, in the analysis of GWAS, all SNPs act separately and conduct the research with significant results. The consequence of this analysis obtains the identification of significant SNP markers, but the relationships between them are ignored. While the network approach is more reliable and logical monitoring all SNPs simultaneously leads to better results efficiently.23. Unlike most studies, Olden, et al.24 examined the performance of the Garson algorithm in the variable selection on simulated data, and have found that it has the lowest efficiency compared with other studied algorithms. Ibrahim25 showed that the Olden and Garson methods had the weakest results. The results of Fischer26 revealed that the Garson algorithm has a higher degree of stability in modeling non-linear relationships. Additionally, other studies have used the Garson and Olden algorithms, which are only applicable to ANN with a single hidden layer.To obtain the best results, allele dosage has been applied to ANNs, which is a completely unbiased estimation. The Garson (weights) algorithm illustrates behavioral instability in the analysis, which can be considered a weaknessTo the best of our knowledge, researchers had not investigated the potential of feature selection by ANN approaches for assigning individuals in horse breeds. We have analyzed the ANN’s potential to characterise, whether ANNs can be used as a tool for tackling the curse of dimensionality of SNP(s) data. We attempted to compare the DNN alongside a brief description of Garson and Olden methods to gain the relative importance of variables (SNP markers). While the DNN is a multiple hidden layer ANN, the two mentioned methods are compatible with a single hidden layer. This paper is one of the first studies to determine the discriminant SNP(s) on a large scale by using the sophisticated methods of ANN approaches. We have conducted this study intending to find distinct SNP markers to reduce the dimensions of the SNP panels as well as comparing different variable selection methods such as Garson and Olden through the ANN approach.25. Many studies have compared different FS methods29. The selection criteria for the variables in the DNN structure were the absolute value of the first hidden layer connection weights that they assumed as the regression coefficient. According to the DNN procedure, 4270 SNP markers had been selected for the rest of the analysis. The Garson and Olden algorithms led to a selection of 4937 and 7999 SNP markers for further analysis, respectively. The reason for choosing a more significant number of SNP tags for the Olden algorithm is the low transparency of the PCA plot. We must have mentioned that increasing the number of tags did not increase transparency anymore, this could be due to no linear relationship between SNPs number and PCA plot transparency. Moreover, the absolute increase of markers did not include a useful index for improvement unless the marker allele frequencies were different across subpopulations.In the current research, we have used the three feature selection (FS) methods namely Olden, Garson, and DNN. Neural networks are commonly referred to as powerful and efficient statistical modeling techniques by various researchersAfter the selection process of SNP markers, all SNP markers were sorted based on the calculated coefficient. The 460 top-rank SNP of each approach was selected, and all sub-SNP sets were compared to each other to find the common markers Table . Table 1Regarding Table 24 revealed that the Olden method was the best overall methodology for processing and identifying the variable importance in the neural network, especially when the inputs had a weak or strong correlation with output. Fischer26 compared the Olden and Garson methods and reported that the results obtained by the Garson method are preferable and more stable than those obtained by the Olden method for nonlinear relationships. Findings from his study have shown that ranks obtained by the Garson approach may be more reliable than the Olden method, especially when those ranks are used for modeling nonlinear data such as positive and negative quadratics and interactive data. The results of these studies indicated that the Olden (Connection weights) method had an excellent performance for different assumptions and, Garson (Weights), as the ancestor of the weighted methods, had a various behavior in these studies.In general, most of the studies have widely used the Olden and Garson approaches. The results of Olden, et al.25, and some others have used simulated data where the data have mostly contained the linear28 or semi-linear relationship27. However, the DNN approach could raise the performance and efficiency of the artificial neural network in circumstances where a large number of input variables  have confronted the system.All mentioned studies used the simulated or ecological data in which the maximum input variables were less than 20 variables. At first glance, both Olden and Garson’s algorithms used the input-hidden and hidden-output connection weights for calculating the importance of variables. The linear regression modeling habe been used as a control method on the real datasets for evaluating the input's significance in some studies30. The actual coefficients of SNP markers have been obtained step by step according to the original PCA plot, which is according to the numerical analysis in mathematics. In other words, after choosing a new coefficient, the PCA plot was drawn, and the breed distinction was compared with the main PCA plot created by 50K SNP markers panel  was applied to determine how the animals were allocated to the groupsnel Fig. . After mThe results indicated an excellent performance of PCA in distinct individuals into separated groups. PCA analysis has identified two subpopulations of Thoroughbred, (TB_UK & TB_US), as one breed, and a similar result was obtained for Standardbred (STBDNor & STBDUS) too. In Fig. 31 approach. Although one particular breed (Shire) had at least one failure assignment by each method. In general, all three feature selection methods assigned most of the individuals to the right population. It resulted in a 9% reduction in the potential of the assignment procedure. Two individuals in the Shire breed failed in all subsets. Red arrows indicate these individuals in Fig. We have estimated the likelihood of assigning 795 individual genotypes to their known origins (or breeds) by the Paetkau, et al.With the analysis of assignment and concerning values of LLR, obtained results showed that one failure was recognized as Belgian breed by three methods, and the other one was known as different breeds like Paint, Quarter, Swiss warmblood, and Thoroughbred-US. By using three methods, the first individual has 97.30% accuracy to be assigned to the correct race (Shire). By DNN, and Olden approaches, the second individual also had 91.89% accuracy for being appointed into the right breed. For further explanation, these failures might be due to hybrid or crossbreeding parentage. There were two Shire individuals in the center of the PCA plot Fig. ; the assAs shown in Fig. We have accurately calculated the percentages of individuals and correct assignments for different numbers of SNP markers. Testing the performance of each approach has been done at four different levels of LLR analysis. We found that DNN performed better than the other two approaches by achieving 93% accuracy at the most stringent threshold (LLR > 4) , Wilkinson, et al.38 worked for assigning individuals in 17 Bovine breeds.The results revealed that the DNN outperformed other methods with fewer SNP markers. Generally, about 500 discriminant SNP markers enabled us to assign new individuals to the right groups using different ways. There are some issues related to the comparison of results in this study with other ones. First, many previous studies used another type of marker with only a limited number of tagsWe have used a separate dataset to test the model. Firstly, we have applied the PCA analysis to find the relationship among the breeds like the training dataset Fig. .Figure 4In Fig. The results of this section revealed that all three artificial neural networks had an excellent performance. The Garson method with a minimum number of markers (fifteen) had a 60% accuracy, which may be due to the low number of animals and the distinction between the source in the test data, because there are significant differences between the countries of Switzerland, France, and England (the continent of Europe) and the countries of the Middle East and the Americas (Asian and American continents).By using one dataset, there is a possibility to observe a negligible amount of kinship relationships. Because all individuals are sampled from one herd, kinship relationships are practically inevitable in the research. Therefore, using new data from other sources reduces the probability of kinship among individuals. If unknown or novel information is introduced to the desired network, the least errors will get. Previously obtained results of the network were reliable enough for DNN to infer the right class of novel information precisely. In this case (DNN), the system undoubtedly possesses much power and much success in correctly determining the essential features.For a better understanding, we have used the Neighbor-Joining tree of allele-sharing distances on the validation dataset. Neighbor-Joining analysis performs better than PCA analysis on topics such as breed-level differentiation, the intermingling of breeds, outliers, genetic isolation, etc. First, we have analyzed whole genomic data  to show the breed-level differentiation in validation data Fig. .Then, the Neighbor-Joining analysis was done for each obtained dataset . Artificial neural networks (ANNs) will receive a great deal of attention in the various scientific fields, given that they are powerful statistical modeling techniques. However, in an attempt to provide useful insights into the contributions of the input (independent) variables in the prediction process, they have been labeled as the “black box” technique. As mentioned earlier, many published studies had been conducted to clarify the interpretation of the connection between the neurons in ANN.By comparing the results, the Garson and Olden procedures only work with a single hidden layer and single output unit, while multiple layer networks (DNN) do not suffer these limitations. Regarding log-likelihood ratio (LLR) for the individual assignment, the obtained results by this research revealed that ANN’s feature selection methods could be used for genomic data, especially for dimension reduction by DNNs. This finding solves the most critical issue for genetics researchers in dealing with the considerable dimension of data. Researchers can use DNN in the field of animal sciences because of the high performance of breed discriminants. Researchers in the field of genetics and breeding are seeking to reduce the number of biomarkers to find a link between the observed phenotype and these markers.The result of this study showed that the DNN has a high potential for feature selection in genomic data along with more flexibility in the application of ANNs in the field of animal sciences. Results also showed that using the connection weight of the first hidden layer in a DN Network provides the possibility to reach a high optimum level of accuracy for ranking and selecting the variables (SNP(s)). Another conclusion of this research is that the most critical weights for output values of every variable in a DN Network are the weights in the first hidden layer because all connected loads of the next layers are functions of the first layer's connected load. If three analyzes of PCA, LLR, and Neighbor-Joining achieve the desirable results, we will get the real discriminative features.It is necessary to point out that the results of this study shed some lights on the using of DN Networks  in genetics and breeding. Feature selection in the genetic field particularly on SNP markers is in the infancy period. The computation time will be reduced significantly. It should also be noted that the DNN network is increasing computing time but it was decreasing the error rate significantly. It can open a new opportunity to extend human insights.Finally, we think that this will be a fruitful approach to the study of existing domestic populations, such as inferior local breeds and strains in developing countries. In general, the present paper highlighted the importance of variable selection from the varying point of view, including the socio-economic perspective .7 have already described the comprehensive description and necessary details of data mining. In summary, Table 40.A total of 795 animals from 37 breeds of horse populations were genotyped by using the Illumina SNP 50k Bead chip . Petersen et al.Moreover, raw predictor variable data (SNP matrix) is used as the input variable in ANN. It is assumed that each of these markers represents a mathematical variable that can only hold 3 inputs .41. Data preprocessing included extracting common SNP markers between panels of 50K and 2M. This process resulted in the identification of 32K markers, and 14K of these markers remained after quality control  for further analysis.To assess the performance of the ANN methods, learning and evaluation were performed using two separate datasets, respectively. The testing dataset contains 120 individuals from eight breeds (Table 22. Neurons and their connections create a specific network architecture such as multilayer perceptron (MLP), self-organizing map (SOM), etc.13. In terms of genomic data analysis, we used two types of ANN architecture. The first one is a feed-forward multilayer perceptron (DNN) with two hidden layers, and the second one is a standard single hidden layer (ANN) with a back-propagation algorithm for the weight adjustments43. In Figure 44 and Neural Net Tools45 packages were applied by R software (version 3.4.0)46 to select informative and unique SNP markers that are within each breed. The mentioned algorithms (Garson and Olden) have been utilized by ANN to detect the relative importance of variables for the breed diversity characterization.Artificial neural networks represent complex structures that are generated by fundamental units (elements) called neurons29 have proposed the usage of a set of neural networks instead of a single one. In contrast to29 in the present work, the high-density SNP chip was partitioned into the sub-datasets with the same dimension and were used as input to identify the discriminant SNP(s).The large dimension of the SNP-panel leads to a stack overflow error in the computing process. De Oña and Garrido47 and has also been modified by Goh48. It was used to identify the relative importance of input variables by the calculated weights within connections in a supervised neural network. The Garson approach indicates relative importance values as the absolute magnitude ranging from zero to one (0-1). Olden and Jackson49 had proposed connection weights, also known as the Olden approach that has been used in this research.Weights (Garson approach), had been described by Garson50. The optimal number of nodes in the first and second hidden layers detected 40 and 38 nodes after testing a range of combinations. Finally, ANN with Garson and Olden algorithms contained 40 nodes in the hidden layer.For the DNN approach, the ANN with two hidden layers was used to identify the discriminant SNP(s) within breeds. Many combinations exist for selecting the number of nodes in the hidden layer12. We considered the SNP markers to retain a direct relationship with the horse breeds.  equation have a significant effect on the output variable, the absolute maximum weight obtained by DNN led to the selection of SNP markers that caused the diversity of the breeds.We have used the final fitted weights of the neural network for selecting the genetic markers. In the DNN approach, we assumed there was a linear relationship between the variable and the responseFigure By considering Eq. , it is a52. The method of Paetkau, et al.31 has been used for the assignment analysis (as had been described by38), and it had high effectiveness on individual assignment when high levels of genetic differentiation between reference populations existed52. It is noteworthy that the SNP markers were applied instead of the microsatellites. We have calculated the log-likelihood ratios (LLR) to accurately assess the performance of the assignment procedure. The log-likelihood ratios (LLR) will be calculated by comparing the probability of an individual assigned to its real population to the probability of it assigned to another population ; the University of Kentucky IACUC; the University College Dublin, Animal Research Ethics Committee; Swiss Law on Animal Protection and Welfare; the Ethical Board of the University of Helsinki; the Animal Health Trust Clinical Research Ethics Committee; Norwegian Animal Research Authority; UK Home Office License; and the Lower Saxon state veterinary office).Testing Data-set: DNA samples were previously collected with approval from the Animal Care and Use Committees at the respective institutions. All animal work was performed in accordance and with approval from international and national governing bodies at the institutions where the samples were collected ; University of California, Davis Institutional Animal Care and Use Committee (protocol #17491); University of Kentucky Institutional Animal Care and Use Committee (IACUC); Ethics Committee for Animal Experiments in Uppsala, Sweden (Number C121/14); Institutional animal care and use committee at Cornell University (protocol 2008-0121); University of California, Davis IACUC 19205; Hebrew University’s approval number AG-23476-07; Institutional Animal Care and Use Committee (IACUC), the Lower Saxony state veterinary office- registration number 11A 160/7221.3-2.1-015/11, 8.84-02.05.20.12.066; University of Sydney Animal Ethics Committee: AEC APPROVAL NUMBER: N00/9-2009/3/5109; permit no. BE75/16, veterinary service of the Canton of Bern; Institutional ethics committee of the University of Veterinary Medicine Vienna Good Scientific Practice guidelines and national legislation; Italian Ministry of Agricultural, Food and Forestry Policies (Mipaaf); Ethical Committee of the Canton of Bern ) No commercial animals were used in this study. Written informed client consent describing the purpose and duration of the study, procedures, potential risks and benefits and containing study contact information were obtained from private owners.Supplementary Information.