Mid-to-high altitude Unmanned Aerial Vehicle (UAV) imagery can provide important remote sensing information between satellite and low altitude platforms, and vehicle detection in mid-to-high altitude UAV images plays a crucial role in land monitoring and disaster relief. However, the high background complexity of images and limited pixels of objects challenge the performance of tiny vehicle detection. Traditional methods suffer from poor adaptation ability to complex backgrounds, while deep neural networks (DNNs) have inherent defects in feature extraction of tiny objects with finite pixels. To address the issue above, this paper puts forward a vehicle detection method combining the DNNs-based and traditional methods for mid-to-high altitude UAV images. We first employ the deep segmentation network to exploit the co-occurrence of the road and vehicles, then detect tiny vehicles based on visual attention mechanism with spatial-temporal constraint information. Experimental results show that the proposed method achieves effective detection of tiny vehicles in complex backgrounds. In addition, ablation experiments are performed to inspect the effectiveness of each component, and comparative experiments on tinier objects are carried out to prove the superior generalization performance of our method in detecting vehicles with a limited size of 5 × 5 pixels or less. With the continuous development of computer vision technology and the improvement of commercial UAV platforms, UAVs have been widely used in various scenarios such as traffic detection, agriculture and forestry, and disaster prevention and relief. Due to its flight characteristics of high altitude and long duration, the information collected by mid-altitude UAVs, such as graphics, videos, and spectrum, is becoming increasingly abundant. Moreover, UAVs can perform tasks in hazardous areas and bad weather conditions with efficient data acquisition capability, playing an increasingly important role in civil and military fields ,2.Object detection, one of the most important tasks in computer vision, aims to identify different types of objects and mark their locations. With the development of deep learning techniques, as well as the establishment of high-quality UAV image datasets ,4,5,6,7,Remote shooting from long distances leads to high background complexity of images and small size of objects. However, small object detection among a complex background presents a challenging problem to be solved in the field of image analysis and processing. There are two typical ways to define small objects. In the COCO dataset , objectsExisting detection algorithms for small objects can be divided into two main categories: traditional image processing and deep learning methods, which have certain limitations for tiny object detection. Traditional image processing methods are mostly adopted in infrared small object detection for infrared search and rescue systems ,11, suchIn this paper, a series of video sequences are collected to establish a mid-to-high altitude UAV images dataset. Different from low-altitude UAV images, the images collected by us are fuzzy with low contrast. The comparison of UAV images at different heights is shown in To address the above problems, this paper develops a multi-stage tiny vehicle detection framework (MTVD) for mid-to-high altitude UAV images based on visual attention and spatial-temporal information. The proposed method takes advantage of both deep networks and traditional methods with a progressive integration of them. Considering the strong dependence relationship between the road and vehicles in the ground scene, the road areas in the image are extracted by segmentation networks to suppress the interference of complex backgrounds. Then tiny vehicles on the road are detected by the improved RSS  algorith(1)We propose a multi-stage tiny vehicle detection framework with deep segmentation and traditional detection components to solve the problem that existing traditional methods have difficulty in detecting tiny objects from complex backgrounds.(2)We improve the RSS algorithm with both visual attention and spatial-temporal information, termed as STRSS, to assist the detection procedure by designing a locally weighted saliency decision, which can remove the false alarm and increase the detection precision.(3)The proposed method achieves effective detection of tiny vehicles under complex backgrounds with an F1 score of 78.32%. Moreover, our method is not limited by the size of the objects, and comparative experimental results show that it outperforms the advanced DNNs-based method for extremely tiny object detection with pixels smaller than 5 × 5.The contributions of this work are summarized as follows:This paper is organized as follows. In this section, several traditional small object detection methods commonly for infrared weak and small object detection are introduced, including single-frame detection and moving object detection. In addition, some algorithms based on DNNs are also mentioned.Single frame detection is suitable for imaging scenes with uniform backgrounds. Classical methods include maximum mean or maximum median filter , morpholIn recent years, algorithms based on human visual attention mechanisms have achieved good performance which uses the difference between object and background to perform background suppression and noise removal, to find the regions of objects. Laplacian of Gaussian (LOG) filter  smooths Since the object is locally uniform and has significant intensity differences from the surroundings, J. Matas proposed the Maximally Stable Extremal Region (MSER)  algorithMoving object detection mainly utilizes the spatial-temporal continuity of the image sequence to improve detection accuracy by introducing more dynamic information. Spatial-temporal continuity means that the trajectory of the object in adjacent frames is continuous and the appearance change of it also has continuity in time. The method of detection-before-trace (DBT) is usually adopted. Firstly, single-frame detection is carried out on the image sequence to obtain the candidate object, and then the candidate is judged by the motion characteristics and other prior information to detect the real object.Background estimation method ,25 detecOptical flow ,29 is a The basic idea of pipeline filtering ,33,34 isIn summary, traditional object detection methods have made good progress in infrared weak and small object detection through visual attention mechanisms. In the task of moving object detection, the spatial-temporal continuity of the image sequence is used to introduce more information to improve detection accuracy. However, the above methods are suitable for scenes with uniform and simple backgrounds and are difficult to apply to tiny object detection task under complex backgrounds of visible images for mid-to-high altitude platforms.In recent years, object detection based on DNN has become mainstream. Most of the existing small object detection algorithms propose some improvement or optimization strategies based on the general detectors. Multi-scale feature fusion ,36 and cIn the field of video object detection, the prediction effect of the current frame is mainly enhanced by using the characteristics or detection results of other frames ,48,49. SDNNs extract different levels of features through multi-level structure, but down-sampling will gradually weaken the feature information of small objects. The proposed method avoids multi-layer feature extraction, and the detection effect is independent of the object size.In this section, the general framework of our method is shown in For mid-to-high altitude UAV images, this paper proposes a multi-stage tiny vehicle detection method. The overall process is shown in RSS  is a smaIn view of the fact that the object in the UAV image, especially the tiny vehicle, generally has the mass effect and exhibits clustering specialty. The shape of the object is relatively regular and there is a conspicuous gray difference between the object and the surroundings so that it can be regarded as a spot in the uniform background. These characteristics are used to construct the object/background priors and obtain the candidate region of the vehicle based on the stability region detection. The main steps are as follows.(1)Multi-level threshold segmentationSince the geometric and statistical invariance of objects during binarization, the multi-level threshold segmentation is applied to extract the stability region of images.c; (3) Minimum outer rectangle b; (4) filling rate f; (5) aspect ratio a.Firstly, the three-channel color image is converted into a grayscale image, then a set of thresholds with a grayscale range of  and a step size of δ are used to perform binary segmentation on the grayscale image to obtain several sets of connected regions and their corresponding five structure descriptors. The five structural descriptors of each connected area in the binary image are: (1) the number of pixels (2)Connected regions clusteringu and v, four stability measure criteria are defined as follows:To describe the similarity between two adjacent connected regions (a)area variation: (b)center distance: (c)fill rate difference: (d)aspect ratio difference: Taking the Euclidean distance between the geometric center points of each connected region as the constraint, the connected regions obtained by binary segmentation are clustered through the spatial relationship between them. It is judged whether the center distance of two adjacent regions is less than the threshold. If so, the regions are in the same cluster.The threshold (3)Stability region post-verificationFill rate difference and aspect ratio difference indicate the degree of difference in the appearance of two connected areas. The smaller the difference between them, the more stable the corresponding target region is. Therefore, among four connected regions corresponding to the minimum difference of filling rate and aspect ratio, we select the region with the largest number of pixels as the maximally stable region of the object and take the minimum enclosing rectangle box of it as the optimal sub-image containing only one object. Then, the Otsu method  is used The threshold Finally, the stability region corresponding to each clustering result is obtained and the stability map The object has visual saliency in a certain area so that the human eye can often find it. RSS utilizes the local contrast mechanism to highlight salient areas and suppress uniform backgrounds. Firstly, the image is transformed into Lab space, and then the Gaussian filter is used to extract saliency information from the Lab color channel by low-pass filtering. The expression of two-dimensional Gaussian filter is:The saliency of each pixel in the image is expressed by the difference between the Lab image and its Gaussian blur, and the saliency map obtained after filtering is normalized. The saliency map is expressed as:The stability and saliency results are integrated by pixel-by-pixel multiplication, denoted as RSS is only suitable for small object detection tasks with relatively simple and uniform backgrounds while having weak adaptability to complex backgrounds. Furthermore, there are a large number of false alarms and object missing problems for vehicle detection in mid-to-high altitude UAV images. Therefore, an improved vehicle detection algorithm (STRSS) based on visual attention and spatial-temporal information is proposed in response to the above problems.The background of mid-to-high altitude UAV images is relatively complicated. Most of the shooting scenes contain roads and the road area has the characteristics of uniform intensity and less interference. Considering the strong dependence relationship between the road and vehicles in the ground scene, we first extract the road area in the image by road segmentation networks to suppress the interference of complex backgrounds.Most deep semantic segmentation networks are based on the encoder–decoder structure presently. The encoder is mainly responsible for feature extraction and context information capture, and the decoder predicts the labels of pixels through the decoded feature map. The network can capture semantic information of different scales by fusing the high-level and the low-level feature maps, which improves the robustness to multi-scale objects and recovers the loss of spatial information caused by the decline of resolution. At the same time, the encoder–decoder structure can better take into account the accuracy and efficiency of the network, making the end-to-end and pixel-level image segmentation methods become the mainstream. Some methods use Atrous Spatial Pyramid Pooling (ASPP) modules composed of convolutional layers of different atrous rates to expand the receptive field to capture multi-scale context information or recursive neural network to explore local dependencies to improve segmentation accuracy.Deep semantic segmentation networks perform well in the road segmentation task of street view image, so they can be applied to the UAV scene to extract the road area from the mid-to-high altitude UVA image to generate a priori to guide the subsequent vehicle detection.Candidate regions extracted by stability detection are the premise of saliency detection and the basis of tiny object detection. Therefore, the stability region extraction stage is further improved to obtain more effective candidate regions of objects.(1)White objects processingRSS believes that compared with the infrared scene, small objects obtained in the visual band usually have a smaller intensity value. However, white vehicles are typical objects in UAV images of mid-to-high altitude, whose intensity is greater than surroundings. RSS algorithm can achieve relatively accurate detection for black objects with low gray value, but will significantly miss white vehicles, as shown in Missing detection mainly occurs in the process of extracting the stability region. In the process of multi-level threshold segmentation, a set of thresholds with a grayscale range of  and a step size of δ are used to perform binary segmentation on the grayscale image. The pixels with grayscale values less than the threshold are binarized into 255, and those with grayscale values greater than the threshold value are binarized into 0, and then the white connected regions are extracted from the black background. With the continuous increase of the threshold, the background area in the image becomes binarized into white, while the white object with a higher gray value becomes black after binarization and cannot be extracted. Therefore, we perform the inversion operation on the binary image with a high threshold according to a certain proportion to ensure that the connected region corresponding to the white objects can be extracted. Additionally, white trucks have black cabins, and some vehicles will form black shadows around them under certain lighting conditions, so there are always two overlapping detection boxes of the real object and its shadow in the final detection result. When clustering the low threshold binary image and the inverted high threshold binary image, clustering conditions are modified by doubling the centroid distance threshold to divide the object and its shadow into the same cluster, and expand the overlapping stability region, so as to obtain more precise detection results.(2)Stability region discrimination based on Hu momentAfter the second post-verification, the stability region of the object may correspond to a real vehicle or noise. To distinguish them, Hu invariant moment is used to constrain the shape of the binary stability region. Hu invariant moment is a statistical feature of an image, which has rotation, translation, and scale invariance. It contains seven eigenvectors constructed by the linear combination of second and third-order normalized central moments. Since the shape of the vehicle targets bear rectangular characteristics, the stability regions with irregular shapes, such as linear and triangular are removed by calculating the Euclidean distance between the invariant moment of each stability area and the feature vector of the rectangular template and taking into account each invariant moment component, so as to remove false alarms and improve the precision of detection.Due to the influence of environmental factors such as illumination, shadow, and occlusion, the stability detection results of different frames are inconsistent. This paper introduces the inter-frame motion relationship of the image sequence to assist the detection based on single-frame detection to improve the detection precision of tiny vehicles. The specific method is as follows: the idea of data association in multi-object tracking tasks is borrowed to associate the candidate regions in different frames so that the same vehicle in different frames in a certain period has the same ID. Then, the number of occurrences of the candidate region corresponding to each ID in the whole image sequence is respectively counted as a probability weighting factor. If a candidate region appears only once in the image sequence, it is more likely to be a false alarm, so its corresponding probability weighting factor is lower. Conversely, the candidate area with a higher probability of occurrence is more likely to be the real object. Therefore, the probability weighting factor is used as the weight of the subsequent saliency detection to further remove false alarms and improve the detection precision.(1)Candidate region associationSpatial-temporal continuity means that the trajectory of the object in adjacent frames is continuous and the appearance change of it also has continuity in time. Therefore, regions in different frames are associated through the relative position relationship and appearance similarity of them. The position measurement is used to calculate the distance between the centroids of candidate regions, and the apparent similarity is measured if it is less than the threshold. The appearance measurement is mainly carried out by converting the original sub-image corresponding to the candidate region into Lab color space and performing color quantization. Each color channel is quantized to 4 bit, so the color number after quantization is reduced to m in the ith frame.The formula for calculating the appearance similarity between two adjacent candidate regions is:Then, the KM matching algorithm is used to perform the candidate region association between adjacent frames, and the apparent similarity between them is used as the weight of the KM algorithm. The object association problem is transformed into the problem of finding the maximum weight perfect matching of the bipartite graph. Finally, the probability weighting factor corresponding to each ID is obtained.The main task of this paper is not real object tracking, but only the use of probability weighting factors to modify the subsequent saliency detection within a certain time range, so the problems of the disappearance of old objects and the emergence of new objects are not considered. (2)Locally weighted saliency decisionThe saliency map is obtained after Gaussian filtering. In this section, we introduce a probability weighting factor to weight the Gaussian filtering results and calculate the final saliency score of the candidate regions extracted by stability detection. The stability regions corresponding to the same ID have the same probability weighting factors. The final saliency score is obtained to determine the authenticity of the candidate. If the saliency score is greater than the threshold, the candidate region is regarded as the true object otherwise, it is judged as a false alarm.k, k.The saliency score is calculated as follows:We collect a series of video sequences with RGB data based on actual flight data with a certain length of time under different atmospheric conditions, different flight altitudes, and imaging distances. The videos are shot in a plain area in eastern China, containing multiple different scenes such as cities, villages, and suburbs. Each scene is captured by the visible light and infrared integrated camera on the medium-altitude UAV. The imaging device has two degrees of freedom relative to the drone equipped with GPS , INS , and altimeter at the same time. The flying height of the UAV ranges from 3 to 7 km and the original resolution of each video frame is 1392 × 1040. Decoding, storage, automatic calibration and annotation of the collected remote sensing data are performed and a database of mid-to-high altitude UAV images containing video images and metadata is established.The vehicle detection algorithm proposed in this paper requires prior segmentation of the road area, so the mid-to-high altitude UAV image dataset we established is divided into road segmentation set and tiny vehicle detection set. The main information is shown in In order to construct the vehicle detection dataset of mid-to-high altitude UAV images, we use LabelImg to annotate the vehicles. A total of 160 images in 4 scenes are labeled, of which there are 827 object instances with a size range of 10 × 10 to 50 × 50 pixels, including stationary vehicles and moving vehicles. To clarify, objects with 50 × 50 pixels are usually large trucks, while most other vehicles in images are only 10 × 10 pixels.RSS obtained a set of optimal parameters through a large number of experimental analyses. This paper makes fine-tuning on the basis of the original parameters to adapt to our mid-to-high altitude UAV image database. The step size of binary segmentation δ is set to 17.2, and the range of the number of pixels of stability region is set to  to ensure that our algorithm adapts to the changes of object size. The threshold of the second post-verification of the stability region Both semantic segmentation and object detection can be regarded as a multi-classification task, and the confusion matrix can be used to compare the predicted output classification results with the truth. The difference is that semantic segmentation compares the predicted output results pixel by pixel, while object detection classifies the bounding box according to intersection over union (IoU) between the ground truth box and predicted box. Both of them can be evaluated by the precision, recall and F1 score. Furthermore, mean intersection over union (mIoU) is usually used to determine the accuracy of the segmentation. (1)PrecisionPrecision represents the proportion of the true positive in the sample that the prediction is positive, which is defined as:TP), otherwise, it is false positive (FP). In addition, the ground truth box that does not match any prediction box is false negative (FN).If the IoU between the predicted box and ground truth box is greater than the threshold, the detection box is true positive F1 scoreF1 score is obtained by the weighted average of precision and recall, which is defined as:(3)MIoUk represents the number of categories. MIoU reflects the degree of coincidence between the predicted image and the real image. The closer the ratio is to 1, the higher the degree of coincidence and the higher the quality of semantic segmentation.MIoU is a commonly used evaluation metric for semantic segmentation, which calculates the ratio of intersection and union of two sets of real value and predicted value, and is defined as:(1)Road segmentationAccording to the UAV image and corresponding single-channel binary label, we train the semantic segmentation model based on Deeplabv3-plus , and we (2)Vehicle detectionThe proposed algorithm based on the vehicle detection dataset of mid-to-high altitude UAV images is evaluated. In the mid-to-high altitude database we established, the quality of the image is greatly affected by weather, light, platform speed and altitude, and the surrounding environment, so the datasets have low contrast and poor clarity, and there are large background differences between different sequences. In addition, there are also certain differences between image backgrounds of different frames in the same scene, mainly including rotation, scaling, translation and other transformations due to the motion characteristics of UAV shooting, which further increases the difficulty of detection.In this part, ablation experiments are performed to inspect the validity of each part. We take advantage of RSS as the baseline and fundamentally fine-tune the parameters to adapt itself to the characteristics of our dataset. Saliency detection is designed for removing false objects based on regional stability results, in other words, objects that are not extracted during the stability detection phase will never be detected. We attempted to improve the seriously missed detection and extract more effective candidate regions by modifying the condition of saliency decision, eliminating the second post-verification and white objects processing. However, at the same time, this will cause a decrease in precision. The methods of stability region discrimination based on Hu moment and locally weighted saliency decision are adopted to remove false alarms and improve detection precision. In this part, we conduct ablation experiments on each idea by changing one condition on the basis of the previous one while the other conditions remain unchanged. We analyze the detection results with precision, recall and F1 scores. Due to the small size of the object, slight positioning deviation will lead to a significant decrease of IoU, and the detection box is closely surrounded by the object, while there is a certain error in the manual annotation. Moreover, due to the motion characteristics of UAVs and vehicles, the objects are prone to motion blur, which affects the accuracy of the location. Different from train-based methods, which can regression the accurate bounding box of the object, our method can only extract the object region according to the low-level feature of the image. For the above reasons, the threshold is set to 0.2 to obtain more reasonable evaluation results. The experimental results are shown in According to the results in The motion characteristics of UAVs and vehicles, as well as environmental factors, lead to no clear boundary between vehicle and background, and the stability region second post-verification leads to the removal of many real objects. Experimental results show that the application of removing second post-verification can help significantly improve the recall without any false alarms. Then, we dealt with white objects by inversion and clustering after which the recall increased by 20.56%, but the precision decreased by 6.22% due to the introduction of white noise during processing.Finally, Hu moment discrimination and locally weighted saliency decision were used to further remove false alarms, and the optimal detection effect was obtained. The precision increased to 71.26%, the recall increased to 86.94% and the F1 score reached 78.32%. In this part, original images are resized to generate series of images with lower resolution, and the size of the vehicles is reduced proportionally. Additional test images are used to explore the detection effect on tinier objects. Since our proposed algorithm is pixel level, the bilinear interpolation method used in scaling will change the pixel value and affect the detection result to a certain extent. During the test, we appropriately modify the parameters related to image resolution and object size, and send low-resolution images into the detection model. The detection results obtained are shown in It can be seen from We also compare our approach with the current state-of-the-art object detection networks, such YOLOv5, on the scaled mid-to-high altitude UAV dataset of different sizes. F1 scores obtained by fine-tuning Yolov5 network on our dataset are compared with the results of our method in This paper brings forward a tiny vehicle detection approach based on visual attention and spatial-temporal information for mid-to-high UAV images. On account of the scene dependence of vehicles, we segment road areas of the image through the deep semantic segmentation network and extract the stability regions from road areas as candidate regions of the vehicles. To remove the false alarms, we introduce the inter-frame motion relation and design a locally weighted saliency decision method to perform the second inspection on the candidate regions. Experimental results demonstrate that our method achieves effective detection of vehicles in mid-to-high altitude UAV images. The detection algorithm does not require large-scale annotation data for model training. Moreover, our method is more robust to the size of objects and performs well in the detection of vehicles with limited pixels below 5 × 5, which can effectively provide support for image data analysis of ground.


This paper proposes a reliable technique for pipeline leak detection using acoustic emission signals. The acoustic emission signal of a pipeline contains leak-related information. However, the noise in the signal often obscures the leak-related information, making traditional acoustic emission features, such as count and peaks, less effective. To obtain leak-related features, first, acoustic images were obtained from the time series acoustic emission signals using continuous wavelet transform. The acoustic images (AE images) were the wavelet scalograms that represent the time–frequency scales of the acoustic emission signal in the form of an image. The acoustic images carried enough information about the leak, as the leak-related information had a high-energy representation in the scalogram compared to the noise. To extract leak-related discriminant features from the acoustic images, they were provided as input into the convolutional autoencoder and convolutional neural network. The convolutional autoencoder extracts global features, while the convolutional neural network extracts local features. The local features represent changes in the energy at a finer level, whereas the global features are the overall characteristics of the acoustic signal in the acoustic image. The global and local features were merged into a single feature vector. To identify the pipeline leak state, the feature vector was fed into a shallow artificial neural network. The proposed method was validated by utilizing a data set obtained from the industrial pipeline testbed. The proposed algorithm yielded a high classification accuracy in detecting leaks under different leak sizes and fluid pressures. Pipelines play an important role in the distribution of liquid and gas resources. However, a leak in a pipeline can lead to severe consequences such as wasted resources, distribution downtime, risks to community health, and economic losses . To avoiIn the past decade, researchers have focused on feature extraction and feature recognition models for leak detection in pipelines ,11. AE hDeep learning (DL) methods can solve the problem of leak-related information extraction and classification . DL algoThe visualization and separation of the leak features from noisy data; the time-domain AE signals were transformed into AE images using CWT. From these AE images, leak-related features could be extracted. To surpass the traditional feature extraction techniques, a novel deep neural framework of CNN–CAE was proposed to extract the features autonomously. To the best of the author’s knowledge, CNN–CAE-based leak-related feature extraction has not been presented in the literature so far;The feature space obtained from the CNN–CAE was classified into “leak” and “no leak” states of the pipeline under variable leak and pressure conditions using a shallow ANN;Real-world pipeline data were used for the validation of the proposed method.A leak in the pipeline results in stress waves. AE sensors installed on the pipelines record these stress waves as they transmit through the pipeline walls. The leak-related stress waves produce transients in the AE signal known as hits or AE events. Therefore, traditional AE features, such as rise time, decay time, and counts, can be extracted from the AE signal by defining a threshold above the level of continuous background noise. As mentioned earlier, the pre-defined threshold for extracting AE features can lead to false alarms due to the fact of noise in the AE signal. Furthermore, extracting AE features from the AE signal and defining a threshold above the level of continuous background noise requires human expertise and domain understanding. To address this problem, this paper proposes a new deep learning-based model that extracts the leak-related features from the AE signal without human intervention. To utilize the leak-related transient in the AE signal, the proposed method converts the AE signal into AE images. Then, the proposed deep learning-based model extracts features from the AE images obtained from an experimental setup for pipeline leak detection. The identification of the leak against the normal signal is made based on the extracted features. Here, many convolutional filters of a CNN, which pass over the input scalograms to pick up the patterns in certain local parts, backed by kernels, are used to extract half of the feature pool, referred to as “local features” in this paper. At the same time, the latent space representations in the bottleneck layer of the trained convolutional autoencoder complete the second half of the feature pool, referred to as “global features” in this paper. These high-level features are the basis of input data that help the autoencoder to reconstruct the input scalogram. At the end, the feature pool with both the global and local features is used to assess the health of the pipeline through an ANN. The main contributions of this study were: The proposed method for leak detection was applied to a metallic steel pipe. Some studies suggest that leak detection in plastic pipes is challenging ,34. In tThe remaining part of this study is organized as follows: the proposed method is presented in The overall flow of the proposed method is illustrated in Step 1: AE signals are collected from the pipeline testbed using AE sensors;Step 2: AE images are obtained from the AE signal by transforming it using CWT. The CWT scalograms obtained from the AE signal represent the time-domain AE signal over different time–frequency scales in a 3D image, referred to as AE images in this paper. The change in the color intensities in the AE images shows the change in energy over different time–frequency scales;Step 3: To extract leak-related discriminant features from the AE images, the images are provided as inputs to the CAE and CNN to extract global features and local features, respectively. The local features represent the change in the energy at a finer level, and the global features are the overall acoustic signal characteristics in the AE image; Step 4: The global and local features are merged into a single feature vector. The single feature vector is provided as an input to the ANN for pipeline leak state identification under different leak sizes and pressures.The above steps are performed using CWT, CAE, CNN, and ANN, which are described in greater detail below.A CWT transforms a time-domain complex signal into the time–frequency domain with the help of the source wavelet function. The source or mother wavelet is usually a short time-based signal such as a vibrating signal having cycles . The motWhen applied to a real-valued pipeline AE signal, CWT results in a 2D transformation matrix. Each row in the transformed matrix represents one scale of the AE signal obtained from the pipeline, while the columns represent the translation or the pipeline AE signal size. Thus, these 2D transformation matrices can be represented in the form of an image called an AE scalogram. The color intensities of the scalogram show the maximum and minimum energy spreads of the wavelet in time and frequency according to the change in the pipeline’s working condition. In this study, a CWT with source wavelet Morse (symmetry parameter = 3) was applied to the pipeline AE signals and AE images are obtained. For complete details about Morse wavelet, readers are advised to refer to . The AE An autoencoder consists of three layers: the input, hidden, and output layers. Together, the input and hidden layers form the encoder, while the same hidden layer combines with the output layer to make the decoder part of the autoencoder. The main purpose of the encoder network is to learn the hidden representation from the input data, whereas the decoder uses these representations to reconstruct the input data. The encoder part of the autoencoder can be formalized using Equation (1):x and transforms it into latent representations, h, by mathematical operations in hidden layers. The weight matrix and bias vector of the bottleneck (hidden layer) are represented by W1 and b1, respectively, where a is the nonlinear activation function.The encoder part receives the input X is the reconstructed data, h is the input of the autoencoder, and W2 and b2 are the weight and bias vectors, respectively.The encoder part of the autoencoder can be formalized using Equation (2):While training the autoencoder, the reconstruction error between the input and output was minimized with the help of a loss function such as mean squared error. To achieve the minimum reconstruction error, the learned representation, known as latent coding, should be discriminative and of better quality. Considering the discriminative quality associated with the latent coding, in this study the autoencoder learned that latent coding were considered global features for pipeline leak state identification.The global features from the scalograms of the AE signals were extracted using a convolutional autoencoder. The CAE used the inherent property of compression in the autoencoder to extract certain useful features from which the image can be reconstructed. The CAE obtained the scalogram as an input. The encoder portion of the CAE consisted of a package of the convolutional layer followed by the pooling layer and, finally, a fully connected layer. The decoder portion of the CAE had a fully connected layer with a transposed set of convolutional layers that deconvolved the features of latent space . The encAs can be seen from Equation (3) mathematically represents the convT:mX is the output; mL is the convolutional matrix corresponding to layer m, which undergoes convolution and summation operations, transformed into vectors ms−1.Here, mL, needs to pass the transposed convolution [For the reconstruction of the input, the convolutional matrix, volution . The recmY having the same dimensions as mS−1.Here, the output of the transposed convolution layer is denoted by The local features from the AE images were extracted using a CNN. The CNN consisted of convolutional and pooling layers that were utilized to extract the important features from the input. A fully connected layer follows the convolution and pooling layers, which perform the flattening of features and, finally, an output layer.In the convolutional layer, several filters were used to convolve the input AE images. The AE images were 3D images; thus, the convolution process took place for each channel of the input image individually. The convolution process at each layer resulted in a feature map with the help of an activation function. The operation of the convolutional layer is expressed mathematically in Equation (5):m is the number of convolutional layers, (*) depicts the 2D convolutional operation of channel k = 1, …, mK−1, at the input of the convolutional layer Cth filter in layer m, and a is a nonlinear activation function. ma(.) is used to obtain the feature map. ReLU is used as a nonlinear activation function in this operation.In Equation (5), In this study, max pooling was adopted for the pooling operation. The main purpose of the pooling layer was to extract useful information considering the reduction in time and memory complexity. Equation (6) shows the general expression of the feature maps obtained using the pooling layer:down(.) stands for the downsampling process, Here, mx = vec(mx−1). The flattened feature map was passed to the fully connected layer. In a fully connected layer, the features were weighted. In this study, the fully connected layer was the output layer of the CNN. Equation (7) shows the mathematical representation of the fully connected layer:A feature map was obtained after the data were passed through the set of convolutional and pooling layers. The feature map was further flattened by mx is the output of the fully connected layer, a is the activation function, and mW and mb are the weight and bias, respectively. The output of the fully connected layer of the CNN were considered as the local features in this study. The architecture of the CNN used in this paper is presented in Here, The ANN classifies the input features into their respective classes. The ANN has three layers: the input layer, hidden layer, and output layer. The input layer takes the features as an input for the ANN. The hidden layer linearly transforms the input features using matrix multiplication operators. The output layer performs the segregation task using an activation function. This study considered the SoftMax function as an activation function in the output layer.Based on the input features, the ANN decides the pipeline’s condition. Here, we used a cross-categorical entropy loss function to classify the data. In this study, the ANN was used for classification; for this reason, the activation function was not applied to the input and hidden layers. The ANN can be represented using Equation (8):mX is the output of the mth layer, mx−1 is the output, and mW and mb are the weight and bias vector of the mth layer. Here, The experimental setup schematic and photos are provided in To evaluate the performance of the proposed method, a proper testing and training data configuration is important. In this paper, leak sizes of 0.3 and 0.5 mm were created in the pipeline, and data were obtained at pressures of 7 and 13 bar. For each pressure, a total of 240 samples were obtained from the pipeline. Thus, the data set contained 480 samples for both leak sizes, out of which 240 samples were normal samples and the remaining samples were leak samples. Seventy percent of the samples were selected randomly for training purposes, and the remaining 30% of the samples were used for testing the model. To ensure the repeatability of the results, the experiments were performed ten times on each data set.For feature extraction, the proposed method used a combined deep neural network, CNN–CAE, which extracted both global and local features from the AE images. The global features were trends and generalized features in all the scalograms, while the local features explored the leak information from the scalograms. The local and global features were merged into a single feature vector. Finally, the feature vector was provided to the ANN for pipeline condition identification. The metrics used for comparing the proposed method against the reference method were accuracy, precision, recall, and F1 score. These metrics were calculated using Equations (9)–(12), respectively:a; a; A depicts the total number of classes. N stands for the total number of data samples in the testing sets.Here, Masoumeh Rahimi et al.  used a dThe results obtained from the proposed method and the referenced methods are shown in The proposed model showed good classification results as well; to prove this we computed the ROC_AUC score which was 0.500 for normal conditions, while the leak condition had an ROC_AUC score of 0.9289. This paper proposed a novel deep learning-based pipeline leak detection approach. First, acoustic emission signals were collected from the pipeline under normal conditions and with different leak sizes. The acoustic emission images were obtained from the time series acoustic emission signal using continuous wavelet transform. To utilize the information in the acoustic images for pipeline leak detection, a novel deep learning framework was introduced. The novel deep learning framework was composed of a convolutional autoencoder and a convolutional neural network. The novel deep learning framework extracted a new set of local and global features from the acoustic emission images using a convolutional autoencoder and convolutional neural network. The new set of features was merged into a single feature vector. The merged feature vector was provided as input into the ANN for pipeline condition classification. In the experimental part of this study, the proposed approach was compared with a state-of-the-art method. The proposed method showed higher accuracies of 98.4% and 96.75% for different leak sizes. The proposed method can only detect leaks but cannot deliver any information about the leak localization. In the future, we will explore our proposed method in different locations of the leak with respect to the sensor and longer distances and leak localization of a pipeline.


This paper presents a very effective interference technique for the sensing and researching of compressible fluid flow in a wind tunnel facility. The developed technique is very sensitive and accurate, yet easy to use under conditions typical for aerodynamic labs, and will be used for the nonintrusive investigation of flutter in blade cascades. The interferometer employs a high-speed camera, fiber optics, and available “of-the-shelf” optics and optomechanics. The construction of the interferometer together with the fiber optics ensures the high compactness and portability of the system. Moreover, single-shot quantitative data processing based on introducing a spatial carrier frequency and Fourier analysis allows for almost real-time quantitative processing. As a validation case, the interferometric system was successfully applied in the research of supersonic compressible fluid discharge from a narrow channel in a wind tunnel. Density distributions were quantitatively analyzed with the spatial resolution of about  High-speed, high-accuracy, whole-field, non-destructive and contactless optical methods are increasingly being used in the study of fluid flow, offering the possibility for deeper insights than single-point conventional techniques such as constant-current anemometry , pneumatHowever, the field of the transonic or even supersonic flow generated in wind tunnel facilities is very specific, and the current state of the art of interferometric techniques in this field allows only their limited use. Environmental harshness, particularly vibrations, the lack of an optical table, temporal flow instability, and high fluid density gradients leading to complex interference patterns often make the quantitative evaluation of interferometric data very difficult and cumbersome. This is why the use of brute force interferogram analysis  or captuIn this paper, we introduce a robust, portable, high-speed and high-resolution interferometric method that can be easily applied in a wind tunnel facility, providing very accurate results with high spatial resolution in almost real time. The method utilizes a high-speed camera allowing for short exposures to avoid the blur of interference patterns typical for unstable flow, as well as to minimize environmental disturbances. Fiber optics ensure the high compactness and portability of the interferometric system, and fast single-shot quantitative data processing based on spatial carrier frequency and Fourier analysis is employed.The method is intended for the investigation of high-speed flow in planar blade cascades, namely, for studying dynamic events during transonic and supersonic blade flutter. As a first step, it was successfully tested in the case of a supersonic highly underexpanded flow at the outlet of a narrow channel by measuring the density distribution of the flow. The present interferometric technique has proven to be an excellent tool to meet these objectives. In addition to a description of the principles, experimental arrangement and results, numerical simulations showing the high reliability of the measured data are presented.Interferometric techniques generally provide whole-field, non-invasive and highly accurate measurements. These properties can be advantageously used for measuring and visualizing such flow phenomena that affect light wave . The ampchniques  that arenditions . For a cThe investigation within this paper leads to the characterization of the air flow in/behind a narrow channel see . Air entIn (6), Equations (2)–(6) provide the relation between the phase change The interference phase ce phase ; howeverce phase , which iF denotes filtered component) is no longer Hermitian, so its inverse Fourier transform will contain non-zero both real and imaginary parts, and hence the interference phase can be calculated as:Once the spectral components are separated, using a bandpass filter around the specific spatial frequencies, it is possible to filter out the desired spectral component The phase along with other quantities is stored in a computer’s memory as a matrix of numbers. Let us assume a measurement at a steady/reference state without the presence of fluid flow. Quantities measured in the reference state can be denoted using a subscript It is worth noting that spatial carrier interferometry could also be called off-axis image plane digital holographic interferometry. As phenomena in fluid flow do not lead to speckles, and there is also no need for the numerical propagation/focusing/reconstruction of wave fields typical for digital holography, we tend to call it interferometry.Interferometry-based research of a fluid flow in a narrow channel including a jet flow at its outlet was carried out at the Laboratory of Internal Flows of the Institute of Thermomechanics of the Czech Academy of Sciences. The laboratory is equipped with a modular in-draft wind tunnel facility that enables low mass flow rates. The vacuum tank is connected to the test section through a pipe with a quick-acting valve and a nozzle that offers the possibility of changing the pressure inside the testing section from The experimental arrangement was designed in order to maintain a constant refractive index along the The experimental setup, consisting of the narrow channel experimental section and a Mach–Zehnder interferometer, is illustrated in The intensity interference pattern (interferogram) of both superposed beams was captured by a Phototron FASTCAM Mini WX100 high-speed camera with a resolution of In the first step, a reference interferogram at a steady state  simulation software was used to cross-check both the measurement and the simulation. The numerical computations were performed using Ansys Fluent 2021R1 commercial code. This software package uses the finite volume method to solve the governing Navier–Stokes equations for compressible fluid flow. The problem was simulated in two dimensions (2D) using the Reynolds-averaged Navier–Stokes equations (RANS) approach to close the system of equations. The flow of air as an ideal gas was simulated as fully turbulent using the two-equation k-ω shear stress transport (SST) turbulence model. The computational mesh consisted of approximately The results of the interferometric measurement of air flowing out of the narrow channel, and their comparison to the results of CFD simulation, are presented in this section.The fluid flow in the narrow channel was investigated in five different regimes defined by different pressure gradients through the channel. The magnitude of the pressure gradient was controlled by a valve that determined the outlet pressure in the settling chamber downstream of the narrow channel. Both inlet and outlet pressure were measured, yielding the pressure ratio Other constants considered for further flow analysis and CFD simulations are the ambient temperature The density distributions computed using (5), together with the results of the CFD simulation, are presented in The measured distributions in Five different regimes with a gradually decreasing ratio In regime #2  due to the unstable behavior of the flow. Steady state repeatability has a negligible effect when compared to the stability of the flow. Therefore, time stability was examined. As aforementioned, we captured This paper presents a very effective technique for sensing and investigating compressible fluid flow. The developed interferometric technique is very sensitive and accurate, yet easy to use, even under pretty harsh environmental conditions  typical for aerodynamic labs. The interferometer employs a high-speed camera, fiber optics, and readily available “of-the-shelf” optics and optomechanics. The interferometer consists of an illumination and a sensing unit, each weighing less than The interferometer was successfully applied for the study of a fluid flow in a narrow channel including a free jet at the channel exit. The experimental section was built in a wind tunnel, allowing for a high pressure difference and thus high flow velocity through the channel. The flow was investigated under five different regimes, ranging from subsonic to supersonic flow with a highly underexpanded jet. The density distributions were quantitatively analyzed with a spatial resolution of about The measurement results were compared with CFD simulations, showing good agreement in terms of the flow pattern similarity  as well as the absolute values of density. An error analysis revealed the error range in units of percent. It can be concluded that the method constitutes a promising tool for studying highly dynamic events, even during transonic and supersonic blade fluttering.


The exponential growth of social media users has changed the dynamics of retrieving the potential information from user-generated content and transformed the paradigm of information-retrieval mechanism with the novel developments on the concept of “web of data”. In this regard, our proposed Ontology-Based Sentiment Analysis provides two novel approaches: First, the emotion extraction on tweets related to COVID-19 is carried out by a well-formed taxonomy that comprises possible emotional concepts with fine-grained properties and polarized values. Second, the potential entities present in the tweet can be analyzed for semantic associativity. The extraction of emotions can be performed in two cases: (i) words directly associated with the emotional concepts present in the taxonomy and (ii) words indirectly present in the emotional concepts. Though the latter case is very challenging in processing the tweets to find the hidden patterns and extract the meaningful facts associated with it, our proposed work is able to extract and detect almost 81% of true positives and considerably able to detect the false negatives. Finally, the proposed approach's superior performance is witnessed from its comparison with other peer-level approaches. Emotion Analysis helps us to understand and to retrieve the potential emotional information from any user-generated content. The user-generated content can be text, images, animations, videos, scan images, etc. But the text is predominantly used due to the emergence of social media platforms and other sources as well. Extracting emotions from the text document is a challenging task as it covers a wide range of ambiguities and anomalies that persist over the text content. Emotion Analysis has been categorized into different models, such as the corpus-based model, appraisal-based model, and knowledge-based model. The lexical affinity of a word or a phrase is determined based on the probability of emotion attached to each term and the cognitive factor will be inclusively considered for evaluation. For example, the term, “eager” takes a high probability score against “curious” than “willing.” To shun these difficulties, an ontology-based analysis has been used predominantly to disambiguate the terms and to yield a high precision score for the emotional terms.At first, the term, ontology was inhibited from psychology but it started making its footprint in computer science from the year 1980s. Ontology-based emotion extraction on text documents has been a research focus for many applications and it has gained huge momentum in recent years . As humaThe conventional Sentiment Analysis is normally applied to the text to determine whether the given text expresses a positive sentiment or a negative sentiment. Generally, the Sentiment Analysis has been carried out for knowing the feedback or the opinion of the customers about the products they had purchased. In particular, the following two Sentiment Analysis approaches were usually performed to get the sentiment score of the text documents: lexicon-based approach and machine learning-based approach. The first approach segments the text into appropriate morphological lexicons and then extracts the opinion words which are usually expressed either positive or negative. A dedicated dictionary that has been established for opinion words is called a lexicon dictionary and mapping of the opinion words on the lexicon-based approach has been done based on this lexicon dictionary. But in this case, tweets are not considered because of the underlying fact that tweets are within 140/280 characters and normally, words in a tweet are condensed into canonical forms due to their limited length. The machine learning-based approach has now gained huge momentum for many research activities and in that case, a specialized sentiment classifier has been trained for the textual corpus to predict the polarity of the text. Typically, the sentiment classifier has trained the model based on bigram or n-gram textual representation and classifies the text into the respective sentiment scores. The major drawback of this approach is the manual labeling of the training datasets and it has not been worked out well for tweets or any short messages.The ontology-based approach proposed by Ali et al.  indicateThe primary objective of this research work is to quantify the efficient classification of the extracted emotional entities from tweets using the proposed emotional ontology. Following are the outlined objectives of the proposed work:The domain entities have been classified with appropriate upper ontology classes and it measures the correctness concerning the metrics given in the Emotion Ontology.Selecting and quantifying some of the general domain emotional entities from the tweets that have significant relevance in the upper ontological class of Emotion Ontology.Determining the word embedding and taxonomy-based similarity measures for the extracted emotional entities.Generating relevant descriptions for those extracted emotional entities and assigning the relevance score based on the results obtained from the SPARQL query.The rest of the paper is organized as follows: Section Related Works summarizes the existing works based on semantic similarity measures, ontological features, emotion analysis, and sentiment analysis of social media content. Section Proposed Approach for Ontology Creation highlights the important features of Domain Ontology and semantic enrichment of entity categorization. Section Utilization of Emotion Ontology describes the core aspects of designing the semantic-based emotion ontology and taxonomy-based entity classification and disambiguation. Besides, it also explains the importance of ontology modularization for partitioning the large-scale ontologies into some self-contained modules. Section Conclusion delineates the polarity calculation and determines the performance of emotion ontology for the fine-grained measures.Extracting emotions from the documents is a tedious task as it involves a wide range of subjects, such as psychology, anthropology, society, and biology. In psychology, emotion is defined as the study of different orientations of human beings and the theories of emotions have been represented using cognitive psychology. Even though many cognitive models had been evolved to denote the emotions incurred in the documents, the most widely implemented affective computing models are dimensional , categorLater, Lacy  had deliLater et al.  have extThe emergency of social media has created a huge opportunity for social users to interact and discuss a wide range of topics prevalently happening around the world. These discussions paved the way for analyzing the significant interest of the social media users over the topics and performing sentiment analysis, such as emotion detection and recent opinion mining. According to Ren and Hong , emotionThe researchers, Baldoni et al.  and Dey Earlier, open information extraction (OIE) has gained huge momentum in extracting the entities and relation sets from the documents. OIE is a full-fledged extraction framework Daniel et al.  for bothMost of the semantic similarity measures attempt to emulate the human ability to evaluate the level of relativeness between the words according to their semantic evidence. According to Madani et al. , semantiAt first, the proposed approach deals with the creation of Domain Ontology for the bio-medical-related subjects to provide an accurate sentiment score for the opinion words present in the COVID-19 tweets. Here, the use of Semantic Web technologies gains a tremendous prospect to create Domain Ontology for the specific subjects, and further, the opinion words from the subject have been enlisted for the ontology with appropriate properties and literals. The Domain Ontology plays a seminal role in supplementing the feature extraction process to yield the precise sentiment score for the COVID-19-related tweets and increase the accuracy rate of the results. The proposed system is divided into two crucial phases. (i) Creation of Domain Ontology for the selective subject and (ii) feature extraction of opinion words included in the Domain Ontology. Normally, the Domain Ontology can be created either by utilizing the existing off-the-shelf ontologies or developing the ontologies based on the suitable requirements. There were many off-the-shelf ontology resources available for specific domains in the Semantic Web Forums that paved a way for the researchers to utilize these resources to map the properties of the ontology and opinion words . Many soFormal concept analysis (FCA) is a mathematical model that has been mostly applied in knowledge representation and information extraction. The sole objective of FCA is to pave the way for creating the Domain Ontology by a step-by-step approach and by considering all the ambiguities that persist on the properties of the Domain Ontology. With the advent of recent Semantic Web Technologies, the OWL helps to derive the potential objects and their properties and establish the Domain Ontology for accurate knowledge representation.The fundamental building block in FCA is the concept that can be derived from two valuable sets: Extension and Intention. The Extension is a set of objects collected for the Domain Ontology in considering the potential opinion words listed for the specific subjects. The set of objects has been chosen based on some grounding rules formulated for creating the ontology. The Intention is a set of attributes for every selected object and each attribute has some inherent properties and literals to represent in the hierarchy of domain levels. Every object that represents the concept has its attributes in the intention and likewise, each attribute linked to the concept of the Domain Ontology can be further shared with all the other objects of the Extension. This mutual connection that exists between the objects and attributes is described through the mathematical representation called Formal Context. A Formal Context is a triple that can be represented as K . Here O is a collection of objects for the Domain Ontology, A is the collection of potential attributes of the objects, and I is a binary correlation that exists between the objects and attributes; I ⊆ O × A, where  ∈ I .Ontology Learning or ontology acquisition is the inbuilt process of automatically inheriting the concepts and their associated properties from the other ontology resources or datasets . This cavia FCA and Ontology Learning and it enriched the taxonomy of a set of concepts and its potential attributes. To boast the semantic representation of the concepts, the ontology has now been filled with appropriate synonyms and hyponyms of the corresponding attributes. For instance, the term, “apple” comes under the class, “fruit” as well as “phone”; further, their hierarchy extends above to “vegetables” for “fruit” and “electronics” for “phone.” The synonyms and hyponyms can be fetched through the well-formed lexical corpus called WordNet , and then considered the words which are deemed to be opinion words after verifying them with the opinion-lexical database. The pre-processed tweets are then added to the second phase of the Sentiment Analysis and here we matched the opinion words which are also called entities in the Domain Ontology. The mutual mappings between the opinion words in tweets and their equivalent attributes in the Domain Ontology pose an ambiguity and create a separate list of ambiguous entities for further disambiguation. For logics and inferences possessed in the Ontology, we have used the first-order logics as well as description logics to express the relations given in the Emotion Ontology.The ambiguous list of entities is then processed for entity disambiguation which is the crucial task of this research work. The core idea behind the disambiguation process is that the mapping between two objects with dissimilar attributes in the Domain Ontology is identified first and then a comparison of properties and literals of two objects can be measured using semantic similarity-based algorithms i.e., Word2Vec and disambiguate the entities which hold the highest similarity score. The fundamental principle of ontology lies in disambiguation so that the original sense of the attributes can be obtained only at the leaf nodes of the hierarchy and hence the semantic similarity measures, such as Word2Vec have been selected to delve deep into the hidden layers of the neural network to get the precise results for the target words. Algorithm 1 illustrates the Emotional Ontology-Based Entity Extraction and Weight Calculation process.According to Li et al. , ontologBasic Emotions , Mid Emotions , Intense Emotions  and Complex Emotions . The Emotion Ontology helps to reduce the complexity of the redundant emotion population and also paved the way for detecting the emotions represented in the tweets. Besides, Ontology minimizes the non-emotional terms, which would be very large in Twitter Streams.Eight emotions were selected based on the recommendation given by Ekman  and obseThe Domain Ontology has been created manually with the help of a psychological-emotional words list and WordNet. But when it comes to the reusability of several classes belonging to different domains, the manual population of ontology would be a hard task. Further, it would lead to duplication of many more properties and their values. Therefore, effective utilization of Semantic Web technologies, such as RDF/RDFS, SPARQL, and OWL would primarily facilitate the mutual connection that exists over other domain ontologies, such as the one given in the Linked Open Data . In thisIn recent times, machine learning techniques have been used to automatically annotate the text documents and to train the classifiers according to the needs of the ontological requirements and domain hierarchy , 4, 29. Dictionary.com, Thesaurus.com, and Oxford Dictionary for increasing the probability strength of the words. The intensity of the words resulting from these dictionaries would be considered and populated for our Emotion Ontology.To gain a high recall score for the emotion extraction from the tweets, we have also crawled for slang words and out of vocabulary (OOV) words present in the tweet. The slang words and OOV words were disambiguated and we generated the correct term for the slang words using online slang dictionaries, such as NoSlang Dictionary, Urban Dictionary, and Translit . Further1. To extract the emotions, we have applied the binary Support Vector Machine (SVM) classifier to effectively identify the eight emotions as proposed by Ekman . Another dissimilarity observed on WordNet Affect is that it focused mainly on bigrams and trigrams for normalization instead of phrases. This would not be largely amicable for tweets due to their constraint in length. However, for the effective utilization of feature extraction for the proposed research, we used the topic modeling approach to discover the hidden terms and phrases between the tweets even though the tweet has no entities in common. Here, we have employed the LDA algorithm for topic modeling and it discovers the similarity that exists among the collections of tweets . For implementing this approach, we have enforced MALLET-based LDA implementation and treated every tweet as its document . This apThe ultimate objective of this proposed approach is to calculate the polarity of every tweet that we have collected from COVID-19 datasets. In the conventional methods, the sentiment score will be calculated based on the word polarity and the overall strength will be estimated on the grounds of the polarity score obtained from the probability of the positive and negative words. But in this proposed approach, we have modeled a novel approach to get the polarity of words on every tweet by using the SPARQL query that crawls through the emotional ontology to fetch the correct sense for the search word. In our proposed model, we have excluded the pronouns, adjectives, and articles from the tweets and extracted the tokens such as nouns, verbs, objects, and determiners. As the Semantic Web gives a well-defined meaning to the sentences and converts the sentences into appropriate triples, such as SPO, we have followed the same pattern for our proposed approach for the Emotion Ontology building process and extracted the nouns (subjects), verbs (predicates), and objects (objects) that satisfy the Semantic Web triple pattern, respectively. The relation between the entities (either noun or object) can be expressed using the RDF/RDFS. The ontology population can be done either by the tool Protégé or using the Semantic Web language, OWL. For logics and inferences possessed in the ontology, we have used the first-order logics as well as description logics to express the relations given in the Emotion Ontology. Mostly, we have utilized the predicates for relationships between two or more entities and the predicate relationships can be expressed using the OWL predicate type method. For each tweet, we execute the SPARQL query to obtain the disambiguated result. The SPARQL query to fetch the entities embedded with emotions is given below:The Emotion Ontology helps to disambiguate the search terms accurately and to return the results precisely. Thereby, the polarity and sentiment score for each term or entity can be obtained and yield an accurate sentiment score for the tweets. Suppose, the search terms used for the execution in the SPARQL query return NIL results, then the context of the token would be considered and an appropriate parameterized SPARQL query will be executed against the Emotion Ontology to fetch the right mix of words for consideration. We then estimated the sentiment score by considering the largest domain value from the list of positive tokens and the smallest values from the other set of negative tokens. Eventually, to determine the polarity of every tweet, the tweet sentiment score is calculated based on the strength score of the tweet, and the strength score of every entity in the tweet is calculated based on the following calculations: (a) positive, if sentiscore is greater than 1, (b) negative if sentiscore is less than or equal to −1 and (c) otherwise neutral.Further, to assess the absolute performance of our proposed Emotion Ontology, we have considered two implicit functions, such as sklearn.metrics.recall and sklearn.metrics.f1 to calculate the potential values, such as recall and F1 score, respectively. These two metrics were used to handle the multi-label classification . Our sheer Emotion Ontology population using Semantic Web technologies and performance measures selected for sentiment score has enabled the proposed method to avoid manual annotation using any machine learning techniques, and the problem of manual annotation has been resolved. Further, many recent studies have sought the help of machine learning algorithms to manually annotate the entities given in the text or any tweets. The proposed Emotion Ontology can deal with dynamic domain vocabularies that would possess different representations in the text.From the COVID-19 datasets retrieved from Twitter, training was strenuously performed over all the sample tweets, and validation was carried out with the baseline gold standard datasets. The total number of occurrences for each emotion listed was calculated by the number of positive values on the respective emotions and identified from class imbalances over the emotions, such as joy and love. To measure the accuracy of the proposed system, it has always relied on the precise calculation of the confusion matrix and mostly it is to obtain a low score over false positives (FP). Our proposed work is able to extract and identify almost 81% of true positives (TP) but it has encountered some serious challenges in detecting the false negatives (FN). This result has been vividly depicted in Eventually, each core emotion that was extracted from the tweets have been analyzed in terms of precision, recall, and f-score, and the details are provided in Our proposed model considers every emotion with its associated context returned from the Emotion Ontology, and the sentiment score has been calculated based on the sum, average, and maximum count of emotion score returned by the Emotion Ontology. It has been witnessed in the analysis that our proposed system has extracted the emotional words and their associated context through our proposed Emotional Ontology. Our proposed model has been compared with some of the benchmarked baseline approaches to highlight the major differences established in terms of F1-score. Some of the limitations that have been witnessed in the existing approaches have been delineated in The proposed Emotion Ontology model has been robust and efficient in extracting the full range of human emotions pertaining to COVID-19-related concepts. The emotions related to COVID-19 have been obtained and several emotional expressions, such as intensifiers, negator, lexical features, interjections, and conjunctions were considered for populating the Emotion Ontology. Semantic Web technologies have been utilized for creating the triples for every tweet, and the ontology construction has been manually done with the use of OWL functions. Compared to other lexicon-based emotion methods and machine learning approaches, the performance of the proposed system is satisfactory. As the tweets have been considerably informal in nature and have sparse content, in particular, the other lexical-based model and machine learning approaches failed to yield the desired sentiment score and ended up with some satisfactory precision and recall score. Our proposed study is able to extract and detect almost 81% of TP and is able to detect a considerable amount of FN. Since this study mainly focused on dealing with the English language, there is a certain limitation for extracting few emotions that exist in other languages, such as French, Spanish, etc. For future work, we will consider extracting the profound availability of emotional occupancy of resources in other languages and level their accuracy on par with standard metrics defined in the baseline approach.The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.SN and C-YC did the conceptualization and supervised the research. C-YC carried out the funding acquisition. SN investigated the data, performed the methodology, and implemented the software code. KS and C-YC carried out the project administration and validated the results. SN and KS wrote the manuscript. SN, KS, and SM reviewed and edited the manuscript. All authors contributed to the article and approved the submitted version.This research was partially funded by Intelligent Recognition Industry Service Research Center from the Featured Areas Research Center Program within the framework of the Higher Education Sprout Project by the Ministry of Education (MOE) in Taiwan and Ministry of Science and Technology in Taiwan (Grant No. MOST 109-2221-E-224-048-MY2).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


The paper demonstrates that the dry clutch friction plate wear rate, measured based on the plate mass difference method, exhibits a transient behavior after each change of friction interface temperature level. The effect is hypothesized to be caused by a temperature-dependent change in the moisture content/mass level in the friction material. To test this hypothesis, a series of synchronized characterization experiments have been conducted by using two friction plates, one for wear tests and the other for drying in an oven under the same temperature conditions. Based on the analysis of test results, a moisture content compensation procedure, which reduces the transient wear rate from being 100% to being 50% higher compared to stabilized wear rate, is proposed and verified. The gained insights are used to set recommendations on the organization of routine wear characterization experiments aimed at avoiding the effect of moisture content influence on the accuracy of wear measurement. The main recommendations are to minimize the number of temperature target level changes through proper design of the experiment, insert a run-in test after every long test pause, and execute a pre-heat, blind wear test at the beginning of each test day. Dry clutches have recently been considered for implementation in different hybrid ,2 and fuw) as a quotient of the worn volume wV and the friction-interface dissipated energy dissE:The term dry wear relates to several types of interactions between two sliding bodies and subsequent loss of material ,11,12. BVarious experimental characterization studies have shown that the wear rate for clutch and brake friction materials is affected primarily by temperature ,14,15, bTherefore, realistic characterization tests aimed at producing an accurate wear model should involve the full friction plate sliding over a disc cut from the real clutch pressure plate or flywheel. In other words, a full-size disc-on-disc tribometer should be used rather than a pin-on-disc tribometer. Such a tribometer has been designed in  and usedd) and the known friction plate surface (frA):Equation (1) suggests that the characterization of wear rate involves the measurement of the difference in worn volume for a given increment of dissipated energy. As described in , when thm) and the known density of friction material (ρ):The second approach is based on sensing the difference in mass , matrixIn this paper, an experimental analysis of the possible influence of a change of the three aforementioned operating parameters on the wear rate transient behavior has first been presented. It is demonstrated that the wear rate transient occurs only with respect to temperature, and the effect is presumably contributed to the influence of moisture content variation. Next, an experimental analysis involving synchronized tests of two friction plates is conducted, where the first plate is subject to wear tests on the rig, and the second one is subject to drying in an oven under the same temperature conditions as on the rig. The test results are used to check the hypothesis of moisture influence on wear characterization, based on designing and applying a procedure of compensating the raw wear rate values based on the dried plate mass difference measurement results. The synchronized tests involving two plates at a time should be avoided in routine tests, as they are time- and resource-consuming. Therefore, the insights gained through the analysis are employed to recommend the proper organization of single-plate wear tests to mitigate the moisture content variation influence.The main contributions of the presented study are: (i) experimental analysis of air moisture influence on the accuracy of the dry clutch friction plate, mass difference-based wear rate measurement, and (ii) recommendation of wear test organization to mitigate the moisture influence.The disc-on-disc-type CNC tribometer  under idling operation (no clutch friction load), (2) bottoming the vertical axis, (3) ramping up the normal force zF , (4) keeping the normal force at the target level zF2 (related to target torque level zM2) until the slip speed drops to zero , and (5) lifting up the vertical axis and imposing a cooling delay needed to control the friction interface temperature at the target level .Each experiment consists of a large number of clutch closing cycles  under a m) measured before and after each wear characterization experiment, and the energy dissipated (dissE) during the experiment. The weight measurement was conducted by means of a scale by Mettler Toledo, model JE3002GE . Due to hygroscopic features of the friction material, the air moisture content may considerably affect the results. To mitigate the moisture influence, the following operational procedure was applied: (i) the friction plate was stored in a closed box with silica gel bags whenever it was not used in testing, and (ii) the friction plate was dried in an oven at 100 °C for 1 h at the beginning of each workday, and the weight was measured before and after the drying procedure. The post-drying weight measurement was used for wear rate reconstruction.According to Equations (1) and (3), the friction material wear rate is determined based on the difference in friction plate mass  multiplied by the sensor placement radius with respect to the main vertical axis  as:rfs; see :Mz = , a set The wear characterization experiments are typically conducted day after day, except for weekends or holidays. The wear rate response with respect to the change of operating point is shown in The wear rate peak occurring after the increase of temperature could not be caused by the friction plate surface oxidation effect because the pauses were either absent or short (2–3 days). Note that the temperatures were kept below extremely high temperatures where a fading effect would occur. The fading effect is manifested in a significant drop of coefficient of friction (COF)  that is The next hypothesis was that the wear rate transient was caused by a difference in moisture content in the friction material due to temperature change. The transition to a higher temperature level may cause moisture content to decrease, thus resulting in the increase of measured mass difference and consequently in the wear rate peak, as observed in In order to gain insight into the source of mass loss at high operating temperatures, a new set of experiments was arranged, where two friction plates were tested in parallel: the first one to be worn on tribometer at different temperature levels, and the second one to be exposed to the same temperatures but in an oven (no wear). The test results are presented and analyzed in the next section.Two new plates, designated as Plate C and D, were simultaneously tested according to the test plan illustrated in It is worth mentioning that apart from being exposed to the same temperature profile, Plates C and D were from the same shipment and were stored together from the end of the manufacturing process to the start of the experiments conducted. The tribometer rig is fully computer- and feedback-controlled, and it has been previously used for long-term experiments related to different aspects of wear characterization ,17,36. HComparative time response of cumulative mass loss of friction plates C and D are presented in The results shown in The details presented in The mass loss of Plate C includes contributions of both the mass of worn-down friction material volume and the moisture and eventually resin mass loss due to evaporation when exposed to high temperatures. The mass loss of Plate D is related solely to the moisture and resin evaporation and may, thus, be used to compensate for the same effect of Plate C and determine, more accurate, net wear rate results.The results in rigm(i), blue line in dr,Tm(i)) with the stabilized/steady-state mass loss at the same temperature level , and then dividing the result by the factor of two to account for the fact that only a half of the friction plate is exposed to high temperatures on the rig. Hence, the compensation law reads  but not on the rig (Plate C). It can be readily shown that if the stabilized dried plate mass loss were not subtracted, the compensated rig-tested plate mass loss (and thus the wear rate) would not show a distinct increase with temperature. Namely, unrealistic overcompensation would occur, which confirms the assumption on non-existing resin evaporation under the real clutch testing/exploitation conditions. The moisture-related correction, shown in Both uncompensated , while the other plate was dried in an oven . Both plates were from the same production batch and exposed to the same temperature change schedule and generally to the same (plate storing) conditions during day-to-day and weekend pauses.A procedure that compensates for the apparent moisture content influence when reconstructing the wear rate has been applied to the rig-tested friction plate data based on a correction drawn from the oven-dried plate data. It has been shown that the observed wear rate transient effect is suppressed after applying the proposed compensation procedure by approximately 50%; thus, indicating that the transient effect can indeed be predominantly explained by the moisture influence.(i)design the experiments in the way that the operating points to be considered are grouped in sets of equal target temperature (other operating parameters should be randomized);(ii)record a run-in set of experiments after a long pause or after the temperature operating point change, which would last until the transient wear effect ceases; and(iii)introduce a ‘blind’ wear rate experiment at the beginning of each workday and set its target temperature to the level of the next, real test to heat up the pressure and friction plates and further mitigate the moisture effect.However, the moisture effect compensation approach is not practical for routine, long-time wear characterization tests, as it requires conducting synchronized tests involving two friction plates. Instead, it is recommended to solely reorganize the rig tests in the following manner, with the goal of suppressing the temperature-caused moisture content variation effect:Future research on the topic might include wear characterization using small samples of the friction materials and dedicated pin-on-disc rigs, which would have greater measurement precision and better control of ambient conditions, including air temperature and humidity. This activity could provide deeper insights into the physical background of the wear processes, as well as the influence of sample size and shape on the wear characterization results.