The accurate severity classification of a bug report is an important aspect of bug fixing. The bug reports are submitted into the bug tracking system with high speed, and owing to this, bug repository size has been increasing at an enormous rate. This increased bug repository size introduces biases in the bug triage process. Therefore, it is necessary to classify the severity of a bug report to balance the bug triaging process. Previously, many machine learning models were proposed for automation of bug severity classification. The accuracy of these models is not up to the mark because they do not extract the important feature patterns for learning the classifier. This paper proposes a novel deep learning model for multiclass severity classification called Bug Severity classification to address these challenges by using a Convolutional Neural Network and Random forest with Boosting (BCR). This model directly learns the latent and highly representative features. Initially, the natural language techniques preprocess the bug report text, and then n-gram is used to extract the features. Further, the Convolutional Neural Network extracts the important feature patterns of respective severity classes. Lastly, the random forest with boosting classifies the multiple bug severity classes. The average accuracy of the proposed model is 96.34% on multiclass severity of five open source projects. The average F-measures of the proposed BCR and the existing approach were 96.43% and 84.24%, respectively, on binary class severity classification. The results prove that the proposed BCR approach enhances the performance of bug severity classification over the state-of-the-art techniques. With the constant expansion of modern software, its reliability has become questionable, as these programs are prone to many problems and even failure. Software bugs are one of the enduring issues in the software development process. These are created in software during the development process by a programmer’s mistake, such as memory flow issues, run-time issues, and deviations from the pre-coordinated running directions . These sIn the case of numerous bug reports, the manual severity classification is a very dull, laborious, and time-consuming task. Manual bug severity classification is performed by utilizing the bug report content (summary and description). Therefore, bug severity classification techniques that automatically classify the bugs according to comments have generally been utilized by developers to dispense their limited resources.On the other hand, the bug severity also acts as a dispute between user and developer which requires prompt consideration and resolution. It is a fundamental characteristic that gives general advantages to the industry, which requires earnestness in its resolution with respect to resources. It proves that 80% of bug reports goes through severity reassignment, even after the bug report has been sent to a developer . AdditioAutomatic bug severity classification can be formulated as a classification problem using the bug report content. However, the information (content) in the bug report has semantic and syntax structure and comes with feature representation and non-linearity issues, which previous feature extraction techniques have failed to resolve. The classifier learns the feature representation of the corresponding class label during the training phase. Previous studies have used the “bag of words” or term frequency for feature representation, which misclassified the model ,13,14. TThis motivates us to utilize the deep learning model. This model enhances the generality by using a high number of kernels on the filter. In spite of this, the deep learning technique has proven to be a powerful technique for automatic feature extraction and has multiple layers within the deep architecture. It can capture highly complicated and important non-linear (latent) features. It also classifies more complex data rather than shallow techniques by utilizing the intermediate multiple hidden layers . To utilLearning the latent features and semantic representation of features from the text of the bug report, preserving the word order, and removing the sparsity and overfitting of data are challenging research problems in bug severity classification. Thus, this paper proposes a deep learning technique called Bug Severity classification using a Convolutional Neural Network and Random forest with Boosting (BCR), which can learn the feature representation directly without the need for manual feature engineering. It also captures semantic and non-linear feature mapping of the bug report text. Firstly, the text of a bug report is preprocessed followed by application of the n-gram technique to extract the n-gram, and then the CNN is applied to extract the latent and high representative features. Eventually, the random forest with boosting is applied to classify the multiple bug severity classes. The results prove that the proposed BCR approach enhances the performance of bug severity classification over the state-of-the-art techniques .Whether it is feasible to implement automated multi-bug severity classification model using deep learning technique and what the best configuration of Bug Severity classification is via Convolutional Neural Network and Random forest (BCR);Whether the proposed algorithm outperforms the Traditional machine learning algorithms in bug severity classification.This research paper seeks to address the following research questions:To the best of our knowledge, we have taken the first step to build the multiclass bug severity classification model via deep learning and random forest for open source projects;The n-gram technique is employed to generate the features from the text, which captures word order and the characteristics of different severity classes in five open source projects;A CNN-based feature representation framework is proposed to automatically generate the semantic and non-linear mapping of features, which preserves the semantic information of the bug report text;The CNN and random forest with boosting acclimates for severity classification instead of the traditional machine learning algorithms. The evaluation results show that the proposed algorithm outperforms the other algorithms in binary and multiple bug severity classification.This paper utilizes the following key contributions: The present study is categorized into six parts. Bug severity classification is an active research field in software engineering (SE). The spotlight of previous bug severity classification algorithms has been mainly on the manual arrangement of new discriminative techniques and the combination of features from the labeled severity data of bug reports. These features are fed into the machine learning classifier to detect severity. Commonly, the severity is classified into binary (BC) and multiple classes (MC).Zhou et al.  proposedJindal et al.  used theThe literature review posits that various feature extraction techniques, such as TF-IDF, n-gram, In-fogain, HAD, BOW, and KL-divergence, were utilized. The selection of the feature extraction technique depends on engineering design, and it is a challenging task because the best feature representation for collecting data depends on the feature model. In this paper, this challenge is addressed, and a CNN algorithm has been designed. It precisely learns the best feature representation from the bug report data in a supervised fashion. The NLP applications and n-gram technique are used to capture the word order, which helps improve the performance.In this section, the proposed BCR approach is elaborated. This framework automatically generates the features from the content of bug reports for accurate bug severity classification. The overall framework of BCR is illustrated in The bug reports with a title, description, and summary are considered for the input, and the code snippets, URLs, hexcode, and stacktraces of the bug report are removed;As seen in The bug report dataset is divided into training and testing with 10-fold cross-validation to avoid the training biases. The supervised classifier is trained with the features that are extracted by the FECNN module in the training part. In the testing part, the extracted features are used by the trained classifier for severity classification.The problem of automatic bug severity classification is defined as the supervised classification algorithm. The preprocessing step aims to remove the unnecessary words from the content of the bug report. The words that are not useful for analyzing the data are removed because their presence can degrade the learning performance. The feature set space is reduced by removing unnecessary words, making it effortless for learning techniques, so as to execute data analysis . This ink−1 and characterizes the sequence of i features into unigram, bigram, and trigram.During feature extraction, the content of the bug report is represented as a vector of feature (word) counts and helps in expanding the network by transforming bug report content into several sets of n-gram data. Hence, the features are expressed in a meaningful way. The n-gram technique is implemented to capture the semantic relationship and calculate the frequency of the feature order . The n-gIn Equation (3),We assume that in the unigram, the next features do not depend on each other and features in the feature string do not have any mutual information. Therefore, the conditional probability of the unigram is as follows:In Equation (4), In the bigram, two contiguous features (another specific feature follows one feature) have language information. The conditional probability of the bigram is given by:The trigram assumption is similar to the bigram; three contiguous features are considered for conditional probability and expressed as:In Equation (6), K+1)th is the feature, Different n-gram models should be combined to exploit its full capacity. The different n-gram models are employed to analyze an individual sentence, and subsequently, analyses of the results are combined. Therefore, the relationship between each n-gram feature is addressed as a word-level analysis as follows:The main operation of the convolutional layer is scanning of the sample data space with various kernels. These kernels demonstrate the various concerned features. CNN can reduce the data size by extracting the highly complicated and important non-linear features. Therefore, the convolutional layer is designed to compute 2D convolutional features. It is divided into a further five layers, which are the Input, Convolutional, Activation, Dropout, Max pooling, and Fully connected layers, as shown in i-th dimensional feature corresponding to j-th feature in the sentence. The representation of the full sentence with length n is:The Input layer contains the pre-processed data and the n-gram representation of the sequence features. These features are fed into the convolutional layer to generate new feature maps. Let The convolutional layer takes the input size [1000 × 200]. The FECNN model uses neurons with the following features: stride S = 1; depth K = 128; filter size F = 20; and no padding P=0. The output size of a CL is equal to ((input width–filter size)/stride + 1 = 90). Every 90 × 200 neurons are associated to a size (1 × 32) of the input region. Furthermore, all 200 neurons in each depth column link to the same (1 × 32) of the input region, with different weights.The convolutional layer is composed of convolutional kernels that produce feature maps; it is followed by the max pooling kernels. The operation of the max pooling layer concentrates the feature map by reducing the size of the feature map. It is achieved by computing the max value. Hence, the valid information is passed from the previous to the next layer by activating the consecutive layers over complex features. Therefore, overfitting is reduced. The max pooling procedure executes reduction of the input dimensions to 1 × 32 × 200–1 × 1 × 200 and reduces each (1 × 32) feature map to a single number by exerting the average of all 1 × 32values.To reduce the co-adaption issue in the FCNN model, the dropout layer is introduced, so as to disconnect neuron connections between the connected layers at a specific dropout rate. Therefore, training samples are generalized efficiently by the network. In this research work, the dropout rate is 0.2.Additionally, the activation function is introduced to decide which neurons must be activated and if the receiving information is relevant or should be ignored. Thus, all the layers are non-linear in the neural network; an activation function computes each neuron value. In this research work, the sigmoid activation function is used in the first 4 layers and the tanh activation function is utilized in the last 3 layers.The fully connected layer neurons have the entire association with all activations in the previous layer. This is connected with the FECNN module, where feature vectors are combined to develop the feature space for the classification layer.In this paper, the CNN layers automatically extract the high representative and latent feature. However, in traditional CNN, the fully connected layer acts as a classifier, which computes activations and accumulation as the final decision. However, in this work, the Random Forest (RF) is used as a classifier, since it is more reliable, highly accurate, and more robust than the fully connected layer due to the number of decision trees and bagging strategies participating in the process. It also overcomes the overfitting problem because it utilizes the all prediction average, which cancels out the biases . The proi-th neuron in the output layer;E, every weight value can be adjusted https://www.powerthesaurus.org/in_accordance_with/synonymsby the E value in the network. The contribution of the weight’s component is stated by the partial derivative of E, which has the weight’s value v and is expressed as u of the corresponding neuron is calculated by using the sigmoid activation function and by using the chain rule,As described in the previous section, considering the feature, instead of training the RF classifier with raw  data of bug reports v is updated by using the gradient descent rule, as illustrated in the following equation:Moreover, the value of After the training of convoluting layers, all training data sets Algorithm 1: The Training Process of CNR Model Input: Training Bug Report Dataset BRD= Batch Size: BS; Iteration Time: ITOutput: CNRFB model cnrfb_modelfecnn_modelInitialize the FECNN model While The new BRD’ = // forward the signle and calculate the errorfecnn_model -> forward (BRD’)error = // back propagation the error and modify the weightsfecnn_model -> batch gradient decent (error)// get the features outputted by fully connected layerfecnn_model -> forward get features (BRD)rf_ classifierInitialize the random forest with boosting rf_ classifier with the extracted features// train the rf_ classifier -> Train cnr :// construct_Classify (sample S):fecnn. forward get features (S)rf ->classify is used as a classifier ,40. The In the equation, The key objective of the RF is to make all possible tress from the features with their respective severity classes, and the boosting approach provides the threshold value of the model for selection of the tree at the testing phase. This threshold value varies in the execution of the training phase, expressed as follows: F (x) represents the final threshold of the model and αt and ht represent the hyper parameter, which externally changes the threshold, and this output goes to sign (sigmoid function). The output of the RFB is considered as the final severity class of the bug report.In Equation (19), The platform of the experiment is a personal computer equipped with an Intel Core i5 (7th generation), with 8 GB of DDR4 memory and an NVIDIA GEFORCE GPU, and the operating system is Windows 10 with 64-bit, 2.70 GHz CPU. The CNN is implemented using Keras and TensorFlow, a Python library that bolsters easy implementation of CNN with the capacity of using both CPU and GPU to accelerate the network training. In this work, the five open source datasets are used  from difThe BCR model should be compiled for adequate training, which outlines the procedure of discovering the best weight sets to perform severity classification. The compilation of the BCR model requires the stipulation of the lost function that is employed to evaluate the weight set, the optimizer that is utilized to examine the different weights towards the network, and the metrics that is adapted to analyze the model performance. The loss function is similar to the metrics, but the outcomes that are evaluated by the metrics are not utilized for BCR model training, while the loss function is reduced by the optimizer to enhance the BCR model during the training. In this research paper, the cross-entropy has been used for loss function, and gradient decent for the optimizer and accuracy of metrics. After the efficient computation, the BCR model is compiled on training samples for the training procedure. The training procedure runs for 10 epochs and 150 iterations, which is stipulated by the epoch parameter. Further, the 150-batch size is used to evaluate the number of samples before updating the weights in the network. The errors and the trials decide the batch size and epoch. The feature extraction model and classifier training are generally offline processes and do not contribute to the testing time. Take as an example the Firefox dataset; the training time of the BCR is about 250 s per epoch. The entire testing and training time of the classifier is 300 s. After the training, a new bug report takes 14 milliseconds for classification (feature extraction + classification).The performance of the proposed BCR model was evaluated by the experiment on five different datasets, which are Mozilla, Eclipse, JBoss, OpenFOAM, and Firefox, which have a different number of severity classes, as shown in RQ1. Is it feasible to implement an automated multiclass bug severity classification model using deep learning techniques, and what is the best configuration of Bug Severity Classification via the Convolutional Neural Network and Random Forest (BCR).Approach: The experiment was conducted on five datasets by using 10-fold cross-validation to determine the capacity of multiclass bug severity classification with the same parameter settings. The method of experimentation is illustrated in Findings: It can be seen from the obtained results  that it RQ2. Does the proposed algorithm outperform the traditional machine learning algorithms in bug severity classification?Approach: The proposed deep learning model is compared with the existing machine learning model  based onFinding: The comparison is shown in The results are manifested using five bug repositories with distinct characteristics to assure generalization. Further, commercial BTS may accompany different patterns, and hence the BCR model outcomes may not be legitimately extensible to such bug repositories;Currently, in the BCR model, only the descriptions and titles of bug reports are examined. While these two-unstructured data are significant, as shown by the experiments, there exists other structured data that are required to classify the bug severity sufficiently;Generally, most of the bug reports are misclassified. Therefore, the experiment is conducted on manually labeled bug datasets from the actual paper. Although the actual paper proposes some rule sets to support their severity classification, errors might occur. These rules and labeling process depend on a personal perspective;Assuring that the statistical methods are reliable is yet another challenge. From this, the collinear variables are removed to confirm that the bug report data is not inflated. The statistical tests are applied, and the bug report dataset is adjusted by using stratified sampling;The other step is to assure that the BCR model does not experience overfitting, therefore the K-fold-cross-validation is applied, where each sample is used for both training and testing, thus providing a more accurate model.In this research work, there are specific threats to authenticate the validity of the BCR model results. The learning-based severity classification model has some common threats, while some of them are peculiar to our research problem, which are given below: Bug severity is a crucial attribute of the reported bug report, which is required to be resolved quickly after the reporting. All of the bugs do not relate to critical issues; some are frivolous and only require improvement. However, manual severity classification is often tedious and time-consuming. The convolutional neural network is successful in text classification; a new deep learning model has been proposed, called Bug Severity Classification via Convolutional Neural Network and Random Forest with Boosting (BCR).As per this model, firstly, the text of a bug report is preprocessed, followed by the application of the n-gram technique to extract the n-gram, then the CNN is applied to extract the latent and high representative features, and eventually, the random forest with boosting is applied to classify the multiple bug severity classes. The BCR model has been validated on five real datasets. The performance of the BCR is compared with the state-of-the-art technique for binary bug severity classification. The results show that the BCR model classifies the bug severity more accurately and effectively. The BCR model significantly improves the average f-measure in comparison to the recent work on bug severity classification. In the future, we will validate other deep learning approaches on more open source projects.


Finding optimal parametrizations for people detectors is a complicated task due to the large number of parameters and the high variability of application scenarios. In this paper, we propose a framework to adapt and improve any detector automatically in multi-camera scenarios where people are observed from various viewpoints. By accurately transferring detector results between camera viewpoints and by self-correlating these transferred results, the best configuration  for each detector-viewpoint pair is identified online without requiring any additional manually-labeled ground truth apart from the offline training of the detection model. Such a configuration consists of establishing the confidence detection threshold present in every people detector, which is a critical parameter affecting detection performance. The experimental results demonstrate that the proposed framework improves the performance of four different state-of-the-art detectors  whose Optimal Fixed Thresholds (OFTs) have been determined and fixed during training time using standard datasets. People detection is one of the main tasks of computer vision with applications in many areas such as video-surveillance or human-computer interaction. Such detection is difficult due to the variety of people appearance and pose, and its performance is also very dependent on the data used for training . ClassicThe information provided by a single camera is limited, so in order to monitor a wide area or to obtain more information from the different viewpoints of a region of interest, it is necessary to use more than one camera. For this reason, the use of several cameras is a common way of developing applications ,4, sinceUnlike the previous approaches, in this work, we propose to transfer the detections from one camera to another instead of just projecting all the detections to the common plane. In the state of the art, the information of the detections is usually projected in the ground plane at the point level (one point per detection) or at the mask level . The work presented in this paper considers the common plane to obtain the different camera views’ information, and it allows transferring  people detections from each camera to the other ones.By employing multiple cameras, the available viewpoints provide additional information that may allow overcoming the limitations of detectors applied to single camera views. However, determining the confidence of the information generated for each viewpoint and, therefore, the automatic parametrization remains a challenging problem. Traditionally, optimal parameters are determined and fixed previously during training time ,12,13,14In this paper, we also propose a method to adapt the configuration of people detectors automatically during runtime detection. Unlike generic approaches fixing confidence thresholds or approaches restricted to single camera limitations, this method adapts the detector’s threshold for each frame and camera. We consider generic threshold-based detectors, trained on standard datasets, making this proposal applicable to most state-of-the-art people detectors.This paper is organized as follows: The frame-by-frame detections of all cameras are extracted, transferred, and homogenized (the position and volume of the transferred detections between cameras must be corrected) to the desired viewpoint The homogenized detections from the previous stage are correlated frame by frame, and an optimal decision threshold is selected for each camera and frame. The correlations are computed for each pair of transferred detection results  segment of the detection bounding box is projected to the common plane, the ground floor plane in our scenario. This plane can be obtained using homographic techniques, or from the intrinsic and extrinsic parameters of the cameras. We use the base segment as it is in the common plane , which allows accurately transferring it. Using the projected segment in the common plane, a circumference is defined so that the projected segment forms one of the sides of a square inscribed therein. In To define the bounding box base segment that will be transferred to the other camera, the inscribed square (blue) is rotated  or from the intrinsic and extrinsic parameters of the new camera. An example of the resulting cylinders and transferred bounding boxes is shown in We apply a method to improve the detection performance at runtime by adapting the detector configuration see . This prN camera frames. Each detector obtains a confidence map in every camera, We start from a set of The correlation is only coherent to be carried out in the common field of view of all cameras, since otherwise, disjoint sets would be correlated and the process would not be useful. To locate the common field of view, the ground plane of each camera is transferred to the desired point of view. Visual examples of this process are shown in http://www.cvg.reading.ac.uk/PETS2009/a.html) presents outdoor sequences from a typical surveillance setup. We consider the available ground truth, which improves direct transfer cameras’ combination results of bounding boxes from [https://cvlab.epfl.ch/data/rlc) was recorded at the EPFL Rolex Learning Center using three static HD cameras. The complete ground truth was not available, so we manually annotated the bounding boxes of the detections for the first 2000 frames of Camera 1 (cam1). We make this ground truth publicly available upon request. Both datasets were calibrated using the Tsai calibration [The objective of this section is to evaluate the presented framework in order to validate the detection improvements. Two datasets, which contain overlapping multi-camera environments, have been considered for the evaluation. The PETS 2009 dataset , Recall (R), and F-score (F) metrics for each frame. We considered the mean F-score for all sequence frames as the final performance value. In order to evaluate the improvement of the framework, we compared the results with the absence of threshold adaptation, i.e., the use of an optimal threshold determined and fixed previously during training time. In our experiments, we learned offline this Optimal Fixed Threshold (OFT) with the training dataset VOC2012 .In order to evaluate the performance, we always transferred all the cameras to the reference point of view of cam1, since it was the one with the annotated ground truth. The results obtained after evaluating the four detectors on the sequences of the two datasets are presented in In addition to these improvements, the proposed approach avoided the critical task of selecting a static threshold fixed offline during training time for every possible scenario. We were able to choose the optimal threshold online automatically in every frame with a minimum requirement of one detector and two cameras.According to the computational cost, each detector’s results had been obtained with the available code, implemented with different tools and programming languages, so a fair comparison was not possible. The computational cost of the detections was not treated in this paper, as this aspect was analyzed by the authors. The used DPM approach was implemented with MATLAB, and the computational cost was about 2 s per frame, considering an image of 352 × 288 pixels. The ACF detector was implemented in MATLAB, and the computational cost was about 20–30 ms per frame with 352 × 288 images. The used faster RCNN approach was implemented with MATLAB and Caffe, and the computational cost was about 150–200 ms per frame , considering an image of 500 × 375 pixels. The used YOLO9000 approach was implemented in C and CUDA (GPU), and the computational cost was about 25–35 ms per frame, considering an image of 500 × 375 pixels.Our experiments have been performed on a Pentium i5 with a central processing unit with a frequency of 2.6 GHz and 8 GB random access memory. The proposed approach included two main tasks: the detection transfer and the correlation stage. Both stages were implemented in MATLAB. We present a framework to choose the optimal people detector threshold automatically during runtime. The proposal accurately transfers detector results between camera viewpoints and then exploits the correlation among multiple camera detections transferred to a common camera to determine the best threshold for each camera. The proposed approach is capable of working over standard state-of-the-art detector outputs (bounding boxes), so any kind of detector and object model can be considered. The cylinder model may need to be adapted in other cases in which the object has a very unbalanced length-width aspect ratio . This framework allows the automatic threshold parametrization without requiring any model (re-)training process and, therefore, is completely online.For future work, more object detectors can be considered. Other additional optimal parametrization can be considered and not only the detection threshold; for example, the position of the bounding box, the scale of detected objects, the pose, etc. Furthermore, following , multipl


In this research, a security paper based on frequency selective structure technologies was designed and fabricated using selective wave reflection characteristics to prevent the offline leakage of confidential documents. Document leakage detection systems using security papers detect security papers using transceiving antenna gates. For the application of such systems, the structure must be designed with excellent reflection performance and stability at the angle of incidence. For this purpose, a loop and patch-type frequency selective structure based on a four-legged element structure was designed to have X-band frequency reflection characteristics. This design was based on optimized variables and was realized through the screen printing method using silver ink on A4 paper. It was verified that both the design and simulation results matched well. To verify its actual applicability, a detector module operable at 10 GHz was manufactured to observe both the security paper detection range in relation to distance with a signal strength of −10 dBm and the detection area in relation to the number of times that the security paper had been folded. In today’s rapidly changing information-oriented society, much effort has been devoted to securing national competitiveness through the development of unique technologies. However, as advanced technologies become connected to national core technologies, the risk of technology leakage is growing. The outflow of core technologies causes enormous losses by damaging the national economy and hindering industrial development. In particular, in the case of the national defense industry, the outflow of core technologies constitutes dangerous criminal acts that threaten the security of the nation. Moreover, leaked industrial secrets are irrecoverable and difficult to compensate. Therefore, it is necessary to actively work to develop systems related to the protection of domestic industrial technologies ,2.In most cases, industrial secrets are directly or indirectly leaked with the help of insiders through various routes and media. Leakage cases carried out through online routes such as e-mail and messenger services can be tracked and used for criminal investigation after leaks have occurred. To prevent the occurrence of such leaks in advance, electronic document systems optimized for effective information management and security have been introduced through the digitization of information. However, the risk of hacking or data replication has not been totally mitigated. Therefore, documents with special purposes, such as design drawings or documents having legal effects, are still produced and kept in the form of paper documents.Since the outflow of paper documents through offline routes such as fax and mail allow criminal acts to be stopped before confidential information is leaked, offline routes are the focus of efforts to prevent confidential leakage through in-advance detection. Thus, the development of a system to prevent offline data leakage is required. To prevent data leakage in offline environments, papers with security functions, such as uncopiable papers, copy-prevention papers, and electronic surveillance papers are being used ,4.Among these, the electronic article surveillance (EAS) system equips papers with electromagnetic induction sensors so that the outflow of papers can be detected at antenna gates located at the exits when printed documents are taken outside of the security area due to theft or leakage, causing an alarm to sound. These systems are divided into the electromagnetic (EM) type, which uses papers with magnetic induction materials, and the radio Frequency (RF) type, which uses papers equipped with radio frequency identification (RFID) tags. The EM type uses tags made of soft magnetic materials inside the paper, allowing the detection of harmonics generated by magnetic field changes caused when the paper passes through the alternating magnetic field created by the detector . In 2002In this research, to develop a microwave-based electronic surveillance paper, a security paper was created based on frequency selective surface (FSS) production technologies using cost-effective printed electronic technology. FSSs have a structure that acts as a space filter due to the periodic arrangement of conductive materials on a dielectric substrate, and they are used for indoor frequency control, radomes, and wave absorbers ,10,11. TIn this research, to develop security paper to prevent the leakage of industrial technologies in offline environments, an A4-paper screen printing method using silver ink was applied to create electromagnetic-wave security paper that operates in the X-band. The frequency reflection properties of this security paper were measured, and its applicability was verified through testing using detectors. The location and status of papers that pass through antenna gates are unpredictable. To enable their detection, patterns must be designed that have excellent frequency reflection characteristics and are stable to the angle of incidence and for the polarization of incident waves. The frequency selective structure is designed with a loop structure to secure the incidence-angle stability through miniaturization based on a four-legged element structure, which is known to have narrowband characteristics . The linThe proposed FSS was fabricated by screen printing on an A4 paper using silver ink with a surface resistance of 0.02 Ω/sq. During printing, it was verified that the paper, which was made of fiber, was not conductive because the low-viscosity conductive ink permeated the paper. To solve this problem, a high-viscosity silver ink of 28,000 cPs was used at 20 °C. Here, 1 cPs is the viscosity of water at 20 °C. A screen printing technique optimized for the use of high-viscosity ink was used. After printing, post-processing including heat, UV-ray, and nitrogen treatment, was carried out to increase the ink’s preservability or durability. Further, to improve the surface resistance of the fabricated security paper, it was heat cured for 20 min at 150 °C. The fabricated FSS is shown in The frequency reflection characteristics of the fabricated security paper were measured in the free-space measurement environment shown in To create a system in which an alarm sounds when the security paper passes through an antenna gate, a detection system with a detector module equipped with a patch antenna and an LED was created, as shown in The transmitting antenna receives signals from the signal generator and sends them to the FSS. Then, the receiving antenna receives the reflected signals from the FSS and verifies the detection area through the LED lights of the detector module. The signal generator applies a signal strength of −10 dBm to the transmitting antenna, and the detector module receives power of 158 mA from the power supply at 6 V, and a power of 150 mA from the LED at 5 V. To clearly observe the LED lights, a multimeter was connected to the LED terminal to verify the output of 5 V. To verify the detection area according to the distance of the security paper and to prevent leakage of folded paper with the detector module system, the detection area was observed according to the number of times the paper was folded. The security paper was folded in half five times for the test. The paper size and number of unit structures in relation to the number of folds are tabulated in In this research, a security paper was designed and fabricated using frequency selective surface techniques to prevent the outflow of industrial technologies in offline environments. To detect paper passing randomly through antenna gates, a four-legged element structure was designed as a loop and a patch structure to realize a paper with excellent reflection performance and stability to the angle of incidence. Considering the characteristics of security papers, a screen printing technique was used to create the security paper using silver ink on A4 paper. Measurements of its frequency reflection characteristics demonstrated that the fabricated FSS displayed similar results to those obtained by simulation, and the security paper was detectable using the detection system within an average distance of 25 cm. In the future, security paper equipped with a frequency selective structure will be created using laminating technologies to prevent damage of the frequency selective structure exposed on the paper surface.


Wearable sensors may enable the continuous monitoring of gait out of the clinic without requiring supervised tests and costly equipment. This paper investigates the use of a single wearable accelerometer to detect foot contact times and estimate temporal gait parameters . The experiments considered two possible body positions for the accelerometer: over the lower trunk and inside a trouser pocket. The latter approach could be implemented using a common smartphone. Notably, during the experiments, the ground truth was obtained by using a pair of sensorized shoes. Unlike ambient sensors and camera-based systems, sensorized shoes enable the evaluation of body-worn sensors even during longer walks. Experiments showed that both trunk and pocket positions achieved promising results in estimating gait parameters, with a mean absolute error below 50 ms. Spatio-temporal gait parameters can reveal important information related to health and well-being. For instance, some studies have shown that abnormal gait is linked with a higher risk of falling, and gait analysis has been proposed for automated fall-risk assessment ,2,3. OthIn recent years, there has been a significant effort in the development of automated techniques to help clinicians achieve a more objective assessment of gait in a controlled environment. Optoeletronic systems and force platforms represent the state-of-the-art in this field. Nevertheless, these techniques typically require a relatively complicated setup, based on costly equipment and/or multiple inertial sensors. As a consequence, subjects are still required to visit a clinic and perform predefined tests under the supervision of an expert. Wearable sensor-based systems have also been studied as a potential alternative for the analysis of gait parameters ,13,14,15single wearable sensor to continuously monitor gait parameters out of the clinic. The motivation is that the use of a single device is key to obtaining an unobtrusive solution and fostering user acceptance in the real world. The contribution is to demonstrate that a single accelerometer can be used to estimate temporal parameters, such as the duration of gait cycles and the duration of phases within a cycle, during unconstrained walks. Hence, it would be possible to constantly monitor the trend of gait parameters over time, enabling early detection of changes that may require clinical attention. Unlike the examples described in Reference [single tri-axial accelerometer for the detection of gait events—heel-strike (HS) and toe-off (TO) as initial and final foot contact during gait—and the consequent estimation of the duration of gait cycles (stride time) and gait phases (stance and swing duration). We considered and compared two alternative body positions for the sensing device: over the lower trunk (close to the L3 vertebra) and inside a front trouser pocket. To the best of our knowledge, the use of a device placed inside a trouser pocket (a common position for smartphones) has not been explored in previous studies. Notably, the results obtained were compared with the ground truth provided by a pair of sensorized shoes. Unlike standard reference systems based on optoelectronics instruments or force platforms, the use of sensorized shoes as reference enables gait analysis even during longer and unconstrained walks. Therefore, the proposed experiments represent a first step towards the evaluation of wearable-based gait analysis in free-living conditions.This paper aims to investigate the capability of a eference ,17, our The proposed technique for gait analysis with a sensor over the trunk was inspired by the techniques presented in previous works ,19. The This article is an extension of the preliminary study we presented at the EAI MobiHealth 2017 conference [Wearable sensor-based systems have been proposed as a convenient tool to monitor gait and other activities in uncontrolled environments ,22,23,24The work presented in Reference  was amonAn interesting evaluation of five different methods ,28,29,30A more recent work  evaluateAn approach based on sensorized insoles to validate body-worn accelerometers is found in Reference . The forAs previously mentioned, the majority of studies from the literature relied on ambient sensors to provide the ground truth, such as optoelectronic systems. This approach provides high accuracy, but does not enable the evaluation of foot contact detection during longer and less constrained walks.gait cycle is defined as the interval between two consecutive heel-strike (HS) events of the same foot (stride time. Gait is further characterized by the instants at which a foot leaves the ground and starts “swinging” forward: this is known as a toe-off (TO) event . For each leg, a gait cycle is characterized by two phases: stance  and swing .The proposed techniques aim to detect foot contacts in order to estimate the main temporal characteristics of gait cycles: the stride time and the duration of the stance and swing phases. A ame foot . The durThe ground truth for our experiments was based on a pair of FootMoov 2.0 shoes and two Shimmer3 devices, shown in As described in the Introduction, we studied the use of a single tri-axial accelerometer worn at two alternative body positions  to estimate stride time and stance and swing duration. In particular, we employed Shimmer3 devices . Note that we have not applied any calibration procedure  to aligIn the following subsections, we first describe the algorithm used to estimate gait parameters using the sensorized shoes. Then, we describe how the same parameters are estimated using the trunk and pocket sensors. Finally, we describe the experimental procedure.1); (ii) the minimum force value on the heel is below a threshold (TH_TO2). Conversely, a HS event leads to the stance state and is detected when the minimum force value on the heel is above a threshold (TH_HS). Thresholds were determined on separate experiments by using a high-frequency video trace (240 fps) as reference. According to our experiments the estimation accuracy of the shoe-based method is mostly limited by the sampling frequency of the in-shoe electronics (50 Hz), showing a mean error of ∼20 ms.The algorithm to detect foot contacts is described by the finite state machine in An example of force sensor signals during gait is shown in The proposed algorithm is an enhanced version of the technique we presented in Reference . InsteadThe method to find temporal gait parameters at the trunk position proceeds as follows. First, gait cycles are identified using the walking detection algorithm presented in Reference . WalkingThe interval including the group of acceleration peaks produced at each step is then used to search for HS and TO events in the AP signal. More precisely, HS events correspond to a local maximum, whereas TO events correspond to a local minimum in AP acceleration.sensor steps. AP acceleration is analyzed to discriminate between sensor steps and the steps made with the contralateral leg: sensor steps show an average AP acceleration above a threshold determined experimentally. Sensor steps are used to detect HS events by finding the local maximum value on AP acceleration, whereas contralateral steps are used to detect TO events by finding the local minimum on AP acceleration.As in the trunk method, the walking detection algorithm described in Reference  is appliThe example in Three volunteers were recruited for data collection: their main characteristics are shown in The force sensor signals on the shoes were used to find stride time and the duration of stance and swing for each gait cycle and for each foot. The same parameters were estimated at the trunk and pocket position using the respective methods and acceleration signals, enabling direct comparison with the shoe-based ground truth.As part of the procedure, we checked that all the methods detected the correct number of foot contacts (HS and TO events). This was verified by comparing the results with the video traces, which also allowed us to verify that no “false steps” were detected and no real steps were missed at the beginning and at the end of walks, when movements are typically less regular.A first comparison of the results obtained through different approaches is contained in To further evaluate the agreement between estimations we performed Bland–Altman (BA) analysis, where the difference of two paired measurements is plotted against the mean of the two measurements. BA plots comparing trunk and pocket estimations with shoe-based estimations are shown in Despite using simple algorithms, the proposed methods were able to detect foot contacts without producing false positives. This is an essential requirement for accurate gait analysis. A glimpse at the results in A low estimation error is confirmed by the analysis of the MAE, presented in Further evidence of the agreement between Shimmer-based and shoe-based estimations is provided by the BA analysis. All the plots show that errors are randomly scattered within the range of agreement, without suggesting a clear relationship with the magnitude of the estimated parameter. In other words, according to our experiments, estimation accuracy is not influenced by walking characteristics such as cadence and stance/swing ratio. Indeed, the proposed methods obtained promising results on all subjects, despite their different pace, age, and health condition. The limits of agreement are in line with the errors reported by other works in the field of gait phase detection for IMU-based approaches ,38, withBA plots also highlight the presence of a limited number of outliers, especially for the pocket approach, with some estimations showing an error in the order of 0.1 s. These sporadic errors can be explained considering that, inside a trouser pocket, the sensor is subject to more spurious accelerations during gait. For similar reasons, pocket also shows a relatively greater range of agreement for stride estimation. Nevertheless, the pocket approach managed to achieve similar results with respect to trunk in terms of mean difference, as shown in Overall, these results are very encouraging and support the hypothesis that an unobtrusive approach based on a single wearable sensor could be used to continuously monitor temporal gait parameters during everyday life. The use of a sensor carried in a trouser pocket is the most innovative aspect of our work. Despite the more challenging positioning (the orientation of the sensor changes during the swing phase), the results of this novel approach suggest that a pocket-worn device (like a smartphone) may be used to estimate mean gait parameters with high accuracy. We envision that such an unobtrusive approach could be used to continuously monitor some key gait parameters out of the clinic, enabling prompt detection of changes in gait patterns that may require medical attention. This could be of paramount importance for patients with a higher risk of developing conditions affecting motor ability, such as older adults. One specific example is frailty syndrome: in this context, early detection enables the adoption of preventive measures to reduce the impact of this chronic condition on the patient’s quality of life.Some limitations of our work should be mentioned. First, we tested our algorithm only on level walking conditions, without considering typical situations such as walking on irregular ground or climbing a flight of stairs. To the purpose of monitoring the evolution of gait parameters in the long term, one possible solution could be to filter out these scenarios, for example exploiting a barometer to detect stairs and a filter based on autocorrelation to disregard gait on irregular terrain .Another important limitation is the reduced number of subjects involved, and the fact that we did not involve patients with an impaired gait, such as frail subjects. However, this study was conceived as a preliminary evaluation of a simple and easy-to-use approach based on a single wearable sensor. The results related to the pocket approach, in particular, will enable extensive studies based on common smartphones. In future studies, we plan to significantly extend the number of subjects involved, and evaluate whether the use of personalized models could help further reduce estimation errors.The pocket-based technique can only estimate the parameters related to the leg which is carrying the sensor. This does not represent a problem in symmetric gait, where both legs show similar parameters. Further experiments will be required to evaluate how to improve the algorithm to cope with patients with highly asymmetric gait. For example, previous works reported that double support (DS) duration is highly correlated with frailty . DS is tFinally, in our method, it is supposed that one of the reference axes of the accelerometer is roughly aligned with the AP direction when the user is standing still. Despite this limitation, we still believe that our results are relevant. Indeed, because of its form factor, a smartphone is typically placed in a front trouser pocket with the screen orthogonal to the AP direction. Hence, the axis orthogonal to the screen is roughly aligned with AP during gait, as in our assumption. Additionally, it is promising that despite the lack of accurate calibration procedures our method managed to obtain low estimation errors. Nevertheless, in future works, the use of automated calibration techniques  will be In this work, we have evaluated the use of accelerometers placed at two different body positions, over the lower trunk and inside a trouser pocket, as a means to estimate temporal gait parameters. Notably, we relied on sensorized shoes to obtain the ground truth during experiments. This enabled the evaluation of longer walks with respect to ambient sensors like cameras or instrumented mats. Results are very promising and show that both approaches (trunk and pocket) are able to provide an accurate estimation of stride time and swing/stance phases. The method for a device placed inside a pocket is novel, and suggests that a common smartphone may be used as an unobtrusive solution to continuously monitor key gait parameters out of the clinic.In future studies, we plan to perform extensive experiments to further investigate the use of common smartphones for gait analysis, possibly including older adults with impaired gait in the experiments. We also plan to perform similar tests with a wrist-worn device (like a smartwatch), which could represent a further step towards unobtrusiveness and ease of use. In the long term, we plan to apply these unobtrusive gait analysis methods to the detection and monitoring of medical conditions affecting gait, such as frailty. In this context, the wearable-based approach may significantly reduce the burden of these conditions on both patients and clinicians, by enabling continuous and automated monitoring of gait during everyday activities.


Various microphone array geometries  have been used to improve the positioning accuracy of sound source localization. However, whether these array structures are optimal for various specific localization scenarios is still a subject of debate. This paper addresses a microphone array optimization method for sound source localization based on TDOA . The geometric structure of the microphone array is established in parametric form. A triangulation method with TDOA was used to build the spatial sound source location model, which consists of a group of nonlinear multivariate equations. Through reasonable transformation, the nonlinear multivariate equations can be converted to a group of linear equations that can be approximately solved by the weighted least square method. Then, an optimization model based on particle swarm optimization (PSO) algorithm was constructed to optimize the geometric parameters of the microphone array under different localization scenarios combined with the spatial sound source localization model. In the optimization model, a reasonable fitness evaluation function is established which can comprehensively consider the positioning accuracy and robustness of the microphone array. In order to verify the array optimization method, two specific localization scenarios and two array optimization strategies for each localization scenario were constructed. The optimal array structure parameters were obtained through numerical iteration simulation. The localization performance of the optimal array structures obtained by the method proposed in this paper was compared with the optimal structures proposed in the literature as well as with random array structures. The simulation results show that the optimized array structure gave better positioning accuracy and robustness under both specific localization scenarios. The optimization model proposed could solve the problem of array geometric structure design based on TDOA and could achieve the customization of microphone array structures under different specific localization scenarios. In the past two decades, microphone array technology has consistently been a hot research field. Microphone arrays are mainly used for sound source localization and identification, and have been an important practical technology with many valuable applications, such as noise source localization ,2, targeIn microphone array technology, there are three main methods for sound source localization, namely, beamforming, acoustic holography, and time difference of arrival (TDOA). The beamforming method applies delay-and-sum to signals from an array of microphones, and in the direction of the source, a beam peak will form to locate the sound sources . AcoustiDifferent numbers of microphones and different kinds of array structures are used in these three methods. In general, the number of microphones used in beamforming and acoustic holography is much larger than in the TDOA method because the number of microphones has a significant influence on the reconstruction accuracy of the sound source mapping . NeverthMany kinds of microphone array structures are applied in sound source localization, which can mainly be divided into three categories: 1-dimensional, 2-dimensional, and 3-dimensional. The 1-dimensional array structure group mainly comprises binaural arrays  and lineFrom the above, it can be seen that a great deal of research work has been done in the field of microphone array optimization for sound source localization. However, these array optimization methods are mainly based on the beamforming and acoustic holography methods. The optimization procedure is also usually based on existing array structures, such as cross array , circle This paper is devoted to an arbitrary microphone arrays optimization method for sound source localization based on TDOA. The method proposed is a numerical approach based on the particle swarm optimization (PSO) algorithm, which can optimize the array structure of an arbitrary number of microphones under any specific localization scenarios without prior array structure information. Examples of localization scenarios were constructed to obtain the optimal array structures through the proposed method. Additionally the optimal array structures were compared with the array structures proposed by Hu et al.  as well This article makes four main contributions. First, a numerical approach of microphone array optimization based on the PSO algorithm for the TDOA method is proposed. Second, the proposed model can perform array structure optimization with an arbitrary number of microphones, and no prior array structure information is introduced into the optimization procedures, which is likely to obtain the more optimal solutions. Third, the array optimization model has general applicability, which can effectively solve the problem of microphone arrangements in sound source localization under different specific localization scenarios. Fourth, the fitness evaluation function constructed in the optimization model can give good consideration to the accuracy and robustness of sound source localization based on TDOA. The two specific localization scenarios established here verify the proposed optimization method.In the following sections, the optimization model is introduced in detail, and the localization performance is compared with the array structures proposed by Hu et al. as well as with random array structures. The sound source localization model is the basis of the array structure optimization. The TDOA method was used to locate the sound source. Therefore, the time difference and the spatial geometric relationship between the array and the sound source were used to establish the localization model.In order to optimize the microphone array, the geometric structure of the array needs to be parameterized first. Because the sound source localization method in this paper is based on TDOA, a reference microphone is needed in the microphone array. For convenience, the coordinate of the reference microphone The coordinates of the other microphones are c is the speed of sound and The constraint of the azimuth angle The spatial source location model was constructed based on the TDOA method, which is a triangulation method. Suppose the coordinate of the sound source is By constructing the time arrival difference from the sound source to the reference microphone and the other microphones, the spatial source location model can be obtained. The model is shown as follows:Suppose that The spatial source location model consists of a group of nonlinear multivariate equations, which is difficult to solve. An alternative method is to transform the model into a set of linear equations. Spatial distance satisfies the relationship shown in Equation :(7)ri2= can be n and Ho  proposedIn order to solve the source localization model by least-square method, Equation  should bSuppose that the noise of TDOA estimation is In practice, the condition Equation  can be iEquation :(13)za=zWhen the source is close to the array, Equation  can be fx, y, and The above solution of Substitute Equation  into EquThe covariance matrix of Equation :(19)za′=The matrix Then, Equation  reduces The final sound source position is estimated as:Given a certain number of microphones, there are infinite spatial geometric structures for microphone arrays. Nevertheless, in various practical scenarios, it is necessary to find the optimal array structure to effectively reduce the positioning error of sound sources in the target area. Because the microphone array consists of multiple microphones and each microphone’s coordinates have three independent variables, it the structure optimization of the microphone array in this paper is a multidimensional optimization problem.The evolutionary algorithm is a global optimization method with high robustness and broad applicability. Unlike classic optimization methods such as gradient descent and quasi-Newton methods, the gradient of the problem being optimized is not required for the evolutionary algorithm. Meanwhile, the evolutionary algorithm makes few or no assumptions about the optimization problem and has great advantages in the application of unsupervised, complex multidimensional problems that cannot be solved using traditional deterministic algorithms . The genj-th particle, and the number of particles is set as A swarm of particles which traverse a multidimensional search space are employed in the PSO algorithm to find optima. Each particle is a potential solution and is influenced by the experiences of other particles, as well as its own experiences. Let A new fitness evaluation function for the array structure optimization is constructed by the mean squared error (MSE) and the variance (VAR) of the localization results, which can comprehensively consider the localization accuracy and robustness. The fitness function is shown as follows:In Equation , the mead-dimensional space.Then, an optimization problem (minimization) is defined as:k denotes the current iteration number, and The PSO algorithm is used to solve this optimization problem. To seek the optimal solution, each particle moves in the direction of its previously best  and (31), and maintain Step 5. If Step 6. Judge the termination criteria: Step 7. Output the The flow chart of the optimization procedure is shown in In order to verify the effectiveness of the method proposed in this paper, two kinds of sound source localization scenarios were constructed for microphone array optimization. One scenario was a ring-shaped sound source distribution, and the other was a cuboid sound source distribution. These two scenarios represent some specific sound source localization scenarios in practical applications, such as surround sound sources localization and road traffic flow noise sources tracking. For each specific localization scenario, two strategies of structure optimization were adopted to generate two kinds of optimal structures. In addition, the regular polyhedron microphone array structure proposed by Hu et al. as well as random array structures were used as a comparative study of the performance of sound source localization. The model established in In scenario I, sound sources were distributed in a cyclic annular band, here referred to as the ring-shaped sound source distribution. The distribution was controlled by Equation , as follThe source distribution for the array structure optimization under scenario I is shown in For the sound source localization based on TDOA, the time difference estimation error is the main influencing factor for localization accuracy. To facilitate the optimization and verification of microphone array structures, the TDOAs were directly obtained by calculating the relative position relationship between the sound sources and the array microphones. A noise component was added to the TDOAs, which was used to represent measurement noise in actual applications.Five microphones were selected to optimize the microphone array structures, which were compared with the tetrahedral structure array proposed by Hu et al. as well as a random array. One microphone in the array was chosen as the reference microphone, and the coordinates were set as N was set to 250. For the second kind of array optimization (Opt-array II), the radial distance N was set to 450 for Opt-array II.In this simulation, two optimization strategies were applied to search for the optimal array structure. For the first kind of array optimization (Opt-array I), the distances Then, the optimization model based on PSO ran on the Matlab platform to obtain the optimal array structure under scenario I. The fitness evolution curves of Opt-array I and the Opt-array II are shown in It can be seen from For the manufacturing of a microphone array, an angle precision of It can be seen from In order to verify the performance of the optimal arrays obtained in the simulation, a scenario with sound sources randomly distributed in the cyclic annular band was constructed, as shown in As shown in In Under four different input noise levels, the mean value and standard deviation of the localization error for the random array were all much larger than that for Opt-array I, Opt-array II, and the tetrahedral array, which illustrates that array optimization—whether the method of this paper or Hu’s method—produced a positive effect. The random arrays may achieve excellent positioning performance, but the possibility is tiny. Also, the mean value and standard deviation of the localization error for Opt-array II were lower than for Opt-array I. Considering that there were differences among the four radial distances In scenario II, the sound sources were distributed in a cuboid space band, here referred to as the cuboid-shaped sound source distribution. The cuboid space band is was 15 m × 6 m × 3 m. The microphone array was located on one side of the cuboid distribution. The location of the reference microphone coincided with the origin of the coordinate system. The constructed scenario II is shown in In the simulation of scenario II, five microphones were also selected to optimize the microphone array structure, which was compared with the tetrahedral structure array proposed by Hu et al. and a random array. The noise component The fitness evolution curve of Opt-array I and Opt-array II under scenario II are shown in It can be seen from It can be seen from In order to verify the performance of the arrays, the scenario of randomly distributed sound sources in the cuboid space band was constructed, as shown in In It can be seen from Under five different input noise levels, the mean value and standard deviation of the localization error for the random array were much larger than that for Opt-array I, Opt-array II, and the tetrahedral array, especially when the input noise component was large, which illustrates that array optimization produced a positive effect. Random arrays have little chance of achieving excellent positioning performance under specific localization scenarios. Also, the mean value and standard deviation of the localization error for Opt-array II were lower than those for Opt-array I, which indicates that the optimization of the radial distance In addition, compared with scenario I, the mean values and the standard deviations of scenario II were much larger. The standard deviation rose sharply with the increase of input noise. The main reason for this is that the location area and the size of the sound sources in scenario II were much larger than in scenario I, and the sound sources were asymmetrically distributed. For scenario II, increasing the number of array microphones may help to reduce positioning errors and improve positioning robustness. Therefore, another optimization case was applied in scenario II, which is that seven microphones were chosen for the array structure optimization. The octahedron structure array proposed by Hu et al.  and a raN for Opt-array-7mic I and Opt-array-7mic II were set to 400 and 650, respectively.Two kinds of optimization strategies were also used in the simulation. In the first kind of array optimization (Opt-array-7mic I), the radial distances The optimal array structures were obtained after running the optimization model on the Matlab platform under scenario II. The geometric parameters of the optimal arrays, the octahedron array, and the random array are listed in It can be seen from In order to verify the performance of the optimal arrays, the scenario of randomly distributed sound sources in the cuboid space band was constructed, similar to This paper proposed a method of microphone array optimization for sound source localization based on TDOA under specific localization scenarios, which can be applied to the optimization of arbitrary array structure without prior information. For any number of microphones, a more optimal array structure can be given under any localization scenario. The proposed method is a numerical approach based on the particle swarm optimization algorithm. The mean squared error and the variance of the localization results combined with a weight value are used to construct the fitness function of the optimization model, which can consider both positioning accuracy and robustness. The geometric structure of the microphone array was established in parametric form, which is assigned as particle attributes and substituted into the optimization model to obtain the more optimal results. Two specific localization scenarios were constructed to optimize the array structures. For both specific scenarios, two kinds of array optimization strategies were utilized to obtain two optimal array structures. The optimized array structures were compared with the regular polyhedron structure array under different input noise amplitude.For scenario I, the mean value and the standard deviation of the localization error for Opt-array I and Opt-array II were much lower than for the tetrahedral array and the random array, and the higher the input noise, the more significant the gap. Under four different input noise levels, the mean value and standard deviation of the localization error for the random array were the largest, and those of Opt-array II were the smallest. The results indicate that the array optimization produced a positive effect, and the optimization of the radial distance For scenario II, the mean value and the standard deviation of Opt-array I and Opt-array II were also much lower than those of the tetrahedral array and the random array. Under five different input noise levels, the mean value and standard deviation of the localization error for the random array were the largest, and those of Opt-array II were the smallest. The array optimization and the optimization of the radial distance For both specific localization scenarios, the comparison results show that the localization accuracy and robustness of the optimized array structures were better than those of the regular polyhedron array structures proposed by Hu et al. and random array structures, which illustrates the effectiveness of the proposed array structure optimization method. The random arrays may achieve excellent positioning performance, but the likelihood is small. The optimization of the radial distance


Considering the lack of precision in transforming measured micro-electro-mechanical system (MEMS) accelerometer output signals into elevation signals, this paper proposes a bridge dynamic displacement reconstruction method based on the combination of ensemble empirical mode decomposition (EEMD) and time domain integration, according to the vibration signal traits of a bridge. Through simulating bridge analog signals and verifying a vibration test bench, four bridge dynamic displacement monitoring methods were analyzed and compared. The proposed method can effectively eliminate the influence of low-frequency integral drift and high-frequency ambient noise on the integration process. Furthermore, this algorithm has better adaptability and robustness. The effectiveness of the method was verified by field experiments on highway elevated bridges. The dynamic displacement response of a bridge contains valuable structural behavior information, which can effectively reflect the structural behavior characteristics  and is often applied to various structural health monitoring (SHM) and structural control (SC) methods ,2. UnderDynamic displacement measurement methods are mainly divided into direct and indirect displacement measurement method. Traditional direct displacement measurements include linear variable differential transformers (LVDTs), laser Doppler vibrometers (LDVs), and vision-based systems. Although the methods have high measurement accuracy, they require a reference point, and in practical applications, it is often difficult to find a suitable reference point at the test site ,6,7. DifRegarding indirect displacement measurement methods, the accelerometer has been widely used due to its wide frequency band response, small size, easy installation, and low cost . The dynKandula et al.  proposedIn this study, the EEMD method was used to successively decompose and reconstruct the vibration acceleration signal, velocity signal, and displacement signal of a bridge, which effectively eliminated the interference caused by the high-frequency noise and low-frequency trend in the integral calculation process, and obtained the dynamic displacement signal that satisfied the calculation accuracy.Let the displacement be the second integral relationship of acceleration:In order to facilitate processing calculations in the computer, the integral discrete form uses a trapezoidal integral formula:Equation (2) is substituted into Equation (1) to obtain the displacement integral calculation formula:As shown in Equation (3), the displacement is related to the initial displacement and the initial velocity. The initial displacement affects the final displacement result in the form of a constant superposition, and the initial velocity is the result of the final displacement with time as a linear function. As time increases, the cumulative error increases. In addition, the low frequency interference caused by the MEMS accelerometer itself and the zero drift of the acquisition system is amplified by integration. Therefore, filtering processing such as low-pass filtering and smoothing filtering is usually used to reduce the influence of the abovementioned phenomenon. However, the limitation of the filtering process is too large, and the cutoff frequency is difficult to select. Therefore, an adaptive processing method is needed to reduce these interferences, and EEMD is an empirical data analysis method. It is decomposed according to the time characteristic scale of the signal itself, and each IMF component has different time feature scales. Therefore, it can be screened according to different time feature scales, thereby filtering out unnecessary components and achieving the purpose of adaptive filtering.The extreme point At any point, the mean of the envelope EEMD is developed on the EMD algorithm, which is the core algorithm of the Hilbert–Huang transform (HHT) . The purAdd a white noise sequence to the target data:Decompose the white noise-added data into multiple IMF components:Repeat steps (1) to (2), and add a white noise sequence different from (1) to the target data:i-th white noise sequence; i-th white noise sequence; Obtain the  mean of the corresponding multiple IMFs as the final result:However, there are problems and deficiencies in EMD. The IMF components obtained by EMD decomposition have modal mixing and an end effect. Modal mixing is reflected in different IMF components with the same scale information. At this time, the low frequency component may be mixed with the effective information, which makes the selection of the IMF component difficult, and the signal is seriously shifted after the integration. The endpoint effect is accompanied by each screening process, which gradually enlarges as the operation runs, thus affecting the entire data sequence. In order to overcome these problems, Huang et al.  proposedThe original signal is decomposed into multiple IMF components by EEMD. Each IMF represents an internal modal of the original signal. Therefore, we can select the modal components we need for analysis and processing, so as to achieve the purpose of adaptive data processing.In the integral process of acceleration to displacement, the effective IMF component is continuously selected by the EEMD algorithm for reconstruction, and finally the dynamic displacement is obtained. i-th white noise sequence with different frequencies; i-th white noise sequence is added; i-th white noise sequence; i-th white noise sequence is added; i-th white noise sequence is added; Where This section covers the experiments and analysis conducted on bridge analog signals, vibration table measured signals, and bridge measured signals.Generally, the frequency of the bridge dynamic response vibration signal is 1–10 Hz, so the ideal displacement signal constructed for this paper was:The acceleration The ideal acceleration signal was constructed in MATLAB, and a random white noise of Gaussian distribution was added on the basis of this, as shown in In order to obtain the ideal displacement signal, the acceleration signal processing with noise was processed by four methods: frequency domain bandpass filtering , EMD adaSince the bridge vibration analog signal frequency was 2.5 Hz, As shown in As can be seen from As shown in The principle based on the EEMD adaptive filtering method was similar to the EMD–based adaptive filtering method, except that EMMD was used instead of EMD. The flow chart based on EEMD adaptive filtering process is shown in The number of ensembles is set 200, and the level of noise added is set 0.2. These parameters are uniform in the article. As shown in The root-mean-square error (RMSe), kurtosis error (Ke), and phase error (Pe) of the abovementioned four methods with respect to the ideal signal were calculated separately, as shown in The RMSe and Ke of the EEMD adaptive filtering method were both below 1.5%, and the Pe was less than 2°, which could meet the calculation accuracy. Compared with the other three methods, the EEMD adaptive filtering method had higher precision for the integral processing of the low-frequency acceleration signal.A multi-channel vibration data synchronous acquisition system was used to collect the vibration acceleration signal of the vibration table, and the abovementioned four methods were used to reconstruct the dynamic displacement of the vibration table. The obtained displacement signal was compared with the displacement obtained by laser displacement sensor to verify the effectiveness of the displacement reconstruction method based on the EEMD adaptive filtering.The vibration acceleration signal was given by the vibration table. The vertical and horizontal two-dimensional variable frequency vibration control was realized by the vibration table frequency conversion controller controlling a pair of eccentric wheels to generate vibration simulation quantities with different dimensions and amplitudes. In the test, the vibration frequency was set to 8.7 Hz and the amplitude was 0.13 mm. The three-axis MEMS accelerometer sampled the acceleration of the vibrating table and transmitted the acceleration signal to the upper computer through the multi-channel acceleration acquisition system. The vibration table displacement reconstruction was realized by the abovementioned four methods on the upper computer platform, and different methods were compared and analyzed.From the time domain and frequency domain spectra in The root-mean-square error (RMSe), kurtosis error (Ke), and phase error (Pe) of the abovementioned four methods with respect to the ideal signal were calculated separately, as shown in The RMSe and Ke of the EEMD adaptive filtering method were both below 0.5%, and the Pe was less than 2.2°, which could meet the calculation accuracy. Compared with the other three methods, the EEMD adaptive filtering method had higher precision for the integral processing of the low-frequency acceleration signal.In summary, the displacement signal obtained based on the EEMD adaptive filter processing method was more accurate. From the acceleration to the speed, and then to the displacement, EEMD decomposition was used in order to eliminate the invalid components, which basically eliminated the effects of high-frequency noise, low-frequency trend terms, and DC components in the original shaker acceleration signal. The calculated dynamic displacement peak-to-peak value of the vibrating table was 0.26 mm, which was consistent with that given in the shaker manual. The validity of the reconstruction displacement of this algorithm was thus verified.The effectiveness of dynamic displacement reconstruction based on the EEMD adaptive filtering method was verified by software simulation and vibration test bench. The dynamic displacement response of bridges is often used for bridge health monitoring and early warning. Therefore, it is of great practical significance and value to provide a basis for bridge health monitoring by analyzing and processing the vibration data of bridges.This experimental test site uses a highway elevated arch bridge with a height of about 15 m and a length of about 800 m. By analyzing the force of the bridge structure, a three-axis MEMS acceleration sensor is installed at the top of the arc of the bridge. For arched bridges, the vibration at the top of the arc is most pronounced when the vehicle passes. In combination with the video recorded by the camera, the dynamic displacement response of the bridge caused by heavy trucks and large slag trucks was selected. At this time, the dynamic displacement of the bridge is large, and the structural behavior characteristics obtained from it are more obvious. According to the dynamic displacement of the bridge, vehicle type, driving speed, rough road conditions, etc., the fatigue stress of short-span and mid-span bridges under dynamic traffic loads can be modeled to evaluate the fatigue reliability of the bridge .The dynamic displacement reconstruction results of the bridge are shown below.As mentioned above, usually due to the influence of the frequency cutoff error, the results obtained by the FFT filtering method will have certain errors at the beginning or end. Since there is no vehicle passing the test point in the first five seconds and the last five seconds of the acceleration signal selected in As can be seen from the In order to verify the effectiveness of the proposed EEMD-based adaptive filtering method, the “ideal displacement signal” is compared with the intercepted signal through the EEMD-based adaptive filtering process, as shown in the It can be seen from the figure that the result based on the EEMD adaptive filtering process and the “ideal displacement” are highly coincident, which embodies the effectiveness of the proposed algorithm.In this paper, the EEMD algorithm was used to reconstruct the vibration acceleration signal to the displacement signal. The vibration acceleration signal, the speed signal, and the displacement signal were corrected in turn and, finally, the displacement signal satisfying the calculation precision was reconstructed. The effectiveness of reconstructing the dynamic displacement of the bridge based on the EEMD adaptive filtering algorithm was verified by software simulation, vibration test bench verification, and a field test. Compared with the frequency domain filtering processing method, the EMD adaptive filtering processing method, and the FFT+EMD filtering processing method, the superiority of the proposed method was demonstrated. Combined with video recordings, the violent dynamic displacement response of bridges was found to be caused mostly by heavy or large trucks. Limiting the number of vehicles passing over a bridge and the speed of the vehicles can reduce the fatigue damage of a bridge structure and provide good protection for the bridge. However, the method proposed in this paper also has certain deficiencies. Through the simple beam hammer test, the authors found that the proposed method is not suitable for broadband excitation signals. The team next plans to make some improvements based on the method proposed in this paper, hoping to realize the reconstruction of the broadband excitation acceleration signal.


A localization and tracking algorithm for an early-warning tracking system based on the information fusion of Infrared (IR) sensor and Laser Detection and Ranging (LADAR) is proposed. The proposed Kalman filter scheme incorporates Out-of-Sequence Measurements (OOSMs) to address long-range, high-speed incoming targets to be tracked by networked Remote Observation Sites (ROS) in cluttered environments. The Rauch–Tung–Striebel (RTS) fixed lag smoothing algorithm is employed in the proposed technique to further improve tracking accuracy, which, in turn, is used for target profiling and efficient filter initialization at the targeted platform. This efficient initialization increases the probability of target engagement by increasing the distance at which it can be effectively engaged. The increased target engagement range also reduces risk of any damage from debris of the engaged target. Performance of the proposed target localization algorithm with OOSM and RTS smoothing is evaluated in terms of root mean square error (RMSE) for both position and velocity, which accurately depicts the improved performance of the proposed algorithm in comparison with existing retrodiction-based OOSM filtering algorithms. The effects of assisted target state initialization at the targeted platform are also evaluated in terms of Time to Impact (TTI) and true track retention, which also depict the advantage of the proposed strategy. Detection and tracking antiship supersonic targets, such as sea-skimming missiles, are challenging problems for any countermeasure and surveillance system . VariousMeasurement acquisition rates differ widely depending on several variables. Therefore, when fusing data from multiple sensors, there is a mismatch in measurement arrival time; for example, data processed from an image processor will be more time-consuming compared to a radar signal being processed. Measurement delays occur due to sensor diversity, transmission delay in the communication network, and uncertainty in preprocessing times when large numbers of measurements are processed in limited resources . As a reNumerous OOSM filtering techniques have been proposed in the past to incorporate arbitrarily delayed target measurements in an effective manner. Although effective, most of these algorithms do not consider a cluttered environment. This is mainly because of the complexity that accompanies data-association techniques. An approach that takes advantage of the information filter recursions for the OOSM update is proposed in Reference , which il-lag OOSM in the framework of the B1 algorithm, known as the Bl algorithm. In the Bl algorithm, associated covariances are stored for the computation of filter gain from all past sampling intervals. In Reference  for LADAR and  for IR in the body frame. The target was launched from a distance of 50,000 m, and was moving with constant velocity of 1000 m/s, aimed toward the targeted ship. The ROS estimates the target position for 40 s of its flight. The remaining portion of the trajectory is estimated by the targeted platform with assisted state initialization. The standard deviation of both sensors is defined by For both velocity and position, the performance of both the proposed localization and tracking algorithm is compared with the optimal AThe measurements from both sensors arrive in such a way that measurements from LADAR are in sequence, whereas a random number of IR measurements are out of sequence with delays of one, three, and five lags. In l1 algorithm contains the effects of process noise; however, suboptimal algorithm Bl1 ignores the process noise during state retrodiction. For multiple-lag OOSMs, such as three and five lags, the RMS error value of both algorithms increases. These RMS error values are presented in l1 and Bl1 is presented for different lag values and based on the information presented in the table. It is clear that the computational complexity of Bl1 algorithm is less than that of the Al1 algorithm. For multiple-lag OOSMs, computational complexity value increases for both algorithms. From the simulation results, it is observed that performance estimation is identical for both algorithms; however, practical implementations and system requirements are also an important factor for algorithm selection.The experiment consisted of 500 Monte Carlo runs, with each run consisting of 391 scans. For velocity and position, simulation results of the suboptimal and optimal algorithms for both single- and multiple-lag OOSMs in a cluttered environment are presented in In the proposed algorithm, RTS fixed-lag smoothing enhancement is performed in the filtered estimates to further improve the estimation accuracy. In each simulation run, the fixed lag of four and seven scans was considered, and a batch of four and seven scans was formed. The eldest measurement scan was replaced with new measurement scan in each cycle. For a fixed lag value of N = 4, the measurement scans for the first and second batch are To limit smoothing delay, an RTS smoothing algorithm that works back in time was implemented in batch form. From the previous examples of fixed lag To evaluate the performance of the proposed algorithm with smoothing lag ) . Where VAfter the first two scans, any measurement falling in the validation gate of a track is not considered for the track-initialization process. When two or more tracks share a common measurement history for four consecutive scans, these tracks are merged. Similarly, a track is considered lost if the estimation error exceeds five times that of the measurement noise standard deviation.The simulation in case of the targeted platform tracking system consisted of only 30 scans with the Monte Carlo runs same as in ROS tracking case. A total of five false tracks were reported in both the assisted and nonassisted mode. However, true track retention was much better in the case of ATI, with a total of 50 lost tracks at the final scan as compared to 84 in the case of no feedback from the ROS, which accounts to a This paper considers the data fusion of a LADAR with IR sensor by incorporating OOSMs to estimate target trajectory and state dynamics of high-speed incoming sea-skimming missiles in a cluttered environment. Optimal and suboptimal OOSM filtering algorithms were incorporated in an NN filter framework to handle single- and multiple-lag OOSMs while tracking the target. A smoothing technique is embedded in the proposed algorithm to further improve the estimation accuracy for assisted track initialization, thus making the tracking system capable enough to deploy countermeasures at an early stage against various high-velocity incoming targets. Monte Carlo simulations clearly demonstrate the improved tracking performance of the proposed algorithm, with a considerable reduction in RMSE and more response time for the countermeasure system as a result. Track retention of the targeted plaform’s tracking system also improved from