Label noise is omnipresent in the annotations process and has an impact on supervised learning algorithms. This work focuses on the impact of label noise on the performance of learning models by examining the effect of random and class-dependent label noise on a binary classification task: quality assessment for photoplethysmography (PPG). PPG signal is used to detect physiological changes and its quality can have a significant impact on the subsequent tasks, which makes PPG quality assessment a particularly good target for examining the impact of label noise in the field of biomedicine. Random and class-dependent label noise was introduced separately into the training set to emulate the errors associated with fatigue and bias in labeling data samples. We also tested different representations of the PPG, including features defined by domain experts, 1D raw signal and 2D image. Three different classifiers are tested on the noisy training data, including support vector machine (SVM), XGBoost, 1D Resnet and 2D Resnet, which handle three representations, respectively. The results showed that the two deep learning models were more robust than the two traditional machine learning models for both the random and class-dependent label noise. From the representation perspective, the 2D image shows better robustness compared to the 1D raw signal. The logits from three classifiers are also analyzed, the predicted probabilities intend to be more dispersed when more label noise is introduced. From this work, we investigated various factors related to label noise, including representations, label noise type, and data imbalance, which can be a good guidebook for designing more robust methods for label noise in future work. The success of supervised machine learning algorithms in classification tasks is dependent on a massive amount of training data, in spite of some alternatives that are being developing for learning models, such as flew-shot learning ,2,3, or gray-zone was defined when intracranial pressure (ICP) value range between normal and great risk, samples in gray-zone were considered as unlabeled, and then a semi-supervised approach was utilized. A similar framework was also conducted for the false alarm detection in patient monitoring system [The major issue of label noise lies in the implication to classification performance of supervised learning algorithms. Several studies have investigated its potential impact and have shown that noisy labels can adversely affect the classification accuracy of trained classifiers . Howeverg system , which ig system . A majorg system ,15. In og system . AnotherIn this study, both random and class-dependent label noises were artificially introduced into the training dataset to emulate the errors associated with human fatigue and annotation bias, which are the main sources of label noise in medical data annotation. The focus of this paper was to assess the robustness of different types of AI-based models to the label noise and bring the discussion of this relevant issue in the automatic approaches to the medical domain, which are usually dependent on the annotated data and the difficulties inherent in the labeling process. We investigate the impact of random and class-dependent label noise on the performance of machine learning (ML) and deep learning (DL) models, undertaking a binary classification task assessing the quality of photoplethysmography (PPG) signal. PPG is a popular tool to inform distal circulatory status and has gained its popularity recently years in cardiovascular research due to its ubiquitous presence in consumer wearable devices . Its sigThis section describes the data collection and annotation process; the methods for artificial label noise introduced in the data, and the learning models used for PPG quality classification. The overview of this pipeline is presented in The training data set consists of PPG recordings (sampled at 240 Hz) from 3764 adult patients admitted to the intensive care unit (ICU) at the University of California, San Francisco (UCSF) medical center between March 2013 and December 2016. Continuous PPG recordings were divided into non-overlapping 30-s records. We randomly selected 25 30-s PPG records from each patient, corresponding to a total of 78,278 30-s PPG records after preprocessing, as described in a previous study ,18. The test set consists of PPG recordings (sampled at 240 Hz) from 13 patients  admitted to the Neuro ICU at UCSF Medical Center between October 2016 and January 2018. The inclusion criteria of the selected patients consist of: (a) being diagnosed with acute ischemic stroke; (b) being at least 18 years old, and (c) English-speaking. Patients with significant problems relating to their attention, alertness, cognitive functions, or communication were excluded unless a legally authorized representative could give consent on their behalf. All the enrolled patients provided written consent after being informed of the protocols approved by the UCSF’s Institutional Review Board [The annotation process followed the same rules as defined in a previous study ,18. The PPG records in the test set were also annotated in terms of artifact proportion. This artifact proportion was defined by selecting all segments in a PPG record that were considered corrupted, then dividing their combined length by the total length (30 s) of the signal. The final distributions of labels in both training and test sets are shown in To investigate the impact of label noise on model performance, we adopted four different prediction models that take various representations of PPG signals as input. The learning models used in this work were selected from the previous works that already made an extensive comparison from different approaches ,18. The −4. Each deep model is trained for 50 epochs, the final network parameter is chosen from the epoch with the least loss on the validation set.In our study, we use the same hyperparameter as our previous study ,18. For To simulate complex situations that can arise during the human labeling process, two common types of label noise were artificially introduced in the training data in this study, so that the impact of label noise on the classification performance can be evaluated. One of the principal sources of label noise is fatigue or environmental distractions during the labeling process. The fatigue and environmental distractions can be considered random since they would not affect more one class than another, and their occurrence is not dependent on the data but on the characteristics of the annotator or/and the conditions under which the annotator is working. This kind of noise was simulated by using random label noise, with different percentages of randomly selected labels flipped to their counterparts . To evaluate the impact of various levels of label noise, 10%, 20%, 30% and 50% of the labels from the training dataset were flipped to generate different versions of the training set.Another common type of label noise is class-dependent noise which arises from intrinsic bias of annotators . Here, tThree types of experiments are designed to evaluate the various impact of label noise on the model performance. The objective of the first experiment is to compare the performance deterioration on three models caused by label noise. Four classifiers  are trained from training sets with three types of label noise  and four levels  of label noise and tested on the independent test set see . With eaThis section presents the performance of three binary classifiers including both ML and DL models trained on training data with three types of label noise and four levels of label noise.The prediction accuracy of all classifiers is presented in In the first experiment, the impact of each type of label noise was investigated based on their percentage in the training data. However, because the distribution of each class in the training data is different, the absolute number of noisy samples from each class could vary a lot. In this second experiment, three types of noise were considered and their impact on the classification performance was evaluated by examining the performance curves against the absolute number of flipped samples. This can be derived by converting the different percentage levels of label noise into the absolute number of samples per class to be flipped. The 1D Resnet model was selected to compare the performance impact of various label noise types, as it was the top performing model trained on the non-label flipping dataset shown in our previous study .To study the effect of label noise on classifier probability decisions, the annotation of artifact proportion for each PPG record in the test set was utilized . We plotThe model trained on the clean training data classifies PPG records with any nonzero artifact proportion as bad quality  because of the stringent definition of good-quality labeling see a. In comIn all cases where label noise was introduced, the distribution of the model output probability started to be more diverse than the distribution in Within two class-dependent label noises, this effect was especially apparent when the label noise related to flipping the labels from bad-to-good. The plausible reason is that the classifier learned from mislabeled ‘good quality’ records which had a large range of percentage of artifacts, leading to the increased prediction probability on certain records with artifacts see d. In thet-tests are performed to compare the accuracies obtained in section A from different models. As reported in p < 0.05).The pairwise Student’s t-test is utilized to compare the accuracy based on the 1D Resnet from three different label noise. As illustrated in p < 0.05). At the same time, the difference between two class-dependent noise types is not considered statistically significant (p > 0.05).To assess if there is a statistical difference between different types of label noise, pairwise Label noise is an inevitable issue in supervised learning tasks and the identification of source can come from many reasons. In this study, the annotation process was undertaken by domain experts, and the potential reasons for artifactual labels can be identified and summarized into random label noise and class-dependent label noise. The random label noise simulates the error caused when fatigue took place, leading to occasional random assignment of class labels, and the class-dependent label noise simulates the labeling errors generated from annotator bias raised from the misinterpretation of annotation rules. In this study, a binary classification framework was adopted to assess and classify signal quality of 30-s PPG records. Three types of label noise were simulated by controlling the type and amount of artificially flipped labels in the training sets, which were used to train different ML and DL classifiers. Such an experimental setup permits evaluation of the different tolerance of prediction models against label noise, different impacts on model performance from different types of label noise, and the subsequent effect on the confidence of model-predicted labels.Between two conventional ML algorithms, the performance of XGBoost is less degraded by all three types of label noise than SVM, although they share the same input–hand-crafted features from our previous study . The corBoth deep and traditional machine learning algorithms suffer from label noise. However, deep neural networks proved to be more accurate and robust against all three types of label noise we studied. As shown in Between the two deep learning models, the 2D model was more robust than the 1D model. When we increased the noise level from 20% to 30% in the bad-to-good type of noise, the 1D model showed an unexpected improvement in terms of accuracy. Nonetheless, the 2D model was still slightly superior to the 1D model. One plausible reason could be that the 1D model had 34 layers, which is deeper than the 18 layers in the 2D model. In the machine learning bias-variance tradeoff theory, a deeper model can more easily adapt to training data with more variance, leading to a reduction in generalizability and an increment of error in the test data. The better performance of the 1D model when there was no label noise further supports this hypothesis.A significant difference in performance was also found within the three different types of noise when the same classifier was adopted. As shown in The calibration ability of the classifier was also vulnerable to label noise, which has been even less studied. We found that even 10% of each type of noise could significantly change the classifier output probability see . Due to The current study represents an evaluation of the impact of label noise on the accuracy and calibration of binary classifiers with a case study of PPG signal quality. However, restrictive definitions for labeling physiological events are highly common in the medical domain. In this study, the label definition of ‘good quality’ class is also strict, which makes the distribution of model output probability skewed to the bottom left instead of evenly distributed. The conclusions of this work may need to be limited to contexts where the class labels are based on a very specific definition in the classification problem. As the study was limited to binary classification, a similar study focused on multiclass classification would also expand the range of possible findings. The current study was focused on the several types of label noise that could be found in the medical data; however, the combination of them was not studied at this time, since the main objective was to compare the robustness of different types of learning models to each individual label noise. The impact of the combination of the label noise on the learning models’ performance will be assessed in future work.This study focused on using the quality assessment of individual 30-s PPG records as a problem to study label noise. This enabled us to explore the influence of different types of noise potentially arising from the human annotation process. The generalizability of our conclusions would be reinforced by examining a wider range of PPG-based tasks. It is also the case that different tasks may be susceptible to different types of noise in the data set. This suggests a need for a more task-specific investigation.The current work was dedicated to assessing the robustness of the previous developed classification models; however, other recent promising approaches have been proposed to deal with sequential time series such as transformers. Transformers and other AI-based models must be continually explored trying to find the best classifier for the quality assessment. Also, our results show deep learning methods are more robust to label noise, our future work would also focus on further improving robustness of deep neural networks.Label noise is a daunting challenge for supervised learning. In this situation, there is a greater risk of classifiers learning from mislabeled records. An ideal classifier should be able to distinguish between patterns of representative data and label noise. Deep learning has shown to be better at handling a certain amount of label noise than shallow machine learning models, of which Resnet is particularly robust against noisy labels. This work has compared the influence of label noise on a classification task aiming to assess the quality of physiological signals using different learning algorithms. The results have shown that deep learning-based models can deal with larger amounts of label noise than traditional machine learning algorithms, with less degradation in their performance. With regard to different types of label noise, learning models are more heavily affected by annotation bias than random noise, in particular towards a minority class. This underscores the importance of selecting more robust models that can deal with real-world label noise. To sum up, the present study systematically characterizes the impact of various types of label noise on model performance and probability calibration and provides valuable insight into model selection that could mitigate the issue of label noise.


Artificial intelligence-assisted otologic diagnosis has been of growing interest in the scientific community, where middle and external ear disorders are the most frequent diseases in daily ENT practice. There are some efforts focused on reducing medical errors and enhancing physician capabilities using conventional artificial vision systems. However, approaches with multispectral analysis have not yet been addressed. Tissues of the tympanic membrane possess optical properties that define their characteristics in specific light spectra. This work explores color wavelengths dependence in a model that classifies four middle and external ear conditions: normal, chronic otitis media, otitis media with effusion, and earwax plug. The model is constructed under a computer-aided diagnosis system that uses a convolutional neural network architecture. We trained several models using different single-channel images by taking each color wavelength separately. The results showed that a single green channel model achieves the best overall performance in terms of accuracy (92%), sensitivity (85%), specificity (95%), precision (86%), and F1-score (85%). Our findings can be a suitable alternative for artificial intelligence diagnosis systems compared to the 50% of overall misdiagnosis of a non-specialist physician. Otolaryngology is a medical-surgical discipline that includes the prevention, diagnosis, and treatment of different structures of the ear, nasal and paranasal cavities, pharynx, larynx, trachea, head and neck . It is eAccording to the Institute of Medicine of the National Academies of Science, Engineering, and Medicine from the US, nearly all patients will experience a diagnostic error in their life . In the Otological analysis has recently attracted the attention from the scientific community to generate new ear databases and to apply deep learning techniques ,11. HoweThe state of the art of spectral analysis for diagnosing middle ear pathologies is limited and recent. One study published in 2015  showed tOur previous work  proposedThis section describes the ear imagery database and data preprocessing in detail. Also, we present the architecture based on a convolutional neural network as a predictor of the four ear conditions under study. Finally, this section shows the metrics used to evaluate the performance of the proposed models.Retrospective analysis of images from video otoscopy was performed in patients who consulted at the Otolaryngology Departament of the Clinical Hospital of the University of Chile (HCUCH) between 2018 and 2019. A total of 22,000 images of the eardrum from 195 patients were used as the database for this work. The images were extracted from 200 video otoscopies acquired by ENT specialists using a digital otoscope DE500 Firefly. The video files were recorded at 20 FPS (frames per second) with a 640 × 480 pixels resolution.The ENT specialists of HCUCH performed at least two otoscopy examinations per patient (left and right ear). They recorded up to three diagnoses for each ear. Our analysis included those video otoscopies of patients with only one diagnosis related to four ear conditions: normal, chronic otitis media (COM), otitis media with effusion (OME), and earwax plug. The first row in The study was approved by the Scientific or Research Ethics Committee of the Clinical Hospital from University of Chile . All patients gave written informed consent, and all procedures were conducted following national regulations and the declaration of Helsinki, revision 2013.We propose an image-level analysis of the frames extracted from each video otoscopy. Some frames showed neither the ear canal nor the eardrum because the otoscopy recording started or ended at a different location than the ear. In addition, blurred frames occurred during acquisition due to camera movement  or camera out-of-focus, which the physician manually adjusted during the examination. We discarded all those undesirable frames by first separating the images that effectively show the tympanic membrane and then applying an algorithm to detect the level of blur in the image (considering only uniform blurring).Several images acquired during otoscopy do not show the tympanic membrane or ear canal. To avoid confusing the network by introducing useless information, we discard all images outside the domain of interest.Useless images were acquired at the beginning or the end of the otoscopy due to the nature of the acquisition procedure. On the one hand, the physician starts the recording before inserting the specula of the digital otoscope into the patient’s ear to ensure that the camera is in focus . On the other hand, the recording ends once the specula have been removed from the patient’s ear producing unnecessary frames. Assuming that the central frame of the video shows the tympanic membrane and (or) the ear canal , we calcTo evaluate the blurriness in the image, we employ the Variance of the Laplacian method as presented in . Such a To avoid the redundancy of the videos recorded, we implement a video summarization technique based on Principal Component Analysis (PCA) and a clustering algorithm . As a result, we obtain a subset of the most informative frames (keyframes) from a video otoscopy to feed the deep neural network.Once all blurred and out-of-domain frames were discarded, PCA was applied to find those components that describe a whole image in the video. The first 20 components were selected as principal components because, in our case, they explain 80% of the variance of data. Also, the dimension of the new feature space allows decreasing the computational cost of the process. Finally, we applied k-means to select N keyframes. The clustering algorithm assigned each frame  to a specific cluster considering the smallest distance . We select only one image per cluster, considering the one whose Euclidean distance is the smallest.The duration of the videos ranged from 30 s until 120 s. The shorter videos belong to the cerumen plug cases because the physician can only observe the earwax plug in the tympanic cavity . In more complex conditions, the physician performs a more extended examination to inspect the entire tympanic membrane and gather as much information as possible. There are more data (frames) of certain ear conditions in this sense. To avoid bias in the results due to an unbalanced database, we apply the keyframe selection algorithm with k = N = 100 and use the same amount of videos for each class. The bottom row of Our proposal is implemented under the configuration of a CAD system. An image of the tympanic membrane feeds a convolutional neural network that predicts four possible diagnoses: normal ear, chronic otitis media, otitis media with effusion, or an earwax plug. The network was previously trained on our otoscopy ear database. The spectral analysis of the input data to the system and the neural network’s architecture are discussed below.The database contains images in the RGB color space. That is, each pixel in the image is composed of three values representing the light intensity in the three channels: red (618–780 nm), green (497–570 nm), and blue (427–476 nm). With the aim to analyze the color dependence in CNN performance, we process the images to obtain single-channel information.The first approach is to keep images of three channels but with two equal to zero, as shown in the top row of Our approach is based on the VGG-16 neural network [The network was adapted to work with a single image channel with an input tensor of l represents the number of layer, j de number of neuron inside the layer, and i de ith element in the input map First convolutional layers extract low-level features such as edges, corners, or lines from the image. The last convolutional layers recognize structures and larger shapes in the input data. Each convolutional layer generates a feature map quations  and 3)..Xil−1 fes (ReLU) .Pooling layer reduces the number of computational nodes and prevents overfitting. There are not learnable parameters just realize a downsampling operation using average or maximum methods. This layer is include to provide a summary of the local distinctive features.Fully connected layer flattens the input data, typically the output of a convolutional or pooling layer, and performs the same operations given in Equations  and 3)..3). The We use a confusion matrix, which is a tool often used to describe the performance of a classification model. The matrix is generally defined for a binary approach with a positive and negative class . However, it can be extended to a multi-class approach by using the One-vs-All method . The conc represents the number of classes , OMC (5500 amples), OME (5500 samples), and earwax plug (5500 samples). The database was randomly partitioned into training/validation and testing sets with no overlap. We employed 500 samples of each class for testing. The remaining 5000 images were used for training and validation in a partition of 80% and 20%, respectively. The results reported below are derived after an average of 10 trials with the respective dataset partitioning.The input tensor of every single model receives an image of given in , using Agiven in  with a lWe compute the confusion matrices for the seven models implemented as shown in We have plotted ROC curves per each ear condition from seven models, as shown in Using the metrics established in Equations –8), we , we 8), We employed Gradient-weighted Class Activation Mapping (Grad-CAM)  to visuaThere is a clear need to strengthen the diagnosis and consequent otolaryngological treatment in health systems. The accurate diagnosis of middle and external ear diseases is challenging due to similar signs and symptoms between several pathologies. In addition, for almost all cases is necessary an inspection using an otoscope, a tool that has remained unchanged for, over a half-century. The inspection-based diagnosis will depend on the expertise of the examiner.Multispectral imaging analysis has several advantages over standard otoscopy, including increased image contrast, clear visualization of middle ear elements, better assessment of tympanic membrane vascularity, and improved demarcation of critical morphological structures . Although there are no commercial otoscopes available that acquire images in different spectral bands in addition to RGB, the research results of the few works ,14,15 inThe findings in this work encourage an analysis in which more pathologies that a physician is confronted with in daily practice can be included, as well as an images analysis in non-visible spectra such as infrared or ultraviolet.Middle and external ear diseases with conductive hearing loss are frequent medical consult of general practitioners and ENT specialists. However, misdiagnosis reaches rates around 30–50%. We evaluate the dependence of different color wavelengths in the performance of a computer-aided diagnosis system based on convolutional neural networks. The results showed that the model trained on green channel grayscale images outperformed others models in terms of accuracy (92%), sensitivity (85%), specificity (95%), precision 12 (86%), and F1-score (85%). In addition, our findings showed that the model trained with grayscale images of the green channel is still adequate for predicting normal ear and chronic otitis media cases. However, a more accurate prediction for otitis media with effusion cases is obtained when working with the red channel. Whereas, the accurate prediction of earwax plug cases is obtained from the blue channel. Also, the system could assist in different areas of medicine such as primary health care consultation, emergency rooms, clinics, and hospitals that demonstrate the usefulness and versatility of the method, especially for the general practitioner and the ENT specialist in case of diagnostic doubt. More studies are needed to associate the improvement in diagnostic performance with the establishment of clinical outcomes, such as early initiation of treatment, reduction in adverse effects and complications, or clinical prognosis.


In order to diagnose an incipient fault in rotating machinery under complicated conditions, a fast sparse decomposition based on the Teager energy operator (TEO) is proposed in this paper. In this proposed method, firstly, the TEO is employed to enhance the envelope of the impulses, which is more sensitive to frequency and can eliminate the low-frequency harmonic component and noise; secondly, a smoothing filtering algorithm was adopted to suppress the noise in the TEO envelope; thirdly, the fault signal was reconstructed by multiplication of the filtered TEO envelope and the original fault signal; finally, sparse decomposition was used based on a generalized S-transform (GST) to obtain the sparse representation of the signal. The proposed preprocessing method using the filtered TEO can overcome the interference of high-frequency noise while maintaining the structure of fault impulses, which helps the processed signal perform better on sparse decomposition; sparse decomposition based on GST was used to represent the fault signal more quickly and more accurately. Simulation and application prove that the proposed method has good accuracy and efficiency, especially in conditions of very low SNR, such as impulses with anSNR of −8.75 dB that are submerged by noise of the same amplitude. The rotating components of machinery usually tend to manifest failures such as scuffing, pitting, and cracking. Detection of these failures in their incipient period can curb fault development, adjust production plans, and avoid huge economic losses and casualties ,2. The iIn trying to solve the problem of impulse extraction, traditional fast Fourier transform (FFT) can only present the energy distribution of modulation frequency . MethodsIn view of the above analysis, we proposed a novel fault diagnosis method using the TEO and sparse decomposition based on multi-resolution GST for rotating machinery in complicated conditions. In this method, we explored the structure of a signal processed by the TEO. Then, we proposed a preprocessing method with a filtered TEO to enhance the incipient fault signals, which can suppress noise and enhance impulses. Next, we used sparse decomposition based on multi-resolution GST to obtain the sparse representation of the enhanced signals. Finally, we rebuilt the initial fault signals and acquired the sparse representation of the original fault signals. Simulation and application verified the validity of the proposed method.This paper is structured as follows: a fundamental introduction to the TEO and sparse decomposition based on multi-resolution GST time–frequency spectrum is presented in For any continuous signal If The energy of the system contains the kinetic energy of the mass block and potential energy of the spring, it can be calculated by:requency .According to Equations (1) and (3), the Teager energy operator is:The Teager energy operator can also reflect the energy of the signal Furthermore, the first derivative of the Teager energy operator is given byt signal .In order to represent the signals with local features, the method of sparse decomposition adopts non-orthogonal and redundant atoms to match these signals, and it can achieve the representation more accurately and sparsely. However, to improve the accuracy of sparse decomposition, the scale of over-complete atom libraries is often very large, which seriously affects the efficiency of the search for the best atoms. Sparse decomposition based on a multi-resolution GST time–frequency spectrum can improve the efficiency of sparse decomposition by converting the atomic search process to an optimal time–frequency spectrum search. The flow of sparse decomposition based on a multi-resolution GST time–frequency spectrum is shown in Realization of this method includes four steps: generation of all time–frequency spectra by GST, determination of optimal time–frequency factors, construction of optimal atoms by selected optimal time–frequency factors, and sparse representation using selected atoms.In order to match the fault impulse signal better and facilitate the use of the fast algorithm, this method improves the model of impulse as:p is the adjustment factor.GST is defined as follows:Set the model of the over-complete atom library as:Thus, It is not difficult to prove thatAccording to the analysis above, the inner product operation in atom searching can convert to searching for normalized time–frequency spectra with optimal k is the number of extracted optimal atoms, then the sparse representation of the signal Suppose d ‖gi‖=1 .Equations (5)–(7) show the results of the TEO when the amplitude is the form of Set the damping vibration signal of the formThus, the expression of the TEO isAs the impulse signal possesses the feature of rapid decay, extremely short duration, and high oscillation frequency, the TEO of Equation (15) shows both the TEO and the first derivative of the TEO are sensitive to frequency, so they are suitable for tracking impulse energy. Their disadvantage is that their shape would be affected by high frequency noise, so a smoothing filtering algorithm was adopted to maintain the shape. The operation is expressed byThe main purpose of signal preprocessing in this paper was to enhance the impulses and suppress noise without damaging the signal structure. Therefore, for the proposed signal preprocessing method, firstly, obtain the impulse envelope through the TEO; secondly, obtain the enhanced envelope through the smoothing filtering algorithm; and thirdly, acquire the enhanced impulses through the product of the enhanced envelope and the original signal.The preprocessed signal with the filtered TEO is expressed byThus, once the optimal time–frequency factors After preprocessing through filtered TEO method, the impulses can be extracted appreciably, and noise is greatly suppressed. If the machinery rotates at a constant speed and runs under a single fault condition, usually taking the FFT for this preprocessing signal can obtain the characteristic fault frequency. If the machinery rotates at variable speed or runs under a multisource vibration condition, the impulses will perform their characteristics with aperiodicity and inconsistency. At this time, sparse decomposition can be introduced to extract all the impulses, by which further analysis and diagnosis can be realized. In order to improve the efficiency of the sparse decomposition, a method based on a multi-resolution GST time–frequency spectrum is adopted in this paper.The flow of the proposed sparse decomposition based on the TEO is shown in Realization of this method includes four steps: preprocessing with the filtered TEO to enhance the impulses, extraction of the best atoms for preprocessed signals, reconstruction of atoms for the original fault signal, and sparse representation with reconstructed atoms.Suppose the time–frequency factors of the extracted atoms from the preprocessed signal are given byThe preprocessed signal can be represented sparsely in the library built from the extracted atoms asThus, the time–frequency factors of atoms representing the initial fault signal can be calculated byThe initial fault signal can be represented byThe simulation was designed mainly to verify advantages in anti-interference and accuracy. Firstly, we used simulated signals to prove the advantage of using the filtered TEO compared to the traditional TEO. Secondly, we used simulated signals with different SNRs to prove the validity of the proposed method.Simulated signals were constructed with impulses and white Gaussian noise, which is given by:The simulated signal To further evaluate the performance of these four methods, correlation of preprocessed impulse and expected impulse was introduced. Obviously, the larger the correlation is, the more similar the preprocessed impulse and expected impulse are, and the better the method performs.Therefore, the filtered TEO has the best performance, exceptionally enhancing impulses in very low SNRs. Preprocessing with the filtered TEO can not only retain the structural characteristics of impulses but can also greatly enhance the impulses.Three simulated signals consisting of impulses and noises were constructed, which is given by:a, simulated signals with different SNRs were generated.Changing the value of The simulated signals The enhanced versions of the simulated signals using the filtered TEO are shown in Comparison between S3 is shown in To describe the good performance of the proposed method, a comparison of the proposed method and traditional sparse decomposition based on a multi-resolution GST time–frequency spectrum for every impulse of To express the effect of these two methods, To verify the practicability of the proposed fast sparse decomposition method based on time–frequency spectrum optimization, a set of outer-ring fault signals with rotation speed linear accelerations from 300 r/min to 450 r/min in 0.25 s were acquired from a fault simulation platform for rotating machinery. The fault simulation platform is shown in The engineering application proves the proposed method can extract non-periodic impulses exactly and can identify the fault type under conditions of changing rotation speed.(1)The proposed preprocessing method with filtered Teager energy operation is more effective in restraining the low-frequency harmonic component and noise and improving impulses with high oscillation frequency. More importantly, it can retain the structure of the impulse, which is critical to the sparse decomposition.(2)The proposed sparse decomposition method based on the Teager energy operation performs well in terms of accuracy and efficiency in extracting impulses from low SNR signals and is more applicable in complex and harsh environments.In this paper, we proposed a fast sparse decomposition based on the Teager energy operator in the extraction of weak fault signals. Results from simulation and engineering applications verify the superiority of the proposed method. Our conclusions are summarized as follows:(1)Improve the performance of the filtered TEO, such as an adaptive filtering strategy.(2)Improve research efficiency in the time–frequency spectra; the proposed research method still needs to generate all the spectra, which need a great deal of calculation.In the future, we will carry out the following research:


Robot-assisted surgery is becoming an increasingly common approach for lung cancer resection. This technological innovation is the subject of numerous publications, which demonstrates the interest in this approach. All these publications report contradictory results on the clinical benefits of robot-assisted surgery.Considering the large number of publications, the quality of the studies raises questions. We are surprised by the small number of randomized controlled trials (RCTs) on robot-assisted surgery. We found three RCTs, one of which is currently being included, while the other two have published their results with small numbers, 113 and 76 patients ,3. In coThe WHO publication on the assessment of new technologies recalls the importance of evidence-based studies . Not onlWe regret the lack of enthusiasm of thoracic surgeons to participate in RCTs to demonstrate the value of robot-assisted surgery. We submitted an RCT for publication comparing VATS with thoracotomy. We had great difficulty in completing this trial, due to the delay in inclusion because of the lack of motivation on the part of surgeons to include patients. In the last ten years, we identified a total of only four RCTs published on VATS ,6,7,8. AThe use of a propensity score applied to an observational cohort helps to limit bias. We can never be sure that these methods control for all confounding factors and that the two groups of patients are truly comparable. The clinical databases used in studies raise the question of data quality. One study showed that only 25% of the surgical teams participating in the national Epithor database included all their lung cancer patients and all postoperative deaths . This fiDespite the widespread use of real-world evidence (RWE) and the use of methodologies to reduce bias, randomized controlled trials remain the main study design with the highest level of evidence to validate an innovative technology. RWE has its place following an RCT to confirm the results or highlight adverse events. We find it difficult to understand the lack of motivation of thoracic surgeons to participate in RCTs when other disciplines have been able to conduct large-scale RCTs to validate innovative technologies.In conclusion, the lack of convincing studies to validate robot-assisted surgery will continue to cast doubts on the real benefit of this approach, particularly for decision makers or evaluation agencies. Robot-assisted surgery is thus penalized if it is to be considered a technology that can transform the management of patients with lung cancer.


The mechanical properties of engineered cementitious composites (ECC) are time-dependent due to the cement hydration process. The mechanical behavior of ECC is not only related to the matrix material properties, but also to the fiber/matrix interface properties. In this study, the modeling of fiber and fiber/matrix interactions is accomplished by using a semi-discrete model in the framework of peridynamics (PD), and the time-varying laws of cement matrix and fiber/matrix interface bonding properties with curing age are also considered. The strain-softening behavior of the cement matrix is represented by introducing a correction factor to modify the pairwise force function in PD theory. The fracture damage of ECC plate from 3 to 28 days was numerically simulated by using the improved PD model to visualize the process of damage fracture under dynamic loading. The shorter the hydration time, the lower the corresponding elastic modulus, and the smaller the number of cracks generated. The dynamic fracture process of early-age ECC is analyzed to understand the crack development pattern, which provides reference for guiding structural design and engineering practice. To ensure adequate safety and reliability of buildings and to avoid disasters, in addition to structural design optimization, there are higher requirements for concrete materials. Professor Victor Li has developed engineered cementitious composites (ECC) made of disordered short fibers with significant strain-hardening properties . As a neA few researchers investigated the basic mechanical properties and strain hardening capacity of early-age ECC. Chan et al.  investigIn addition to experimental studies, Victor Li et al.  improvedSilling  proposedThe previous studies have shown the great advantage of peridynamics in simulating crack propagation in cementitious composites. However, these studies are all qualitative or quantitative analyses of the properties of cement materials in the normal use phase and lack numerical simulations of the early age of these materials. The mechanical properties of cementitious composites exhibited at different ages vary greatly, which has a great impact on the construction schedule of the project. Considering the time-dependent behavior of bond stretch between material points in peridynamics, it can be used to model effects such as aging and fatigue . In thisx and point Peridynamics ,19 is a For simplicity, in the following discussion, the relative positions of the two points in the reference configuration are denoted by x with other material points in its horizon is called bond, which is the fundamental difference between peridynamics and classical continuum mechanics. Here, the horizon can be understood in such a way that there is no interaction between these two points when x′ is not in the neighborhood of x, i.e.,The interaction of material point f can be derived from a scalar micro-potential function w, i.e.,w denotes the energy stored in a deformed bond per unit volume squared, so that the energy contained in each unit volume of matter point isAccording to , for micFactor 1/2 appears because each endpoint of the bond possesses only half of the energy in the bond.c are discussed in detail in [A linearized version of the pairwise force function for a micro-elastic material takes the following form, as shown in etail in . Where sw is obtained according to Equation (3) asTherefore, in the prototype microelastic brittle (PMB) material, the micropotential energy Taking Equation (6) into Equation (4) leads to the strain energy density in peridynamicsc is obtained [Letting Equation (7) be equal to the strain energy density obtained from classical linear elastic mechanics for the same deformation, the micromodulus obtained ,(8) c= Damage in peridynamics is characterized by the failure of bonds among material points, which happens when the elongation of bonds exceeds a critical stretch. When all bonds in a plane fail, a crack is formed. In the ideal case, the expression for the critical stretch ase rate .(9)s0=5t.In order to describe the breakage of bonds between material points, it is necessary to introduce a judgment function At this point, the pairwise force function can be expressed asAs can be seen from the above equation, the definitions of damage and fracture are included in the pairwise force functions of the peridynamics, and no additional failure criterion and stress intensity factor are required. Instead, they can be solved in a unified framework, which is where the peridynamics approach differs from the traditional numerical approach. The degree of damage When Some scholars have already utilized peridynamics to model ECC and good results have been obtained. Reference  used a sThe constitutive model of PMB material is shown in found in ,35.(13)The performance of ECC at early age is influenced by the type and properties of the admixtures and the curing conditions. The elastic modulus of the early age cementitious material gradually increases with age and eventually tends to a constant value after the end of the hydration process. Instead of directly defining the amount of strain, peridynamics uses the elongation and shortening of bonds between material points to describe changes in the relative positions of the material points. In this study, a time-dependent function is introduced to describe the different properties of the bonds between material points at different ages. The peridynamics model of the cement matrix is shown in ith time . κ is thECC is an ultra-high toughness cementitious composite material with high tensile ductility. The degrees of freedom of fiber nodes often determine the computational efficiency and computational cost of the simulation process, so this study uses the semi-discrete fiber model proposed by Kang et al.  to solveLin et al.  proposedThe mechanical properties of the cement matrix vary with time as the cement hydration process proceeds. However, the time dependence of the strength of the fiber/matrix bond interface differs from the time dependence of the mechanical properties of the matrix material due to the difference in microstructure. Chan et al.  investigt is the curing age. Taking Equation (18) into Equation (17) we can obtain the relationship between force and tensile displacement at different ages, which can be written asHere, Peridynamics can consider discontinuous solutions. The whole model is discretized into material points defined in PD and a certain volume is assigned to each point. When the space and volume assigned to each point are uniform, the discretization is shown in The basic equation of motion Equation (1) for PD can be rewritten using the finite summation method asHere,In this study, a central difference algorithm, namely velocity-Verlet algorithm, is implemented to solve the PD equations of motion satisfy :(24)Δt<2The damage–fracture process of concrete materials can be typically reflected in the uniaxial tensile test. We consider a thin rectangular ECC plate with dimension of 350 × 50 mm under dynamic symmetric tension loads, as depicted in g et al.  on conveThe experimental data in Reference  shows thThe model proposed in this study was used for numerical simulation, and the crack propagation paths of ECC plates with ages of 3 days, 7 days, and 28 days at 90 µs were captured, as shown in In actual engineering structures, ECC as a protective reinforcement material exhibits good impact resistance when under impact loading. Previous studies on early-age ECC have focused on material properties, and there are fewer studies on mechanical properties such as damage and fracture. We consider a thin, rectangular ECC plate with dimension of 200 × 50 mm under dynamic impact load, as depicted in By introducing the time-varying effects of early-age ECC materials into the peridynamics model and writing the corresponding codes, the crack propagation paths of ECC plates under uniaxial tensile and three-point bending loads at different ages were simulated. From the results of numerical simulations, the crack paths at different ages are basically in agreement with the experimental as well as the theoretical analysis results of the relevant literature. It also provides a powerful numerical tool for the analysis of damage fracture processes in early age cementitious materials. The proposed improved peridynamics model also provides a reference and some implications for future numerical modeling. From the engineering practice, in order to reduce the risk of cracking of ECC at an early age, firstly, the particle ratio should be appropriately selected to reduce the porosity. Secondly, as the fiber bridging effect cannot be ignored, fibers with stronger bonding to the cement matrix or fibers whose surface has been treated should be selected. It is worth noting that this study only presents the improved model and simulation results for the curing age before 28 days. Our next work is to study the results of longer curing age in order to better generalize PD theory and guide the application of ECC in engineering.


So far, little is known about how conscious attention to internal body signals, that is, interoception, affects the synchronization with another person, a necessary or required social process that promotes affiliations and cooperation during daily joint social interactions. The effect of explicit interoceptive attentiveness (IA) modulation, conceived as the focus on the breath for a given time interval, on electrophysiological (EEG) correlates during an interpersonal motor task compared with a cognitive synchronization task was investigated in this study. A total of 28 healthy participants performed a motor and a cognitive synchronization task during the focus and no-focus breath conditions. During the tasks, frequency bands  from the frontal, temporo-central, and parieto-occipital regions of interest (ROIs) were acquired. According to the results, significantly higher delta and theta power were found in the focus condition in the frontal ROI during the execution of the motor than the cognitive synchronization task. Moreover, in the same experimental condition, delta and beta band power increased in the temporo-central ROI. The current study suggested two main patterns of frequency band modulation during the execution of a motor compared with the cognitive synchronization task while a person is focusing the attention on one's breath. This study can be considered as the first attempt to classify the different effects of interoceptive manipulation on motor and cognitive synchronization tasks using neurophysiological measures. The “concentrated attention to a particular interoceptive signal during a predetermined time interval” is known as interoceptive attentiveness (IA) Schulz, . As a toOnly recently, research on interoception started focusing on the link between the perception of inner signals derived from within the body and their link with social processes  during the synchronization tasks. To the best of our knowledge, the literature that delves into the effects of IA on single brain neural correlates during a motor or cognitive synchronization task is still scarce. To better understand this phenomenon at the cortical level on a single individual, a previous study explored the hemodynamic changes related to the effects of a brief focus on the breath session on a single brain.Indeed, in a recent pilot study, the hemodynamic correlates of simple tasks requiring cognitive (linguistic) or motor synchronization were examined in order to determine the impact of the focus on the breath on such tasks. According to the results, adjusting the attention to the breath might broaden the benefits of boosting the brain PFC response to synchronization during basic synchronization tasks. Furthermore, it was shown that when the intentional focus on the breath was achieved during the cognitive task requiring synchronization with a partner, brain areas linked with sustained attention, such as the right PFC, were more involved. It is interesting to note that this evidence was significantly observed for the cognitive  task, and not for the motor task. If taking into account the direct correlation between interoceptive correlates (and the focus placed on controlling them) and motor performance, as well as the neuroanatomic closeness between the interoceptive and the motor region, this finding appeared paradoxical  occurrences, MW awareness, and the return to concentrate on the breath while performing a breath monitoring task  makes it simpler for people to converse in an ecological and naturalistic continuous stream. Despite the low spatial resolution of EEG limits the degree to which specific areas of neural activity can be precisely localized, it nonetheless provides an indication of regional scalp activation.Thus, we believe it could be of interest to investigate the EEG cortical effects of the focus on the breath  in relation to distinct regions of interest while a person is performing a motor compared to a cognitive task requiring synchronization with a partner. For these reasons, in the present study, we propose the concomitant execution of the focus on the breath task and a motor or cognitive synchronization task, to test the effect of explicit IA manipulation on EEG correlates of a task requiring synchronization.On the basis of the research and data described before, it was proposed that the frequency bands associated with sustained attention and attention control during interoceptive tasks  = 24.2 years; standard deviation (SD) = 3.11] were involved in the study by employing a non-probabilistic convenience sampling method. Exclusion criteria for participation in the study encompassed physiological conditions such as chronic or acute pain, severe medical and chronic diseases, seizures, traumatic brain damage, pregnancy, prior meditation experience, and any mental or neurologic abnormalities. All people who took part in the experiment were right-handed and had a normal or corrected-to-normal vision. All participants signed a written informed consent form before the experiment and were informed they would not be remunerated. The approval for this study was provided by the Ethical Committee of the Department of Psychology at the Catholic University of the Sacred Heart of Milan (protocol number: 2020TD), in accordance with the Declaration of Helsinki (1964).The participants were situated in a dimly lit room with a researcher in charge of providing the experimental instructions and completing the synchronization tasks. Before beginning the experimental activities, EEG was used to record a 120-s resting baseline.The subjects were asked to perform two simple motor and cognitive synchronization tasks by imitating the experimenter while their electrophysiological activity was recorded continuously. In this version of the alternating speech task, the participants were asked to pronounce four syllables “LA,” “BA,” “CA,” and “DA” sequentially and alternately. Each linguistic synchronization task session lasted about 3 mins, consecutively. Instead, the motor synchronization task consisted of finger movement. Specifically, the participants had to synchronize their finger movements with the experimenter sitting in front of them. The total duration of the task was of 3 mins  and acquisition software (Neuroscan 4.2) were used to collect EEG data during task execution. An ElectroCap with Ag/AgCl electrodes was used to record EEG from active scalp sites referred to earlobes  Jasper, . The EEGIn the statistical analysis of the data, three regions of interest (ROIs) grouping by averaging frontal , temporo-central , and parieto-occipital  electrodes were considered .ROI  were applied to dependent EEG data . In the case of significant interactions, pairwise comparisons were utilized to explore the significant interactions, and Bonferroni correction was applied to decrease potential biases in repeated comparisons. When applicable, Greenhouse–Geisser epsilon was used to correct the degrees of freedom in all ANOVA testing. The kurtosis and asymmetry indices were also employed to assess the data distribution normality. The size of statistically significant effects was evaluated through partial eta-squared (η2) indices. The statistical tests were performed with IBM SPSS Statistics version 25.A total of four repeated-measures ANOVA with independent within-factors Task (2: motor and cognitive) × Condition (2: focus and no-focus) × Task × Condition × ROI was found . Pairwise comparisons revealed greater mean delta values in the frontal ROI during the focus condition in the motor than in the cognitive task . Also, higher mean delta values were found in the temporo-central ROI during the focus condition for the motor than in the cognitive task   = 8.09, p < 0.001, η2 = 0.476]. Pairwise comparisons showed greater mean theta values in the frontal ROI during the focus condition in the motor than in the cognitive task   = 8.90, p < 0.001, η2 = 0.490]. Higher mean values of the beta band were found in the temporo-central ROI during the focus condition in the motor than in the cognitive task , as shown by pairwise comparisons  software was adopted for EEG data visualization.The current work is the first to describe the effects of an explicit IA manipulation (conceived as a brief breath awareness practice) on the EEG correlates in response to a motor and a cognitive synchronization task. In particular, the EEG frequency bands  were observed and analyzed considering three main ROIs grouping frontal, temporo-central, and parieto-occipital electrodes. The results showed significantly higher delta and theta power in the focus condition in the frontal ROI during the execution of the motor than the cognitive synchronization task. Furthermore, in the same experimental condition, delta and beta band power were increased in the temporo-central ROI. To the best of our knowledge, this is the first time such findings have been observed. The following discussion is consistent with the body of extant neuroscientific knowledge in the field.First, the delta band showed double synchronization (related to both interoception and motor synchronization), which implies the activation of both frontal and temporo-central regions during the motor compared with the cognitive synchronization task in the focus condition.As far as the functional significance of the delta band is concerned in this context, the literature is still scant. In former studies exploring Zen and Qi-Gong  again during the motor compared with the cognitive synchronization task in the focus condition. Beta-band activity is classically considered as being related to motor functions , which is necessary for keeping tabs on the state of affairs and readjusting the sensorimotor system  region occurs due to the effect of consonance, intentionality, and awareness showing maximum response in the motor synchronization task, which requires focused attention. Indeed, rhythmic synchronization activities need sensory–motor coordination. However, one alternative explanation of the beta increase could also be related to the normalization procedure adopted in the present study. Future research could better disambiguate this aspect, adopting different methodological approaches.Taken together, this evidence suggests that the interoceptive focus goes hand in hand with the motor synchronization task, which requires full synergy, without cognitive interference, which the cognitive task could have produced, obscuring the effect of IA. Indeed, different from what was expected and stated in our hypotheses for the cognitive synchronization task, the presence of mainly alpha and theta band wave activities in the temporal and parietal regions during the cognitive synchronization task performance was not detected. No evidence was also observed in the focus condition while the participants were performing the cognitive synchronization task. A possible explanation for this lack of significant results could be due to the nature of the task, which required multiple cognitive processes. In fact, a modified version of the human-to-human alternating speech task, in which the participants had to syllabize with the experimenter for a total of 3 mins, was adopted for the current study. It could be speculated that the mediation of the verbal register has made the request to focus on the breath and synchronize with the speech too complex, and this could have increased the cognitive load in the individuals requiring the activation of a more distributed and differentiated neural network in a non-significant way.Prospective studies are needed to disentangle the psychophysiological correlates of the effect of IA on cognitive synchronization tasks, adopting more ecological tasks or other neuroscientific methods.We acknowledge that in our previous fNIRS study, we observed an increase in the right PFC when the explicit focus on the breath was induced during the cognitive task requiring synchronization: however in comparison to these hemodynamic results, the EEG montage was here extended to additional brain regions, and data were, in this case, more precise with reference to the function and the source of the EEG signal. Indeed, cortical oscillations provide useful information to monitor the process beyond localization that could be not linked to the specific processes. Future research in this field would benefit from some methodological improvements, such as (i) the integration of a fNIRS-EEG co-registration over multiple cortical sites; (ii) the implementation of an EEG hyperscanning paradigm, collecting the information of the other partner's brain; and (iii) the integration of more extensive EEG biosignal analysis to parameterize neural power, for instance, the analysis of the oscillatory portion of the spectrum of specific frequency bands  improves the manifestation of EEG brain correlates related to mental concentration, coordinated and controlled motor activity during motor synchronization. It might be plausible that this EEG pattern could have potential benefits on improving motor imitation performance at the behavioral level; however, future studies are needed to confirm this effect. Indeed, this study can be considered as a first attempt to classify the different effects of interoceptive manipulation on cognitive and motor synchronization tasks by comparing them using neurophysiological measures.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.The studies involving human participants were reviewed and approved by Department of Psychology, Catholic University of the Sacred Heart, Milan, Italy. The patients/participants provided their written informed consent to participate in this study.MB and LA contributed to the conception and design of the study, wrote the first draft, contributed to manuscript revision, read, and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Aiming to address the problems of the high bit error rate (BER) of demodulation or low classification accuracy of modulation signals with a low signal-to-noise ratio (SNR), we propose a double-residual denoising autoencoder method with a channel attention mechanism, referred to as DRdA-CA, to improve the SNR of modulation signals. The proposed DRdA-CA consists of an encoding module and a decoding module. A squeeze-and-excitation (SE) ResNet module containing one residual connection is modified and then introduced into the autoencoder as the channel attention mechanism, to better extract the characteristics of the modulation signals and reduce the computational complexity of the model. Moreover, the other residual connection is further added inside the encoding and decoding modules to optimize the network degradation problem, which is beneficial for fully exploiting the multi-level features of modulation signals and improving the reconstruction quality of the signal. The ablation experiments prove that both the improved SE module and dual residual connections in the proposed method play an important role in improving the denoising performance. The subsequent experimental results show that the proposed DRdA-CA significantly improves the SNR values of eight modulation types in the range of −12 dB to 8 dB. Especially for 16QAM and 64QAM, the SNR is improved by 8.38 dB and 8.27 dB on average, respectively. Compared to the DnCNN denoising method, the proposed DRdA-CA makes the average classification accuracy increase by 67.59∼74.94% over the entire SNR range. When it comes to the demodulation, compared with the RLS and the DnCNN denoising algorithms, the proposed denoising method reduces the BER of 16QAM by an average of 63.5% and 40.5%, and reduces the BER of 64QAM by an average of 46.7% and 18.6%. The above results show that the proposed DRdA-CA achieves the optimal noise reduction effect. The improvement of the SNR plays an important role in both the cooperative and non-cooperative fields of wireless communication systems. In cooperative communication, improving the modulation signal’s SNR can enhance the signal transmission quality and reduce the demodulation error rate. When it comes to non-cooperative communication, the improvement of the SNR is conducive to improving the modulation classification accuracy, and then signal interception or interference can be performed more effectively. Therefore, the denoising method of modulation signals has always been the research focus of wireless communication.In the traditional signal denoising methods, Parolai et al. in Ref.  use the Recently, deep learning has not only achieved great success in the fields of computer vision and natural language processing, but has also provided a new solution for the noise reduction of modulation signals. Chang et al.  propose Due to the limitations of the simple network model, the above five methods cannot extract deep signal features, resulting in limited SNR improvement. In deep convolutional networks, shallow convolution can extract detailed edge data, while deep convolution can selectively acquire useful semantic information. Therefore, deep convolutional networks are also widely used in signal denoising. In 2017, Zhang et al.  extend tThe channel attention mechanism is a very powerful method for optimizing network performance. It can selectively strengthen feature values of important channels by changing the weight of the channel, thus improving the expression ability of the network and achieving higher performance. Yancheng et al.  employ tBased on the above analysis, in this paper, we adopt a deep convolutional network to construct a denoising autoencoder, in which more effective residual connections between different scales and depths are explored to extract richer features and optimize network degradation problem. We also try to integrate the squeeze-and-excitation (SE) module  into the(1)We choose the convolutional denoising autoencoder as the noise reduction model for modulation signals. Residual connections are added not only inside the encoding (decoding) block but also between the encoding (decoding) blocks of different depths to form a double residual connection.(2)The SE module is improved and introduced into the denoising autoencoder to optimize the feature extraction of modulation signals. The improved SE module retains the advantages of selective enhancement for channel features, and the number of parameters is fewer, which makes the network model easier to train and achieve a better noise reduction effect.(3)In the decoding module of the denoising autoencoder, transpose convolution (TConv) is selected to complete upsampling. The parameters inside TConv can be initialized randomly and updated iteratively. At the same time, the SmoothL1Loss(SLL) function is used as the loss function of the denoising autoencoder, which can better measure the error between the estimated value and the real value, reconstruct a more accurate modulation signal, and improve the accuracy of the model training.The remainder of this paper is organized as follows. In this paper, we focus on the communication scenarios under the additive white Gaussian noise (AWGN) channel  with a lThe received modulation signal Equation .(1)s′, as the champion model of Imagenet 2017, is very similar to the Deep Residual Network (ResNet). However, it adds an SE module so that the information between the channels can be exchanged, thereby improving the performance of the network. C represents the number of channels in the feature map, H represents the height of the feature map, and W is the width of the feature map. The task of the squeeze part is to compress the feature map dimension (1) The squeeze part: Assume that the dimension of the original feature map is The squeeze operation is implemented using the Global Average Pooling (GAP) method, and its calculation is described as follows: C/r output channels, namely a dimensionality-reduction layer with reduction ratio r. Then the convolution result is activated using the LeakyReLU function, and finally the activated feature graph is dimensionally augmented through a convolution layer with C output channels to guarantee channel dimensional consistency between the output (2) The excitation part: By adding a convolutional layer above and below the nonlinear activation layers, we parameterize the gating mechanism to predict the importance of each channel in the feature map Based on above analysis, the excitation operation can map the feature map The decoding block of the noise reduction autoencoder is depicted in the N is the length of the input signal, In the existing work, most denoising autoencoders use the two loss functions of Mean Absolute Error (MAE) and Mean Square Error (MSE), and carry out parameter estimation by minimizing the error. MAE is the average absolute value of the true value minus the predicted value. By contrast, MSE is the average square of the absolute value of the predicted value minus the true value. Both MSE and MAE range from 0 to infinity, and they can be calculated, respectively, usingIt is clear that SLL is an integration of MSE and MAE. When taking the derivative of SLL, we have: In Equation , it can The dataset of the denoising autoencoder includes eight modulation signals generated by MATLAB; they are BPSK, QPSK, 8PSK, CPFSK, GMSK, OQPSK, 16QAM, and 64QAM, and the detailed parameters of the dataset are shown in N is the length of the signal, In order to facilitate the addition of noise and the measurement of the SNR, as well as to accelerate the training of the model and reduce the computational complexity of the data, we normalize the power of the modulation signals generated by MATLAB simulation using the following formulae: We conduct simulation experiments to evaluate our denoising autoencoder, and we also compare it with a set of typical models including RLS, LMS, PCA, and the deep learning method DnCNN proposed by Khan et al. . The evas represents the noiseless modulation signal, and s is the same as before, and The SNR is defined as the ratio of noiseless signal power to noise power, which can intuitively measure the quality of signal. The calculation formula is given by the following: In this paper, the SE module is first improved and then introduced into the encoder as the channel attention module, so that the modulation signal features can be better extracted and the computational complexity of the model can be reduced. In order to validate the efficiency of the improved SE module as the channel attention mechanism, we carry out the first ablation experiment based on the control variable method. We compare the improved SE module (impr-SE) using the convolutional layer with the original SE module (orig-SE) using FC and the denoising autoencoder without a channel attention mechanism (withoutCA). In this case, the SE module is removed in the coding block in In the proposed DRdA-CA, “dual” represents the residual connections added not only between different encoding (decoding) blocks but also inside the encoding (decoding) blocks of different depths, so it is called “dual”. In contrast, “single” refers to the residual connection only existing inside the encoding (decoding) block, and each encoding (decoding) block is the improved SE-ResNet module, which contains one residual connection. In order to demonstrate the advantages of “dual residual connections”, one of the main innovations, we have also conducted an ablation experiment based on the control variable method, comparing it with the “single residual connection” method, where the residual connection only exists inside the encoding (decoding) block. The loss curves are given in In the second experiment, we compare the SNR and EVM variations of different denoising methods. It would be complicated to use curves to display dynamic ranges of SNR and EVM after denoising with the increase of SNR for all eight modulation modes. As a result, we only present four typical modulation modes in For non-cooperative communication applications, the receiver needs to firstly perform modulation classification, followed by demodulation and decoding. Therefore, the modulation classification experiment is also done to further verify the performance of the denoising autoencoder. The LMS and RLS algorithms need to know the modulation mode and send the training sequence when they are used to reduce noise in communication systems, leading to low transmission efficiency. Thus, they are unsuitable for non-cooperative communication applications. PCA also has the lowest denoising performance. As a result, in the experiment of the noise reduction cascade AMC, we compare the DnCNN to the proposed method. Moreover, the CNN  is adoptThe experiment schemes are as follows. (1) The CNN model is used for AMC directly, denoted as CNN. (2) The DnCNN method is used to reduce the noise of modulation signals, and then CNN is used for AMC, denoted as DnCNN+CNN. (3) The proposed DRdA-CA is used to decrease the noise, followed by CNN for AMC, denoted as DRdA-CA+CNN. However, the recognition performance of the high-order modulations is poor. In particular, the recognition accuracy of the 16QAM and 64QAM signals fluctuates widely. This is attributed to the highly similar characteristics of higher-order modulation signals and the insufficient noise reduction capability of DnCNN. It should be noted that the classification accuracy of QPSK and OQPSK is lower than that of 8PSK in most of the SNR intervals, although their modulation order is smaller than that of 8PSK. This is because the QPSK and OQPSK signal patterns are too similar, and they are very easy to confuse when the noise reduction effect is limited with the DnCNN method.According to the In order to further verify the performance of the proposed DRdA-CA, the demodulation experiments after denoising have also been done. We present two higher-order modulation modes, 16QAM and 64QAM, since their demodulations are more challenging in the low SNR range for practical communication systems. The RLS algorithm with better performance in the traditional methods and the DnCNN are selected as the comparison methods. From In this paper, a dual residual denoising autoencoder with channel attention (DRdA-CA) is designed based on CNN to deal with the problems caused by a low SNR in wireless communications. First, as a channel attention mechanism, the SE module with one residual connection is adapted and applied to the proposed DRdA-CA to improve the feature extraction ability of neural networks. Then, the other residual connection between the different coding (decoding) blocks is introduced to solve the network degradation problem, which promotes the fusion of model characteristic information at different depths, and further enhances the network noise reduction performance. Moreover, a dataset including eight modulation types is created under the AWGN channel model.The ablation experiments are done to verify the efficiency of the improved SE module and the dual residual connections, and the dual residual connections are more helpful in improving the denoising performance. Then, the experiments on SNR improvement, modulation classification, and demodulation are also done to verify the advantage of the proposed method. The simulation results show that the proposed DRdA-CA surpasses the traditional denoising algorithms and the DnCNN noise reduction method in both improving the SNR and recovering the original signals for all eight modulation modes. Taking GMSK and 8PSK as examples, the proposed DRdA-CA improves the SNR by 16.84 dB and 9.47 dB on average, respectively. Compared to the DnCNN denoising method, the proposed DRdA-CA makes the recognition accuracy of each modulation signal increase significantly. Even the recognition accuracy of 16QAM and 64QAM goes up steadily with the increase in the SNR. Besides, demodulation performance is also dramatically improved after using the proposed DRdA-CA to reduce noise. Taking the SNR of 8 dB as an example, for 64QAM signals, the proposed DRdA-CA lowers BER by 43.34% compared to the DnCNN method. These experiments further demonstrate our method’s strength. In the future, we will extend our designs and make it suitable for the fading-channel scenario, where modulation signals are prone to be affected by multipath propagation and the Doppler frequency shift.


This paper proposed an improved flow-focusing microchannel with a constricted continuous phase inlet to increase microbubble generation frequency and reduce microbubbles’ diameter. The design variables were obtained by Latin hypercube sampling, and the radial basis function (RBF) surrogate model was used to establish the relationship between the objective function (microbubble diameter and generation frequency) and the design variables. Moreover, the optimized design of the nondominated sorting genetic algorithm II (NSGA-II) algorithm was carried out. Finally, the optimization results were verified by numerical simulations and compared with those of traditional microchannels. The results showed that dripping and squeezing regimes existed in the two microchannels. The constricted continuous phase inlet enhanced the flow-focusing effect of the improved microchannel. The diameter of microbubbles obtained from the improved microchannel was reduced from 2.8141 to 1.6949 μm, and the generation frequency was increased from 64.077 to 175.438 kHz at the same capillary numbers (Ca) compared with the traditional microchannel. According to the fitted linear function, it is known that the slope of decreasing microbubble diameter with increasing Ca number and the slope of increasing generation frequency with increasing Ca number are greater in the improved microchannel compared with those in the traditional microchannel. Nowadays, microbubbles have become one of the most effective contrast agents in medical ultrasound imaging or as a carrier for targeted drug delivery ,4,5,6,7.Microfluidics has been used as a promising interdisciplinary technique for generating and manipulating monodisperse and discrete micron-sized bubbles. It is possible to reduce the size of the generated microbubbles by adjusting the flow parameters and narrowing the orifice structure to produce microbubbles with very narrow dimensions. It has the advantages of monodisperse bubble generation, easy control of bubble size, small sample volume, and low cost. Standard microfluidic techniques for generating microbubbles can be divided into three main types: the T-type, coaxial flow type, and flow-focusing type. The T-joint device has a simple structure, including liquid and gas channels that vertically intersect with the liquid channel . Microbu5 Hz by properly adjusting the flow rates of the gas and liquid. Sheng et al. [Numerous visualization experiments have been conducted to investigate how microbubbles are produced in microchannels. Dietrich et al.  used flog et al.  first reIn addition to experimental methods, numerical simulations based on computational fluid dynamics methods provide detailed fluid dynamics information and become an effective tool for studying multiphase fluid flows. Jia and Zhang  conducteIn summary, most researchers have studied the two-phase flow from its operational parameters and physical properties. Most early studies only considered the role of individual geometrical parameters without integrating the effect of geometrical parameters on microbubble generation. There was a lack of studies on the optimal design of flow-focusing microchannels. Geometry in flow-focusing devices plays an essential role in microbubbles’ size and generation frequency. In addition, these effects are necessary for the optimal design of microbubble generation devices. Therefore, it is crucial to study the geometry of flow-focusing microchannels.This paper proposed an improved flow-focusing microchannel with constricted continuous phase entrance. A two-dimensional numerical model of microbubble generation in a flow-focusing microchannel was established using the VOF method. Four design variables of the microchannel were selected, and the radial basis function (RBF) surrogate model was used to establish the relationship between the objective function (microbubble diameter and generation frequency) and design variables. Based on this, the NSGA-II algorithm’s optimal design was carried out. Finally, the optimization results were verified by numerical simulations, and smaller microbubbles with higher generation frequency were obtained in comparison with the traditional microchannels. The fluid flow characteristics of microchannels were also further studied to investigate the effect of microchannels on microbubble generation.=The flow-focusing microchannel is a typical axisymmetric microchannel, consisting of a dispersed phase inlet and two continuous phase inlets to form focusing holes. The microbubble generation is achieved by passing the dispersed and continuous phases through the focusing holes. The geometry of the flow-focusing microchannel is shown in 3,3,2), and the liquid phase flow (water) penetrates into the main passageway in the connection area. At the junction, the gas phase transforms into bubbles. At the end of the fragmentation process, the generated bubbles flow in the straight channel to the outlet.In this paper, water was selected as the continuous phase :Inlet: Both phases are uniform inlet, and the flow velocity of the continuous and dispersed phase inlets are set to 1 m/s and 0.2 m/s, respectively, to ensure that microbubbles can be generated in the dripping regime.Outlet: The ambient pressure is used as the outlet fluid’s reference pressure, equal to 0 Pa.Wall: The contact angle is set to 50°, and the channel wall is set to no-slip boundary condition.In addition to the control equations, the relevant boundary conditions of the flow-focusing microchannel are as follows:−6. In the simulation, the variable step size is automatically adjusted according to the criterion that the global Courant number is less than 0.25, thus maintaining the stability and convergence of the solution.In this study, a double-precision pressure solver was used for numerical simulations. The pressure implicit algorithm with operator splitting (PISO) is used to implement the pressure level coupling, which is based on a higher order of the approximate relationship between pressure and velocity corrections. Pressure staggering option (PRESTO) is chosen as the pressure interpolation algorithm; the momentum term is in the second-order windward form, and the volume fraction is in the geo-reconstruct format. Due to the high accuracy of curvature calculation, the geometric reconstruction format is used for interface interpolation. The convergence criterion is set to 1 × 10) are selected for the meshing from cases 1 to 7. As shown in =Usually, the smaller the grid is, the more accurate the calculation results are; however, the smaller the grid is, the more computational resources are consumed. In order to find the most economical mesh size, the unstructured mesh is used to mesh the 2D geometric model of the flow-focusing microchannel. Since different design variables affect the number of meshes, seven scale sizes  method can select the final optimal solution from the Pareto front .The dimensions and scales of different objective spaces in multi-objective optimization problems should be unified, since the TOPSIS method used utilizes the Euclidean dimensionless objective In the TOPSIS method, the spatial distance between each solution on the Pareto boundary and the ideal point represented by The solution distance from the non-ideal point is denoted by The calculation of relative intimacy is shown as follows:In the TOPSIS method, all solutions are ranked, and the final solution required is the solution with the maximum value; therefore, the compromise of the final selected solution isThe RBF neural network training model uses 30 of the 40 sets of data from the sample space that the Latin hypercube sampled, while the post-training prediction model uses the remaining 10 sets. This post-training prediction model then necessitates an analysis of the microchannel design variables and the fitting accuracy of the objective function. The coefficient ications .(18)R2=It can be seen in In sensitivity analysis, the input variables of Pearson and Spearman linear correlation analysis are correlated with a certain objective function ,37. The For the generation frequency of microbubbles, the constriction angle was the most important influencing factor (−87%), negatively correlated with the generation frequency. This is followed by the expansion angle (−67%) and the continuous phase inlet constriction width (−38%). The outlet width has the least effect on the frequency (31%), positively correlated with the generation frequency.The optimization aims to obtain the best target values (highest generation frequency and minimum diameter). The diameter is measured by the microbubbles produced. The formation frequency is calculated from the time difference between the microbubbles’ separation from the orifice and the moment of the previous bubble separation. It can be seen from Most research efforts have focused on fluid properties and operating conditions for traditional flow-focusing microchannels. In this paper, improved performance was achieved by enhancing the flow-focusing microchannel, and the performance of the improved microchannel is compared with that of the traditional microchannel . To be cThe time-dependent pressure and velocity contours near the intersection allow us to understand the generation of microbubbles under different microchannels. It is clear from the pressure contours  that theFor the velocity contours, the maximum velocity in the improved and traditional microchannels during fragmentation occurs during the pinch-off stage (see t = 10.36 μs in The evolution of the thread profile helps understand the fluid dynamics behind the generation of microbubbles in different microchannels. gion see .For the improved microchannel, the The contribution of pressure in different microchannels to the fragmentation mechanism during microbubble generation can be further investigated by observing the pressure gradient with time near the orifice . The preIn both microchannels, pressure changes exhibit similar behavior on the time scale. The improved microchannel a has Pd>ochannel b, it canThe pressure fluctuations at the inlet of the main channel shown at the p-point during microbubble generation in traditional and improved microchannels are given in In this paper, numerical simulations and optimization algorithms investigated the fluid flow characteristics of microbubble generation in an improved flow-focusing microchannel with a constricted continuous phase inlet. A two-dimensional numerical model of microbubble generation in a flow-focusing microchannel was developed using the VOF method. Four design variables were obtained by the Latin hypercube sampling method, and the RBF surrogate model was used to establish the relationship between the objective function and design variables. The


The interest in therapeutic monoclonal antibodies (mAbs) has continuously growing in several diseases. However, their pharmacokinetics (PK) is complex due to their target-mediated drug disposition (TMDD) profiles which can induce a non-linear PK. This point is particularly challenging during the pre-clinical and translational development of a new mAb. This article reviews and describes the existing PK modeling approaches used to translate the mAbs PK from animal to human for intravenous (IV) and subcutaneous (SC) administration routes. Several approaches are presented, from the most empirical models to full physiologically based pharmacokinetic (PBPK) models, with a focus on the population PK methods . They include the translational approaches for the linear part of the PK and the TMDD mechanism of mAbs. The objective of this article is to provide an up-to-date overview and future perspectives of the translational PK approaches for mAbs during a model-informed drug development (MIDD), since the field of PK modeling has gained recently significant interest for guiding mAbs drug development. Antibodies, also known as immunoglobulins (Igs), are large proteins used by the immune system . They caMonoclonal antibodies (mAbs) are antibodies naturally produced by the same clone of activated B lymphocytes or plasma cells  and diremax) is generally observed for a few days (1 to 8 days) after single dose SC administration . Th. Th65]. up) was estimated across species and scaled well with bodyweight, and the affinity to FcRn has been identified as an important factor for increasing the persistence of mAbs.The last value to translate is the clearance, and this value can be scaled with the same rules as seen in the “allometric scaling” section. In the improved model including endosomal space, the affinity to FcRn, and endosomal transit time T(w) can be translated using experimental values to improve mAbs clearance across mice, monkeys, and humans . For exaa) and the bioavailability (F). Richter et al. showed in 2020 that mAbs SC absorption was generally faster in animals than in humans due to their physiology ..77].The interpretation of PK data with observed ADA is another challenging point during the mAbs PK translation. In fact, immunogenicity will induce the development of Anti-Drug Antibodies (ADA), which, by linking with the mAb, can increase the mAb elimination. ADA alters mAb PK and mAb efficacy consequently ,80. ADA The mAbs dose selection during the drug development is a crucial point, especially for the first-in-human dose. Generally, higher therapeutic doses are required to effectively suppress soluble targets compared to membranous ones. However, Tang et al. also showed in a large review the importance of the mechanism of action  during the therapeutic dose selection rather than the nature, turnover, and abundance of the target . Even ifOur review of the literature clearly indicates that the translation of linear mAbs PK (distribution and linear elimination) is generally well described. Empirical methods extrapolated from small molecules models are largely described with good prediction results and are easy to set up. There is a consensus on the scaling of the simplified 2-compartment model and the minimal PBPK model. The allometric scaling can be used with some small adjustments of the exponent values depending on the case, with an exponent of around 0.85 for the clearance and 1 for the central volume of distribution. Physiological values used to scale mPBPK are also well characterized in the literature. Other empirical methods have been used, like time-invariant scaling or the half-life method; these methods are straightforward and easy to set up. Whereas the use of monkeys is largely preferred to predict human mAbs PK. The more recent half-life method would allow the reduced use of such animals during drug development, even to replace the use of monkeys by human FcRn transgenic mice. Use of PBPK in this context of linear PK could bring more physiological information especially for PK predictions in tissues. However, its use is not generalized yet and seems to be not always useful as empirical methods allow the prediction of human PK reasonably well.However, it should be highlighted that these translation methods are not adapted for TMDD prediction. This issue can be extended to the translation of PKPD modeling, where the impact of a target and the dynamics of the target need to be quantitatively assessed. The description of TMDD can be performed using models (2-compartment and mPBPK) described for the linear mAbs PK adding TMDD equations, using either Michaelis–Menten equation or more mechanistic TMDD models including the dynamics of the target and the drug-target complex. These TMDD equations can also be included in a full PBPK model. Moreover, recent Quantitative System Pharmacology (QSP) are increasingly used and integrate drug mechanism of action with cellular interactions and communications.QSP modeling allows for a better description of cells dynamics and their interaction with the mAb, thus enables prediction of TMDD in a mechanistic way . Such apAnother issue during the translation of mAbs PK from animal to human is the potential development of ADA. ADA can largely impact the PK of mAbs, increasing the clearance of the drug. It is necessary to distinguish a non-linearity in the PK profile due to a TMDD mechanism from an ADA effect. This is possible by the quantification of ADA during drug development and accounting for ADA clearance in the PK model. However, ADA development is species-dependent, and its impact can vary dramatically. To date, no model has been published to predict the translation of ADA and this issue is a key point during animal studies for interpretation purposes in the case of the presence of an ADA. The non-linearity of mAbs PK can also be induced by time-dependency (non-stationarity). This non-stationarity in mAbs PK is mainly found in chronic disease, like tumors, because it is often explained by the course of the disease. This point was not discussed in the article because PK time-dependency is generally not observed during animal studies. Indeed, drug treatments in animals are shorter than in patients, and disease models in animal do not model the course of the disease well. However, it is important to anticipate this phenomenon in patients PK profiles.Translation of non-linear mAbs PK is thus complex due to a combination of several issues. In conclusion, translation of TMDD remains challenging and there is no standardized method to solve this issue. Whereas first translation models were mainly empirical to consider only the linear part of the PK, inter-species extrapolation in mAbs PK has recently improved but still requires more complex models, including as much mechanistic aspects as much as possible. The most recent models still in development are full PBPK and QSP models to develop physiology in the model. Although the ability to predict non-linear patterns of mAbs should be improved by the development of these models, such approaches require a clear understanding of the mechanism of action and access to physiological information on the species  and drug-dependent parameters .