Considering various fault states under severe working conditions, the comprehensive feature extraction from the raw vibration signal is still a challenge for the diagnosis task of rolling bearing. To deal with strong coupling and high nonlinearity of the vibration signal, this article proposes a novel multilocation and multikernel scale learning framework based on deep convolution encoder (DCE) and bidirectional long short-term memory network (BiLSTM). The procedure of the proposed method using a cascade structure is developed in three stages. In the first stage, each parallel branch of the multifeature learning combines the skip connection and the DCE, and uses different size kernels. The multifeature learning network can automatically extract and fuse global and local features from different network depths and time scales of the raw vibration signal. In the second stage, the BiLSTM as the feature protection network is designed to employ the internal calculating data of the forward propagation and backward propagation at the same network propagation node. The feature protection network is used for further mining sensitive and complementary features. In the third stage of bearing diagnosis, the classifier identifies the fault types. Consequently, the proposed network scheme can perform well in generalization capability. The performance of the proposed method is verified on the two kinds of bearing datasets. The diagnostic results demonstrate that the proposed method can diagnose multiple fault types more accurately. Also, the method performs better in load and speed adaptation compared with other intelligent fault classification methods. Rolling bearings are widely used as indispensable components in modern mechanical equipment. However, the rolling bearings usually work under the severe conditions of varying speed, heavy load, variable load, and high temperature for a long time. They are vulnerable to occur deformation, abrasive wear, or other faults. These faults may lead to equipment performance degradation and even lead to severe economic loss . TherefoFrom the perspective of pattern recognition, an intelligent bearing diagnosis process based on machine learning generally include three steps: data preparation, feature extraction, and fault classification. The purpose of feature extraction is to mining or summarize representative features. This operation can present the health condition of hardware devices and is beneficial to improve the accuracy of downstream fault classification tasks. Traditional bearing fault classification methods are difficult to extract features from the raw input signals, such as empirical mode decomposition , local mAll features are naturally hand-crafted. The process of feature extraction requires much prior knowledge about diagnostic experience and signal processing technology, which needs to consume much labor and time resources. Complex and sophisticated modern equipment is difficult to extract the comprehensive and detailed internal features of rolling bearings.The feature extraction and fault classification of the diagnostic system are separately designed and performed, both of which impact the final classification result. However, the strategy cannot be optimized simultaneously.The limited inductive feature ability of shallow learning models cannot flexibly identify the complex state changes of the bearing. Fault diagnosis methods of the specific domain cannot be applied to other engineering fields. Therefore, a general-purpose method is needed to extend to new application areas.Given the above drawbacks, deep learning (DL) combining feature extraction and fault classification may provide an effective solution for the bearing fault diagnosis system. In recent years, DL has made great achievements in many application fields such as face recognition, speech recognition, and computer vision. Schlemper et al.  utilizedThe purpose of this article is to design an end-to-end bearing fault diagnosis system based on a deep convolutional encoder (DCE) and bidirectional LSTM (BiLSTM). The framework is motivated by their strong feature extraction abilities and classification effects. In essence, the DCE network is a cascade of a series of convolution neural network modules in structure. In the diagnostic field of rotating machinery, the equipment works in harsh environments and complicated working conditions. Thus, vibration signals of a rolling bearing are nonlinear and nonstationary caused by varying working conditions, along with much information irrelevant to fault diagnosis. From the mathematical viewpoint, the DCE may provide a novel solution to the bearing fault classification problem. DCE  can deepBesides, LSTM has the advantage of dealing with nonlinear and long-term dependent dynamic problems in sequence data . It can In the first aspect, the rolling bearing usually works in varying operation conditions, especially under varying speeds and loads. On this account, the nonlinear vibration signal measured by the sensor is commonly coupled and complex . If the In the second aspect, there is only one next layer structure for network propagation in the DL architecture, and the network outcome only contains the fault feature of the last layer in the feature extraction process. As the network deepens, these features will be more robust. Although traditional network frameworks can extract some robust and invariant features for bearing fault diagnosis, these features will lose some precise and detailed information that distinguishes the varying fault conditions. Known from the field of image recognition , CNN canIn the third aspect, it is well known that rolling bearings are an important component of mechanical systems. There are many interactional and coupling effects among rolling bearings and other mechanical components. Vibration signals measured from rolling bearings commonly contain complex signals of various mechanical vibrations. Vibration signals usually exhibit multiscale properties . HoweverIn the fourth aspect, DL algorithms generally employ dropout coefficients in network propagation to prevent network overfitting. The operation may weaken the importance of some features or lose the integrity of some features. To overcome the difficulty, the multifeature protection layer is introduced into the fault diagnosis system and forms a series structure behind the GMSL fusion layer. Vibration signal is a time series signal with the attribute of data dependence. The bidirectional long and short-term memory network with feature protection (PBiLSTM) is designed after the GMSL network. Based on the robust features extracted by the GMSL network, PBiLSTM considers the relationship between current and future information to extract data-dependent features. Meanwhile, PBiLSTM employs the internal calculating data of the forward propagation and backward propagation to dig more sensitive features at the same network position. This PBiLSTM network is, to a certain extent, the integration and protection of multifeature fault signals.Combining the above four innovations, in this article, a framework scheme with multikernel scale learning and multilocation scale learning cascading the feature protection layer (MLKDCE-PBiLSTM) is proposed. The scheme can adaptively extract and fuse bearing fault features from multiple network locations and time scales of the raw data.This article combines the skip connection and encoder network and proposes a multilocation scale learning network that extracts global and local features from the network layers of different depths. The advantages of this feature extraction can be accumulated in the entire network by adding multiple skip connections.Multikernel scale learning is introduced into the CNN integration module of the DL with different kernel sizes to simultaneously learn vibration characteristics at the different time scales. The advantages will be accumulated in the entire network by adding multiple kernel scale branches.The feature information fusion layer is employed to automatically fuse the feature space and optimize the rich features extracted from the multilocation learning network and multiscale learning network.The PBiLSTM network is used to deeply excavate the efferent robustness features of the GMSL network and captures dependent and sensitive fault features.Based on the above improvements, the MLKDCE-PBiLSTM scheme is proposed to extract comprehensive fault features. The MLKDCE-based network can autonomously extract and fuse useful and comprehensive features using multilocation and multiscale learning. However, the PBiLSTM-based network is designed to deeply excavate and protect high-purity features of GMSL network output. Consequently, under the complicated working conditions of varying speeds and loads, the proposed feature learning method is used to accurately diagnose various fault types of rolling bearings.The main contributions of this article are summarized as follows:The progress of this article is organized as follows. X(t) is iteratively decomposed into [X(t) is { N is the raw vibration signal length). At the ith level, nth. At (i+1)th level, 2nth; (2n+1)th nodes. Therefore, the full vibration signal can be divided into a multiscale frequency band for the precise signal analysis by MSWT, which is achieved through a recurrent filter of LPF and HPF. At the ith layer, X(t) is decomposed into i, n) (X(t) can be decomposed into eight different time-frequency subspaces by a three-level MSWT.Various methods of transforming and extracting features of the original data are used for fault diagnosis of bearings. MSWT can observe the signal gradually from coarse to fine with the multiscale characteristics, which is adjustable for various frequencies in the time domain sampling step. The rules of slow changes under low-frequency signals and rapid changes under high-frequency signals have better effects for diagnosis tasks. MSWT overcomes the inflexible time-frequency window characteristics of Fourier transform. In short, using low-pass filter (LPF) h(k) and high-pass filter (HPF) l(k), the vibration signal sed into ,29.(1)pp. To reconstruct signals of the same length as X(t), except for the  node, it is necessary to set all the coefficients of the other nodes in the ith reconstruction node to zero. This is just to preserve the frequency information of reconstructed nodes. In Contrary to the operation of aforementioned recursive split in Formulas (1) and (2), and the reconstruction process based on MSWT coefficients can be expressed asAs the most common nonlinear unit of deep learning activation function, the choice of activation function in a deep network has a great impact on the training process and classification result. Swish is a new self-gated activation function. The researchers conducted experiments on multiple complicated datasets and proved that the Swish activation function is better than ReLU on the deep model . It is sThe Swish derivative isThe Swish activation function has the following advantages.The functions have three characteristics of lower bounds, no upper bounds, and non-monotonic.Both Swish and its first derivative have smooth characteristics.In this article, all CNN integration modules employ the activation function Swish. In the coding process, the autoencoder can perform both linear transformations with a linear activation function and a nonlinear transformation with a nonlinear activation function. When PCA performs a nonlinear data process, it is assumed that the data conform to ideal data distribution. Otherwise, PCA can only perform linear transformations .In this article, the input data is processed into an image by the Wavelet Transform. The bearing dataset is highly nonlinear and complicated. For the autoencoder, it can learn the linear and nonlinear features with encoder and decoder. However, PCA can only learn the linear features.The dimensions of the kernel PCA method are dependent on the number of input data in the eigen-decomposition. The autoencoder is flexible. In structure construction, because of the network representation form of an autoencoder, multiple nonlinear layers can be used for feature extraction.Autoencoder and PCA are similar, both of them can dimensionality reduction and feature extraction for data. However, among the autoencoder, PCA, and kernel PCA, there are lots of differences. They are summarized as follows:The autoencoder has more advantages compared with PCA.The structure of the autoencoder is much more flexible than PCA, which can process more diversified vibration data.The application of autoencoder is wider, such as data denoising, visualization and dimension reduction, image compression, and feature learning.PCA is just a special case of a single-layer autoencoder with a linear activation function.The architecture of DCAE consists of two parts: encoder and decoder, which can be seen as the combinations of convolution layer, pooling layer, deconvolution layer, and unpooling layer. This architecture employs the backpropagation theory to extract the key feature information and expresses the information through feature compression, which well stains spatial information of the 2D signal. The encoder is a neural network model that can learn and discover the hidden features of the input data. The decoder is a neural network model that can reconstruct the original input data using highly compressed features of hidden layers. Suppose we have k convolution kernels, and each convolution kernel is composed of parameters Loss is obtained by comparing the input sample and feature reconstruction result with Euclidean distance, which is optimized with the BP algorithm. The error of the DCAE is expressed asLoss represents the loss of each weight The error mth input and nth output at (i-1)th layer in the feature extraction. mth input nonlinear transform and the nth output nonlinear transform and k expresses the sum of convolution kernels. nth output nonlinear transform, and DCAE is composed of multiple convolutional neural networks, so the encoder of DCAE is designed to extract and compress features of the input signal level to level. The purpose of the fault diagnosis system is to obtain high-purity discriminative features, so we only introduce the encoder to the fault diagnosis system. Generally, the architecture of a convolutional encoder is regarded as an integration of a feature learning layer, a nonlinear transform layer, a normalization layer, and a feature pooling layer. At the feature learning layer, the input of each unit is connected to the output of the previous module, and the local features are extracted by the convolution kernel. In the nonlinear activation layer, the features of the lower dimension are mapped to the higher dimension space by selected activation functions. This convolution function can be expressed asMultikernel branch of the proposed MLKDCE-PBiLSTM employ convolutional encoders. The specific operations are described in The standard RNNs  structure is a chain form of repeated neural network modules, and a directed connection is established through mathematical relationships. Different from the basic model structural multilayer perceptron, RNNs can map the target vector from the entire input history input, while the multilayer perceptron can only map from the original input to the target vector. Due to the characteristics of RNNs that allow historical states to be kept in the memory of the network, for supervised learning, RNNs can be repeatedly trained through backpropagation. Due to the characteristics of RNNs that allow historical states to be retained in the memory of the network, for supervised learning, RNNs can be repeatedly trained through backpropagation. To capture the semantics in the long sequence, we need to run the RNN on multiple time steps and turn the unrolled RNN into a deeper network. However, this method brings about the gradient vanishing situation as RNN training, which seriously affected the accuracy of the fault classification. This means that traditional RNNs may not be able to capture long-term dependencies.The emergence of LSTMs solves the above gradient vanishing and gradient explosion problems. In LSTMs, memory units including input gates, forget gates, and output gates replace each neuron in hidden layers of RNN. In each component, input gates update the unit states; forget gates selectively discard certain information and reset memory units to prevent the long-term dependence; output gates output unit states. For capturing the valid dynamic characteristics of nonlinear time series data, LSTMs perform more advantages than traditional CNN. LSTMs have been successfully used in speech recognition, natural language processing, subtitle translation, picture description, and many other occasions. The MLKDCE-PBiLSTM in this article employs PBiLSTM to build a time series model. PBiLSTM consists of two ordinary RNNs, a forward RNN that uses past information, and a backward RNN that uses future information. The network diagnoses more accurately than the prediction result obtained by using LSTM alone. When extracting fault features at time t, each unit gate can simultaneously apply the calculated data at t−1 and t+1 time. The structure of PBiLSTM is illustrated in t depends on The final output data at time This article intends to provide a network scheme that can automatically learn the multifeature from various time scales of the input data by the varying skip connections and the multiscale learning. The operation of fusing the feature space can improve the fault diagnosis performance of rolling bearings under variable conditions of loads and speeds. Especially, the multifeature protection layer is cascaded to the GMSL network of the fault diagnosis system. A reliable intelligent fault diagnosis system should accomplish comprehensive and in-depth feature extraction of the vibration signal, and simultaneously performs global and local features learning. The core contribution of this article is to construct an end-to-end framework that integrates feature transformation, multiscale learning, multifeature fusion, multifeature protection, and fault classification. The network combines multiscale learning and multifeature protection to comprehensively and deeply extract the signal feature. The overall MLKDCE-PBiLSTM framework of bearing fault diagnosis is shown in The fusion features of GMSL are relatively pure . The features are fed into the feature protection module for sensitive and ultimate feature extraction that all features are re-extracted by the network and given more reasonable weight coefficients at this time. In this case, it is more conducive for PBiLSTM to extract new and in-deep discriminative features by considering the front and back sequential relationship of the fault signal. Finally, the newly extracted features are fed into the softmax layer, so that the probability distribution of each sample is clearly obtained. Thus, the multiscale features extracted of MLKDCE-PBiLSTM are much more robust with precise details, which effectively realizes the task of feature recognition and fault classification.The bearing fault diagnosis method with multilocation scale learning and multikernel scale learning is defined as the generalized multiscale learning. The method has been verified to be much more robust in this article.In the academic research of diagnosis task, only the single layer of the convolutional encoder network is commonly employed for the next layer input in network propagation, and the network outcome only contains the fault information of the last layer in the feature extraction process. Although the deep features extracted from the last layer of multilocation feature learning (MLFL) are more invariant and robust than the features of the lower layers, the multilayer convolution operation may lose many sensitive and detailed features that exist in the middle layer. Therefore, it is insufficient to directly use the features extracted from the last layer.In each branch, MLS considers the feature mapping which contains a certain middle layer of the network and the last layer of the network by a skip connection as the input of the feature fusion layer. Therefore, the network can learn the fault features of different convolutional layers, effectively combining the MLFL with the convolutional encoder. The core of MLFL is to learn invariant features  and detailed features  of vibration signals in different network locations. So better classification performance can be achieved. Specifically, MLFL uses skip connections in the multilayer convolutional structure to select the middle layer (one or more) of the network at different locations to combine with the final convolutional layer. The network can simultaneously learn the discriminative fault features of different locations and adopts the convolution operation to fuse multilocation features across channels.X(t) at different locations in the network. The output of MLFL fusion layer is expressed asi denotes the ith feature map of the (n-1)th convolutional layer. j denotes the jth feature map of the nth largest pooling layer, n-1)th layer and the nth maxpool layer, respectively. In the illustration, we only use the penultimate convolutional layer as skip layer for the limited computing power. According to ion (BN) .First, The core of multikernel feature learning (MKFL) is to learn the multiscale complementary features of vibration signals in different time scales. MKS can skillfully adjust the size of the convolution kernel to enable CNN integration modules to extract the fault characteristics of different time scales. The module successfully combines the MKFL with the encoder. Specifically, MKFL uses several parallel network branches, that is, each CNN integration module in the multiple branches has different sizes of the convolution kernel. Therefore, the network can learn the rich vibration features of different scales at the same time, and use the convolution operation to fuse the features of multiple time scales across channels.X(t). The output of the MKFL layer of each branch is expressed asAccording to The outputs of n MKFL branches are connected into feature vectors In this article, to give full play to the comprehensive capabilities of fault feature extraction, each network branch is designed to simultaneously embed the MLS module and MKS module. The combination is regarded as a new network module with comprehensive feature learning (CFL) module. This is more conducive to extract abundant and in-depth feature of the raw input signal. The output of each CFL branch network is The proposed GMSL fuses the fault feature of multiple network locations and multiple time scales. Such deep features are much more abundant and complementary with precise details. However, these robust features may not promote each other, resulting in the features weakening. Therefore, it is necessary to use a valid feature fusion mechanism for the multikernel structure. The features of multiple CEL branch network are the different levels of understanding for the raw signal.X(t) can be obtained by In the following two aspects, MLKDCE uses the multifeature fusion method. One is to employ the feature fusion layer In the bearing fault diagnosis system, the fused multifeature vector is directly put to the classification layer. The operation may weaken the importance of some features or even lose the integrity of some features, and may not performs the best classification effect. Therefore, it is necessary to use valid feature integration and protection mechanisms. The basic LSTM network is limited in its ability to effectively use the context. In the process of bearing fault feature learning, there is a strong dependence between sequential perception data. Considering the connection between current and future data is a kind of protection for feature integrity. PBiLSTM has achieved remarkable success in feature extraction of dependent sequence data ,34. TherPBiLSTM can process the sequence data forward and backward through two bidirectional units which are fed forward to the same output layer .(13)it→Then, the representation of complete PBiLSTM hidden unit In this article, the classification task of rolling bearing is a multi-classification task. Generally, the features obtained by the last layer of the traditional BiLSTM network are vectorized and then sent to the conventional fully connected layer and feature classifier. This approach makes the parameters of the fully connected layer very large and prone to overfitting. Thus, in the output layer, we use the global average pooling layer (GAP) instead of the fully connected layer (FCL)  and outpkth class is calculated as (24). The diagnostic output is the fault label corresponding to the maximum It is assumed that there are n types of input samples, and the output probability For MLKDCE-PBiLSTM training, we use the cross-entropy as the loss function, which is the absolute value of the true class label and predicted class label. The Adam optimization algorithm  is adoptMeanwhile, the proposed MLKDCE-PBiLSTM scheme is general and flexible, which may have multiple network branches and different convolution depths in each branch. Specifically, each branch can have a different skip location and different convolution kernel scales. MLKDCE-PBiLSTM can effectively learn abundant and complementary diagnosis information at different time scales by using the skip connection structure in multiple branches. PBiLSTM can effectively capture abstract fault features by adopting the hierarchical learning framework in multiple branches.In the bearing fault diagnosis, to verify the effectiveness of the MLKDCE-PBiLSTM in the complex feature extraction, this article conducts several experiments. The experimental data are the bearings datasets of Paderborn University (PU) and Case Western Reserve University (CWRU).Lessmeier et al. provided a PU benchmark bearing dataset for bearing condition monitoring and diagnosis  and descAll the bearings are stalled in the experiment system. The modular setup in the test rig is designed to collect the PU datasets in various load conditions. The platform is shown in To simulate the varying working conditions of rolling bearing as much as possible, in the following case study, this article uses the real damaged datasets to study the fault diagnosis of the motor bearing. Under the four working conditions described in According to the experimental requirements designed in the paper, four datasets with normal state, rolling element fault, inner raceway fault, and outer raceway fault are selected. Single point faults with sizes of 0.007, 0.014, and 0.021 are set on the four kinds of bearing drive-end fault types . All beaThe intelligent fault diagnosis system is an end-to-end learning method, but the types of vibration data and the methods of normalization have a great effect on its performance. The signals we collected are time series, which are the raw signals, and it generally performs poor results when are employed directly. A reasonable type of input signal is significant to the performance of the DL model. So MSWT methods and BN normalization methods are employed in this article to process the raw data, which is detailed in For the CWRU and PU datasets, we adopt a data augmentation method, which alleviates the difficulty of Few-Shot Learning . The datThe MLKDCE-PBiLSTM algorithm is implemented by the PyTorch library under Python 3.7. Model training and testing of the network are performed on workstations with Windows operating system, Intel Core i7-9750H CPU, and GTX 1660Ti GPU. In the whole process, the network learning rate is 0.001, and the batch size is 32.We compared the proposed MLKDCE-PBiLSTM with five advanced methods. They are a DCAE network with five-layer convolutional network , BiLSTM The neural network based on multiscale learning proposed by Jiang et al.  solved tMultilocation learning: The MLKDCE-PBiLSTM employs skip connections in the branch network to perform multilocation feature learning. The MSCNN neural network employs multiscale coarse-grained operations to down-sample the raw signal, which is probable to lose some features of the input signals.Multikernal operation: In the MSCNN structure, three branches are copy networks, and the extraction of information is insufficient. However, MLKDCE-PBiLSTM uses multiple parallel encoder branches with different convolution kernels and network parameters to extract multiscale fault features.Multifeature fusion: MSCNN does not adopt any feature fusion method, and directly puts the learned features into the final classification layer. The MLKDCE-PBiLSTM uses a multifeature fusion layer to optimize the fusion and optimization of the characteristics learned from multilocation learning and multiscale learning. The network scheme improves the accuracy of the model.Multifeature integration and protection operation: The MLKDCE-PBiLSTM uses a multifeature protection layer to extract long-term dependent fault information in the vibration signal after multifeature integration processing. It is used to maximize the integrity and accuracy of the fault features. However, other comparison networks directly perform dropout or classification operations, which will affect the accuracy or even lose important information.TP and TN refer to the numbers of true positive classes and true negative classes respectively. FP and FN denote the number of and false positive classes, false negative classes. The indicator ranges from 0 to 1. The larger the value of accuracy, the better the fault diagnosis performance.The fault diagnosis problem studied in this article is essentially a multi-class classification task . We use the indicator of accuracy, which is a generally comprehensive indicator metric defined in (25).Domain adaptation is a significant bearing diagnosis task under variable loads and speeds. It uses the knowledge gained from the training dataset to improve the performance of the network in the test dataset, that is, the study is one of the transfer learning. We cannot obtain the data and distribution of rolling bearing under various working conditions. Therefore, the model trained under the existing load states should accurately diagnose the faults under the new working condition. In this case, the training data and test data should conform to the same characteristic domain and class domain, but the characteristic distribution is inconsistent. In the real world, both the inconsistency of characteristic distribution and the inconsistency of characteristic domain and class domain objectively exist. The above contents are the research content of transfer learning. In this article, two kinds of datasets are used to verify the network domain adaptability.We design five experiments, 0.7–0.7 Nm (A), 0.1–0.1 Nm (B), 0.7/0.1–0.1/0.7 Nm (C), 0.1–0.7 Nm (D), and 0.7–0.1 Nm (E) experiments. 0.7–0.1 Nm means that the training dataset is 0.7 Nm load under the speed of 900 rmp, and the test dataset is 0.1 Nm load under 1500 rmp speed, others are similar. This experiment setup can not only verify that the data feature distribution of the training dataset and the test dataset is consistent, but also verify that the feature distribution is inconsistent.The testing results repeating three times are shown in We analyze the accuracy trend of the experimental results from three aspects. (1) The performance of six models in the prediction experiments of group A and B is better than the prediction accuracy of group C, D, and E, respectively. This can be explained by using the consistent distribution of nonlinear data features. (2) The maximum experiment accuracy appears in groups A and B, the minimum experiment accuracy appears in group C, and the accuracy of group D and E is between the above-mentioned load accuracy. This can be explained by the concept of subspace learning. The more the similarity of the subspace, the higher the accuracy of the prediction. In the test of A and B, the principal component features distribution of the two data is consistent, and the correlation is extremely high, so the accuracy of the diagnosis result will be high. In the C group test, the training dataset and the test dataset are a mixed distribution of multiple loads features that the complexity of its subspace is the highest, and the correlation analysis becomes complicated. Therefore, the experiment accuracy compared with the above same characteristic distribution will decrease. For the test experiment of D and E group, from the training dataset to the test dataset is from one feature distribution to another, the accuracy is higher than the above situation. (3) The accuracy of the test experiment D group is always higher than the accuracy of the E group. Basing the vibration knowledge of rotating machinery in dynamics, the smaller the external excitation is, the smaller the rolling bearing vibration response is, and the corresponding vibration characteristics are weaker too. Especially, the load and speed represent the external excitation force. Thus, the discriminant vibration characteristics extracted from a bigger excitation may be weak in a smaller excitation. Contrarily, the discriminant vibration characteristics extracted from a smaller excitation are usually retained in a bigger excitation. Therefore, in the bearing fault diagnosis test, the accuracy of the E group is lower than that of the D group.The accuracy of the MLKDCE-PBiLSTM in the C group is only 90.97%. Although the accuracy was not as high as under other working cases, the MLKDCE-PBiLSTM result is the best compared with the other five calculation models. The result indicates that our network has a strong generalization ability. Also, MSCNN and LeNeT-5 have better domain adaptability than the other three comparison models. The accuracy shows that multiscale learning can efficiently extract abundant and abstract fault features from the input signals. However, the accuracy of the above two models is 3% and 2% lower than that of the MLKDCE-PBiLSTM. It again indicates that MLFL and MKFL have stronger feature learning ability and feature fusion ability. Besides, an interesting phenomenon can be discovered that BiLSTM also obtains outstanding performance under various load domain adaptations. Thus, we think it is an available neural network with stronger feature extraction ability. In summary, the MLKDCE-PBiLSTM shows the best fault diagnosis performance and generalization capability in the domain adaptation test.Datasets of F–J contain 10 bearing fault conditions under 1, 2, and 3 hp load. For the datasets of H, all samples of 1 hp and 3 hp are employed as the training set and 2 hp are employed as the test set. For datasets of I and J, the training data is obtained under the load of 1 hp and 3 hp, respectively, and 3 hp and 1 hp are used as the test data.Similar to the above PU tests, the testing results repeating three times are shown in The testing time of the six models is shown in The performance of a rolling bearing fault diagnosis system has close relations with the quality of network learning outcomes. The core contribution of the MLKDCE-PBiLSTM is to learn and fusion various discriminative fault features with multilocation and multiscale learning; finally, the feature is fed into a multifeature protection module. To accurately evaluate the result of feature learning advantages of each part of the model and the ability to fuse abundant and complementary features, we will explore the impact of different scales features on the classification effect from the following three aspects. The domain adaptation experiments of load and speed are implemented under the data of groups C and H, and the test accuracy of each epoch is shown in the process.To facilitate the representation of the network structure in the subsequent research, four basic modules of the encoder are set up, including DCNN-M0 (only the basic CNN), MLS-M1 (with skip connection based on the M0), MLS-M2 (only the DCE), and MLS-M3 (with skip connection based on M2). We name the multilocation scale deep convolution encoder as MLDCE.To accurately evaluate the performance of MLS learning, four network structure, including MLDCE-M0, MLDCE-M1, MLDCE-M2 and MLDCE-M3, are set in this experiment.The experiment results are shown in According to To accurately evaluate the performance of MKS learning, three parallel deep encoder network structures, including MKDCE-B1(n = 1), MKDCE-B2(n = 2), and MKDCE-B3(n = 3), are set in this experiment. It is worth noting that the abovementioned three MKS networks do not include the MLS modules, so we call them multikernel scale deep convolution encoder (MKDCE).The test results are shown in It can be seen that the performance of network mapping is enhancing along with the network width. That is, this advantage will be accumulated in the whole model by more network branches. In practical applications, we can select an appropriate number of MKLF branches for testing according to our own needs and hardware configuration.To accurately evaluate the performance of the multifeature fusion, the MLKDCE, MLKDCE-NLF (no multilocation fusion layer in the MLS), the MLKDCE-NBF (no multikernel feature fusion layer in the MKS), and the MLKDCE-NLF-BF (no multilocation and multikernel feature fusion layer) are set in this experiment. The GMSL structure is designed in the four networks.The test results are shown in To accurately evaluate the performance of multifeature protection, the MLKDCE-PBiLSTM and MLKDCE (no multifeature protection layer before the classification layer) are set in this section.The test results are shown in To extract the multiscale and sensitive feature from the complicated vibration signals, this article proposes a novel MLKDCE-PBiLSTM scheme suitable for the rolling bearing intelligent fault diagnosis under varying conditions of load and speed.Unlike the traditional multiscale structure, MLKDCE-PBiLSTM combines the skip layer and the last layer of the encoder in each branch, which uses MKS and MLS modules in all GMSL branches. In this way, the multiscale features have stronger invariance and robustness  with precise details . Then, the former network of MLKDCE-PBiLSTM is fed into the feature protection layer for further mining sensitive and complementary features. The multifeature protection layer can deeply mine and protect weak and sensitive fault feature information from the high-purity feature representation of multiple signal components . Thus, the MLKDCE-PBiLSTM architecture can effectively diagnose the fault states of the rolling bearings. Compared with the five latest networks with respect to the load and speed adaptability, our method is more accurate and robust. Experimental results prove that multilocation scale module, multikernel scale module, multifeature fusion, and multifeature protection layer can significantly improve the performance of traditional encoders. Therefore, the MLKDCE-PBiLSTM architecture is convinced to be effective applied in the field of intelligent fault diagnosis on the rolling bearings.In future work, we intend to optimize the network structure for reducing the number of parameters and improve the model stability. In addition, in recent years, deep learning methods have been increasingly applied in the fault diagnosis field. Embedded learning is booming, which is a combination of software and hardware. However, the application of deep learning in embedded systems is less. We want to integrate deep learning into embedded learning for the bearing fault diagnosis in the future. The robustness and effectiveness of the proposed method make it promising and possible for fault diagnosis.


The effect of population structure on evolutionary dynamics is a long-lasting research topic in evolutionary ecology and population genetics. Evolutionary graph theory is a popular approach to this problem, where individuals are located on the nodes of a network and can replace each other via the links. We study the effect of complex network structure on the fixation probability, but instead of networks of individuals, we model a network of sub-populations with a probability of migration between them. We ask how the structure of such a meta-population and the rate of migration affect the fixation probability. Many of the known results for networks of individuals carry over to meta-populations, in particular for regular networks or low symmetric migration probabilities. However, when patch sizes differ we find interesting deviations between structured meta-populations and networks of individuals. For example, a two patch structure with unequal population size suppresses selection for low migration probabilities. This approach can be motivated by several biological systems, from the spread of cancerous mutations through colonic crypts to the invasion of ecosystems structured by rivers. One of the major goals of evolutionary graph theory1 is to assess the effect of underlying population structure on the fixation probability (the probability of ultimate fixation of a mutant)6 and the fixation time10. An important aim is to find an optimized structure to speed up or slow down the spread of a newly arising mutant13 .Evolutionary graph theory16. Replacement of individuals via the links is then exchanged with migration where the immigrant displaces one from the resident individuals. This leads to network-structured meta-populations, where a network is formed by individual populations connected via migration. An important question that arises in the application of evolutionary graph theory is thus whether the results derived for networks of individuals carry over to networks of small populations.However, to apply such models, a change in perspective is often necessary: In many biological applications, the nodes correspond to small populations and not to single individuals19. The dynamics of the meta-population is driven by the exchange of individuals between subpopulations. Nevertheless, the focus of these studies is usually quite different, as they aim to address complex questions arising in ecology by asking for the impact of such population structure. Also in population genetics, the fixation probability in subdivided populations (meta-populations) has been investigated extensively, see e.g. the study by Maruyama in 197020, who has shown that the fixation probability is not altered by the subdivision of a population into partially isolated patches under certain assumptions. Later, other types of structured populations and alternative modes of selection and evolutionary dynamics have been discussed in this context25. However, the focus in the context of population genetics is not on the network structure of the meta-population structure. Typically, in these systems individuals from every patch have the freedom to migrate to all the other patches, whereas in our system individuals migrate to a small subset of adjacent patches.Traditionally, such meta-populations have been analyzed extensively in ecology, where they correspond to fragmented habitats32. Yet, with few exceptions, see e.g.33, these studies assume a well-mixed population of patches.Also, evolutionary game dynamics and the evolution of cooperation have been studied in subdivided populationsHere, we extend the typical models of evolutionary graph theory to network-structured meta-populations. In contrast to previous work that heads into this direction, our focus is on very basic models of populations of fixed size and a small number of patches. We discuss limiting cases and show how they can be addressed with the available analytical tools. We show that the isothermal theorem holds for any meta-populations with the regular structure. Since an exact solution for the fixation probability is not feasible when the network structure is complicated, an approximation for small migration probability is used to compute the fixation probability. Finally, we discuss the two-patch meta-population and meta-star, where several leaf patches are connected only to one central patch, when the migration probability is high.N distributed in a network of M patches Eventually, the offspring of one mutant selected from patch 1 for reproduction will migrate to patch 2 and reach fixation there. The wild-type goes extinct. This happens with probability (ii)Eventually, the offspring of one wild-type selected from patch 2 for reproduction will migrate to patch 1 and the mutant goes extinct. This occurs with probability Therefore, the probability that a single mutant arising in patch 1 reaches fixation in the entire population is The simplest non-trivial case is the fixation probability in a two-patch meta-population with different local size for small migration probability Similarly the probability that a mutant arising in patch 2 takes over the whole population equalsIf we assume that the mutant arises in a patch with a probability proportional to the patch size, the average fixation probability  of Eqs.  and 3b)\documentIn the case of neutrality, =N2, Eq.  simplifiFor mate Eq.  for weakNext, we consider weak selection, For identical patch size 39.When migration probabilities become larger, our approximation is no longer valid and we need to rely on numerical approaches. Figure sing Eq.  and comp40, a close inspection reveals an interesting property: For low migration probabilities and 1: For advantageous mutations, While the fixation probability of the two-patch meta-population is very close to the fixation probability of the well-mixed populationWhile the difference to the well mixed case vanishes for While the structure remains a suppressor of selection for most values of the migration probability entclass1pt{minimaj and use a lower index to represent the state of the central patch, which is either occupied by wild-types ((i)the center is occupied by mutants and is selected for birth and its offspring migrates to one of the peripheral homogeneous wild-type patches and reaches fixation in that patch, (ii)the center is occupied by wild-types and one of the homogeneous mutant leaves is selected for birth and its offspring migrates to the center and gets fixed there, Here, we approximate the fixation probability (i)the center is occupied by wild-types and is selected for birth and its offspring migrates to one of the homogeneous mutant leaves and gets fixed there, (ii)the center is occupied by mutants and one of the leaves is selected for birth and its offspring migrates to the center and gets fixed there, Note that the number of homogeneous mutant leave nodes cannot increase if the center is occupied by wild-type individuals, i.e. 42, we find j mutant patch leaves isj mutant patch leaves isFor the star graph, depending on where the mutant emerges, two different fixation probabilities are defined: The fixation probability when a single mutant emerges in the center j in our particular case. Thus, also their ratio j, which makes our calculation of Note that the two probabilities Evaluating Eq. , we findTherefore, the fixation probability of the whole population equals1. A numerical investigation of Eq. . Doing the same comparison in the whole range of selection, we find that the meta-star under Birth–death is an amplifier when the hub has greater or equal size to the leaves, and a transient amplifier when the leaves are larger than the hub.The meta-star in low migration is equivalent to the “star of islands” discussed by Allen et al.For moderate migration probabilities, it is challenging to calculate the fixation probability. However, in the case of the maximum possible migration probability, 46 using the specific features of Martingales. The probability to reach fixation if the initial mutant arises either in patch 1, The fixation probability of complete bipartite graphs has been calculated analytically previouslyN such that the population distributing homogeneously in all the patches is obtained by replacing As discussed above, a star-structured meta-population in  in Eqs.  and 20)\document47 that a complete bipartite amplifies selection for weak selection. This can also be seen from a Taylor expansion of the difference between Eq. In the very low migration probability we implement an analytical approximation for the fixation probability using the time scale separation between migration and fixation in a single patch.(ii)39. The entries of the transition matrix represent the transition probability between different possible states. The number of states depend upon the symmetry of the network.For intermediate migration probability we compute the fixation probability numerically using the transition matrix based on approach published in Ref.(iii)50.For very high migration probability we calculate the fixation probability analytically using the Martingales introduced in Refs.We model the evolutionary dynamics using the Moran process. The code to reproduce our figures is available at https://github.com/s-yagoobi/fixation-probability.Depending on the network and the range of migration we adopt three different approaches to calculate the fixation probability:


Dynamic Adaptive Streaming over HTTP (DASH) is a promising scheme for improving the Quality of Experience (QoE) of users in video streaming. However, the existing schemes do not perform coordination among clients and depend on fixed heuristics. In this paper, we propose an adaptive streaming scheme with reinforcement learning in edge computing environments. The proposed scheme improves the overall QoE of clients and QoE fairness among clients based on a state-of-the-art reinforcement learning algorithm. Edge computing assistance plays a role in providing client-side observations to the mobile edge, making agents utilize this information when generating a policy for multi-client adaptive streaming. We evaluated the proposed scheme through simulation-based experiments under various network conditions. The experimental results show that the proposed scheme achieves better performance than the existing schemes. According to the Cisco Annual Internet Report 2018–2023), the total number of global mobile subscribers will increase from 66% of the population in 2018 to 71% of the population by 2023 8–2023, t,5. MoreoQoE refers to the degree of delight or annoyance of a user for an application or service. Many factors associated with network, device, user expectation, and content affect the QoE. In adaptive streaming, the QoE is determined by considering startup delay, packet loss rates, average video quality, video re-buffering events, and quality variations, etc. According to studies for the QoE, a user wants to view a video as clearly and smoothly as possible. The long startup delay prevents a user from viewing a video smoothly after the streaming session starts. The packet loss rate and average video quality are related to the clarity of play-back scenes to be watched by users. Once the video re-buffering events occur, the QoE is significantly degraded compared with the other influencing factors. The quality variations leading to abrupt changes of perceived quality also have negative effect on the QoE. Various schemes have been proposed to measure and evaluate the QoE, focusing on modeling using a combination of influencing factors. A linear combination QoE model is commonly used to measure the QoE in adaptive streaming . We utilIn general, many users stream videos through the same network. This causes the users to compete for the limited bandwidth of the network . The greWe present a reinforcement learning-based mobile edge framework for multi-client adaptive streaming. The mobile edge collects the information of clients and feeds them as inputs into a neural network model to determine the bitrate of video segments to be requested.We formulate the multi-client QoE fairness problem as a Markov Decision Process (MDP) to define the optimization objectives of the neural network model. The multi-client QoE fairness problem considers both the improvement of the QoE of clients and the achievement of the fair QoE.We trained the neural network model with a state-of-the-art reinforcement learning algorithm to generate the optimal policy for multi-client adaptive streaming. In training, network datasets with various conditions were used for generalizing the neural network model.We compared the proposed scheme with the existing schemes through simulation-based experiments. The experimental results show that the proposed scheme achieves better performance than the existing schemes in terms of the overall QoE of clients and the QoE fairness among clients.In this paper, we propose an adaptive streaming scheme with reinforcement learning in edge computing environments. The main contributions of the proposed scheme are as follows.The remainder of this paper is organized as follows. In this section, first we illustrate the background of DASH-based bitrate adaptation. Thereafter, we present several challenges of bitrate adaptation for multi-client adaptive streaming. Finally, we illustrate the application of reinforcement learning to adaptive streaming. Existing schemes utilizing reinforcement learning are also illustrated with respect to this.HTTP adaptive streaming standardized as DASH is the main scheme for video delivery over the Internet today. Through video transmissions based on HTTP, content providers can utilize the existing Content Delivery Networks (CDNs) at a low cost.In DASH, the videos are stored on the server as multiple segments in advance, each of which includes a part of video playback. The video segments are encoded at multiple discrete bitrates to support bitrate adaptation according to the time-varying network conditions. A higher bitrate indicates a higher quality and leads to a larger segment size. The fundamental reason for performance degradation in multi-client adaptive streaming is the lack of coordination among clients. Each client recognizes the network conditions based on bandwidth measurements according to the segment throughput. The throughput-based measurement performs well when the number of clients is small, and the network conditions are stable. If there are many clients in the same network, leading to the worsening of multi-client competition, the segment request patterns of clients make accurate bandwidth measurement difficult . MoreoveTo solve this problem, a bitrate adaptation scheme considering the trade-offs be-tween reducing traffic volume and maintaining the QoE has been proposed . ThroughThe existing solutions for multi-client adaptive streaming aim to achieve high levels of efficiency, stability, and fairness . ImproviAmong the existing schemes for multi-client adaptive streaming, we used ECAA as the comparison scheme to evaluate the performance . ECAA adThe adaptive streaming clients recognize the current state after the video segments are completely received. The bitrate of the next video segment is determined according to the states derived from the interactions between the server and the clients. In other words, the QoE optimization in adaptive streaming means finding the optimal bitrate selection policy in the current state. Reinforcement learning has the objective of maximizing the future cumulative rewards for a given environment. The agent learns a sequential policy to satisfy this goal. Therefore, the QoE optimization of adaptive streaming is appropriately modeled through reinforcement learning.The existing adaptive streaming schemes based on reinforcement learning consider the following challenges. First, the network conditions fluctuate over time and vary across the environments of clients. This complicates bitrate adaptation, as the input contexts have different importance when adapting the video bitrate. In the time-varying links with high and stable average bandwidth, the best choice for QoE optimization is to maintain a high bitrate even though the buffer level becomes low. In the opposite case, the clients need to consider the buffer level first when determining the bitrate of video segments to reduce the re-buffering ratio. Second, the bitrate adaptation should strike a balance among the various objectives, such as maximizing the average quality, minimizing the quality variations, and reducing the re-buffering events. These objectives are in conflict with each other. For example, requesting video segments with high bitrate maximizes the average quality, but the re-buffering ratio increases. Changing the bitrate at the right time improves the fairness, but the quality variations increase, and the buffer level of clients becomes unstable. Moreover, the preferences for the factors that influence QoE may vary significantly across users ,32. The The reinforcement learning-based adaptive streaming schemes enable the agents to experience various network conditions through learning ,35,36,37We used Pensieve as the comparison scheme to evaluate the performance, choosing it from among the existing schemes that are based on reinforcement learning . PensievThis section describes the basic assumptions and framework of the proposed scheme. We formulate the multi-client QoE fairness problem by using the concept of MDP and define various parameters and approaches required to train the neural network model.Before presenting the details, we illustrate the basic assumptions of the proposed scheme. All clients request the same video from the server. The network conditions perceived by the clients are derived from multi-client competition and channel variability. To this end, we ignore the effect of the slow start of TCP in simulations for training and testing the neural network model. The slow start makes gathering accurate simulation difficult because it associates network throughput with the adaptation algorithm being used. For example, the algorithm filling the buffer frequently and quickly causes the slow start phases; thus, the bandwidth utilization of clients is degraded. Next, we assume that device heterogeneity among clients does not exist; hence, each client expects the same video quality if the network conditions are similar. Finally, the clients periodically report their observations to the mobile edge. One solution for the reporting process is to add the QoE-related information to the HTTP header of the segment requests. In the proposed scheme, the clients use this method to communicate with the mobile edge. The details of capturing the segment requests at the mobile edge and modifying the request information are beyond the scope of this study. The proposed scheme ignores the computation delay from capturing and modifying the segment requests of clients at the mobile edge. In the simulations, the packet loss of links is not considered, and the delay of communication links is fixed.The agent generates and improves the policy by considering the input states. Therefore, we should carefully determine the observations to be used as the states. If the state space is extremely small, the agent experiences a loss of information in learning, leading to limitations in policy improvement. It is difficult to train the agent to generate the optimal policy for the given environments when the state space is extremely large. For the policy improvement to move in the correct direction, the states should be matched well to the optimization goal. The goal of the proposed scheme is to generate the policy that improves both QoE fairness among clients and the individual QoE. To this end, the states should be related to the factors influencing QoE in bitrate adaptation. We define the state space at time step Unlike Pensieve , we do nThe policy is determined by the states and the corresponding actions. The action taken by the agent affects the next states and the resulting rewards. The proposed scheme defines the action space at time step Setting the rewards is a critical challenge when generating an effective policy in reinforcement learning. First, we determine the basic form for the QoE of clients. Many factors influence the QoE of clients, such as the video quality, the quality variations, and the video re-buffering. The existing schemes to optimize the QoE of clients generally quantify the QoE value as a linear combination of these influencing factors.We extend the QoE value by considering the multi-client QoE fairness to concretize the rewards. If the QoE value is low, then the client needs to receive more bandwidth to improve their QoE. Contrary to this point, if the QoE value is high, then the client needs to allow the other clients to improve their QoE by sacrificing some bandwidth. The clients do not have the same QoE, even if they use the same bandwidth. Achieving a fair QoE means that all the clients use the bandwidth as much they need to improve or maintain their QoE. We formulate the utility value as a combination of individual QoE and QoE deviations among clients. The value of the multi-client QoE fairness is calculated as below.Let us explain an example to understand how the impact of individual QoE and QoE deviations varies according to the value of the weight parameter. Immediately after the video streaming starts, the buffer level is low and there is insufficient information on how to adapt the bitrate of future video segments. The risk of video re-buffering and the quality variations increase in this case; hence, the QoE of clients becomes low. By considering this situation, the agent tries to learn the policy that reduces the QoE deviations in the initial phases of video streaming. This policy means that the clients select the low bitrate at the initial phases of video streaming and increase the bitrate gradually. While the video streaming continues, the QoE of clients becomes high if the agent has learned the policy with the strategy that we mentioned before. In this case, the agent tries to learn the policy that maintains the high individual QoE and changes the bitrate at the right time. Thanks to this strategy, the QoE of other clients does not suffer from abrupt changes in the individual QoE. Furthermore, the proposed scheme does not change the bitrate unnecessarily if a sufficient buffer exists, reducing the instability in the buffer and the bitrate variations.To formulate the multi-client QoE fairness problem by using the MDP, we have illustrated the basic elements, such as the states, the action, and the rewards. In the following subsections, we present the architecture of the neural network model, the training algorithm used to generate the policy, and the multi-agent approach for considering how multiple clients compete for limited bandwidth.To learn the optimal policy, it is important to construct the architecture of the neural network model appropriately. This means that the neural network model should be sufficiently large to generate an elaborate policy, and the complexity of the architecture should be low to avoid the burden of training. We adopt a simple architecture using convolution layers and fully connected layers. The proposed scheme uses a training algorithm based on the actor–critic approach. This approach trains an actor network and critic network simultaneously . The actWhenever downloading each video segment, the agent passes the states into the neural network model. The agent selects the next action based on the policy as according to the probability distribution over the actions. After the action is performed, the simulated environment of the proposed scheme provides rewards for the received video segments to the agent. The proposed scheme maximizes the expected cumulative rewards by using a policy gradient method . The basThe neural network model is used to represent the policy with a manageable number of adjustable parameters. Therefore, updating the parameters for the actor network and critic network is important when generating the policy that can maximize the future rewards of the agent. First, each update of the parameters of the actor network is represented as follows, according to the policy gradient method.We need to estimate  network .(5)θv←θfunction . Note thFor discovering a good policy, we should ensure that the agent explores the action space sufficiently during training. To this end, adding an entropy regularization term to the parameter update rule of the actor network is a practical solution. We modify Equation (4) by considering this scheme as follows.The proposed scheme aims to optimize the performance of multi-client adaptive streaming. Therefore, we need to determine a training methodology that considers multi-client competition. From this perspective, the multi-agent training approach can be used, which is effective in the case where multiple agents should share their situations with each other and there exists heterogeneity among the agents ,43. MoreIt is important to set the appropriate number of training agents for efficient multi-agent training. In general, the content server provides video segments to many clients. However, only a few dozen clients are handled momentarily by the content server. First, we set the maximum number of clients performing bitrate adaptations simultaneously. Thereafter, we determine the number of training agents to be the same as the maximum number of clients, such that the agents experience the most dynamic network conditions in the given environments. This value is also used in the experiments as the maximum number of clients to evaluate the performance of the proposed scheme by considering multi-client competition.Each client cannot recognize the changes in network conditions due to the multi-client competition if there is no proxy or mobile edge. The agents share information about the individual QoE and the current QoE deviations. The central agent in multi-agent training plays a role in managing multiple agents by using the global knowledge for bitrate adaptation, similar to the mobile edge for adaptive streaming. The multi-agent training is conducted offline based on simulations, and the neural network model is trained during a fixed number of epochs. We deploy the trained model to the mobile edge to perform multi-client bitrate adaptation with edge computing assistance. Once the video streaming starts, the mobile edge generates multiple instances of the trained model according to the number of clients. The outputs of the instances denote the video bitrate that improves the rewards in the given states. The mobile edge adjusts the video bitrate of the clients to the outputs of the instances.In this section, we first illustrate the details of implementing the neural network model. Then, we present the network datasets utilized in training and testing the neural network model, the existing schemes for performance comparisons, and the evaluation metrics. The evaluation results are presented to compare the overall QoE and the QoE fairness of the proposed scheme and the existing schemes. Moreover, we analyzed the bitrate changes in each scheme during video streaming.TensorFlow was used to implement the neural network model . To utilWe set the number of multiple inputs passed to the 1D-CNN layers to eight and the discount factor to 0.99 while training the neural network model. The value of the discount factor determines the impact of the future steps on the current action. Setting the value of the discount factor to 0.99, the agent selects the current action by considering 100 future steps. The learning rates for the actor network and the critic network are set to 0.0001 and 0.001, respectively. The learning rate of the actor network is determined to be lower than that of the critic network to avoid unstable behaviors during training. Moreover, we control the value of the entropy weight, making it decay from 5.0 to 0.1 over a total of 100,000 training epochs, to derive the parameter update of the neural network model in the correct direction. We need to set the values of the weight parameters used in the QoE of clients to measure the multi-client QoE fairness. We calculate the rewards by adopting the values of the weight parameters used in reference [Ideally, the neural network model should be trained by emulating the actual environments of adaptive streaming. However, this method is slow because the agent must wait until all the video segments are downloaded and to explore the current environment before updating the parameters of the neural network model. The proposed scheme uses a simple simulator to model the bitrate dynamics of adaptive streaming. The simulator measures the segment download time based on the video bitrate of the segments and the available bandwidth of the network traces. The simulator then drains the buffer as much as the current download time allows and adds the playback length of the segments to the buffer. While simulating the adaptive streaming for the given network traces, the simulator keeps track of the video re-buffering events. When the buffer cannot accommodate video data, the simulator pauses the request process for 500 ms before retrying the segment requests. After downloading the video segments, the simulator passes the current states to the agent, and the agent determines the next action. The neural network model experiences hundreds of hours of adaptive streaming in tens of minutes by using this simulator.We use several network datasets to evaluate the performance of the proposed scheme and the existing schemes. These datasets include the broadband dataset provided by FCC, the 3G/HSDPA mobile dataset collected in Norway, and the 4G/LTE mobile dataset collected in Belgium ,47,48. TFor the experiments, we set up the networking model for multi-client adaptive streaming with edge computing environments, as shown in We used the existing schemes that utilize edge computing assistance or reinforcement learning in the experiments. First, the scheme, named ECAA, adopts edge computing assistance for efficient multi-client bitrate adaptation . ECAA suReasonable metrics should be defined to evaluate how much the QoE of clients is improved and how fair the QoE distribution among clients is. To this end, we measure the overall QoE and QoE fairness. The overall QoE is the average value of all the clients’ QoE for the streaming sessions. We first calculate the average QoE of clients for each streaming session as follows.The QoE fairness refers to how equally the QoE of clients is distributed. We adopt Jain’s fairness index to measure QoE fairness . This inBefore presenting the experimental results for the proposed scheme and the existing schemes, we illustrate the configuration of the video streamed to the clients, the evaluation scenarios, and the other factors. We use the Envivio-Dash3 video from the DASH-246 JavaScript reference player . This viAccording to the evaluation scenarios illustrated in the previous subsection, we first measured the overall QoE and the target reward for the proposed scheme and the comparison schemes. To analyze the reason behind the QoE improvements, we measured the individual QoE components for all the schemes according to the number of clients, as shown in As shown in In this paper, we presented the adaptive streaming scheme based on reinforcement learning in edge computing environments. The proposed scheme aims to optimize the performance of multi-client adaptive streaming. The optimization goal is achieved by training the neural network model using a state-of-the-art reinforcement learning algorithm and managing multiple clients under the supervision of a mobile edge. Through the experimental results, we have confirmed that the proposed scheme generates the optimal adaptation policy for multiple clients. As for future work, we plan to apply the proposed scheme to real network environments.


With the advancement in ceramic restorations bonded to the tooth structure, the treatment has become a practicable and conservative option to restore teeth with shape alteration owing to the high demand for aesthetics, with minimum tooth preparation. This article describes the case of a 25-year-old man who was dissatisfied with his smile. After an assessment of the clinical situation, a decision was made to place a full-mouth prosthodontic rehabilitation  with minimal tooth preparation, which figures as a conservative full-coverage approach. The restoration of an aesthetic smile resulted in the patient’s satisfaction. Bonding of all monolithic veenerlay and crowns onto minimally prepared teeth figures as conservative treatment in cases of hypodontia. Root resorption associated with orthodontic treatment is an unintended iatrogenic side effect of multifactorial etiology that occurs in almost all orthodontic treatments. The mechanisms of root resorption are similar to those responsible for bone resorption, i.e., inflammatory pressure reactions. According to Bassigny, apical root resorption occurs in 50 to 60% of cases after orthodontic treatment. Root resorption is irreversible and difficult to avoid . The preComputer-aided design (CAD)/computer-aided manufacturing (CAM) technology is one of the fastest-evolving aspects in modern restorative dentistry. The prosthetic management of these patients can be carried out by CAD/CAM technology, providing a minimally invasive approach. An increasing number of chairside systems are now available . IntraorThe materials used in CAD/CAM technology combined with the appropriate bonding technique offer good long-term resistance and allow a satisfactory aesthetic result. In addition, the minor dental preparation required for this type of procedure is fully integrated into a conservative approach . HoweverThe objective of this case report is to present a full-mouth prosthodontic therapeutic by CAD/CAM technology as an alternative to orthodontics in the case of multiple agenesis in a young patient.A 25-year-old man visited the prosthodontics department of the Odontology unit of the Clermont-Ferrand University Hospital following an aesthetic problem. He had no particular medical history. He complained about the shape of his teeth, the absence of some of them and his unaesthetic smile that he wanted to improve. The patient had been followed for 3 years by his orthodontist and treated with multi-attachments and bimaxillary retention trays during this period in order to close the spaces created by the agenesis. His oral quality of life was evaluated using the GOHAI  questionnaire . He obtaFirst, alginate impressions were made to obtain diagnostic casts. These casts were mounted in an articulator in order to analyze the case. Face photos and intraoral views were taken and are presented in Exobuccal clinical assessment showed a leftward convergence of the facial lines, an asymmetric smile, and the presence of oral corridors. The profile examination showed a lowering of the lower face, an open naso-labial angle, a concave subnasal profile with a reduced cervical-chin distance and a biretrochelia.Endobuccal clinical assessment revealed dental agenesis of the upper left and right lateral incisors and lower left and right central incisors, generalized small teeth size, infiltrated maxillary grooves, and an amalgam restoration on the occlusal surface of the lower left second molar. Occlusal analysis showed complete overbite, a left unilateral articular inversion on the two upper left molars, an Angle Class I canine and molar on the right side and an Angle Class II canine and molar on the left side. In addition, the anterior overhang is reduced, so the patient is in class II division 2. The radiological examination revealed voluminous dental pulps, apical root resorptions on the lower molars and the absence of the upper left third molar and the two lower third molars.According to the patient, his younger sister also suffers from agenesis (absence of seven teeth), but no genetic test has been conducted in the family. To improve the quality of life and limit the psychological impact of his unattractive smile, it was necessary to restore aesthetics and function. The patient had previously undergone orthodontic treatment to close the spaces created by the agenesia. Therefore, implant treatment was not possible due to the lack of space and financial reasons. The patient did no longer wish to undergo orthodontic treatment, and the presence of significant apical resorption on the patient’s mandibular molars contraindicated further orthodontic treatment. In order to meet the patient’s aesthetic needs, it was decided to realize a full-mouth rehabilitation with a fixed prosthesis. In order to realize the fixed prosthesis, it was necessary to increase the vertical dimension of occlusion. The treatment consisted of a minimally invasive prosthetic procedure for the aesthetic rehabilitation of the complete arch. The objective was to preserve the enamel in order to optimize the adhesive bond of the luting agent to both the tooth surface and the ceramic restoration. Monolithic veneerlay  to the posterior teeth (upper and lower premolars and molars) and monolithic crown to the anterior teeth (upper and lower incisors) were proposed and accepted by the patient. The prosthetic challenge is to restore an aesthetic and functional smile while preserving as much as possible the structure of the teeth.The centric relation was defined as a reference position to increase the vertical dimension of occlusion. It was recorded using an anterior deprogramming device (Lucia Jig technique), and then a silicone occlusion was placed posteriorly. The initial study casts were mounted at the new vertical dimension of occlusion (VDO) on a semi-adjustable articulator  using a facebow transfer for the maxillary and with the registration in centric relation for the mandible . The diaThese modifications were evaluated with a direct mock-up in the full-mouth prior to the initial preparation to evaluate aesthetics and function . The patSubsequently, an optical impression of the two arches with the composite resin mock-up was taken with intra-oral scanner Trios 3 . The Trios intra-oral scanner was operated on the confocal principle, with the video bitrate image capturing method based on the real-time rendering technique. The software 3Shape Dental Designer was used for modelling veenerlays and crowns.Once the volume of the final restoration was defined by the complete mock-up, the preparation of teeth for the definitive adhesive restoration could be performed with calibrated burs to achieve an occlusal reduction of 1.5 to 2 mm. A very light chamber preparation was made on posterior teeth in vestibular and proximal faces with a supragingival position, and a finish line on the cervical area of anterior teeth was positioned in the sulcus to optimize the aesthetic result . As a result, it was possible to maintain most of the enamel necessary for bonding. The bonding procedure followed a very strict operating protocol requiring the use of a rubber dam at each step. The inner surfaces of the crowns were etched with 9.5% hydrofluoric acid  for 20 s, rinsed, and air-dried. A silane-coupling agent  was then applied for 60 s. Subsequently, the surface was dried with free air. The veenerlays and crowns were individually luted, following the same sequence for each. The tooth surface was etched with 37% phosphoric acid for 15 s in dentin and 30 s in enamel, copiously rinsed, and carefully dried. Then, the adhesive was applied and light-cured with a Woodpecker LED-C curing light. The cementation was carried out using a light-curing composite cement , which was applied to the inner surface of the indirect restoration. Each prosthetic component was carefully positioned and light-cured for 5 s. Excess cement was removed, and the restorations were light-cured for 40 s using a LED unit in vestibular and lingual/palatal surfaces. The remaining excess cement was removed with a scalpel blade, and the occlusion was checked. Finishing and polishing were performed with silicone rubber polishers.The preparations and impressions were made in three separate sessions and by sector in order to always maintain the defined VDO. The management of dental treatment for the upper and lower right side is detailed as an example.Once the mock-up had been validated by the patient after 2 weeks, the first clinical session was to take an impression of the mock-up in the mouth using the Trios 3 intraoral scanner. The resulting image of the virtual working model is shown in Teeth of the upper (a) and lower (b) right pre-molar and molar sectors were prepared through the mock-up as shown in Once teeth were prepared, an optical impression of the preparations was made, and a temporary resin  was placed on the teeth with silicone keys  to proteA digital cast was obtained by intraoral scanning, and the quality of the preparations was reevaluated. Veenerlays and crowns were designed by using the Dental Designer software program. A laboratory modelled the prosthetic elements according to the mock-up, and then the design file was sent to the DWX—42 W machine from DG Shape. The laboratory’s in-house dental technician applied make-up to the prosthetics components to enhance the aesthetic appearance and then applied the glaze layer, which was crystallized in an oven according to the manufacturer’s recommendations. During the second clinical session, the temporary resin was applied, the teeth were thoroughly cleaned, and the fixed prosthetic components  were triOnce validated, a rubber dam was installed  to allowThis second clinical session ended with the removal of the rubber dam and adjustments of the bonded prosthetic components . Self-glazed monolithic veenerlays and crowns fabricated with the completely digital workflow provided efficient and satisfactory clinical performance.The final view of the upper and lower right side is shown in During treatment, the patient presented severe pain in the lower left second molar and lower right first molar that was not relieved by analgesics. After clinical examination, these two teeth were diagnosed with acute reversible pulpitis. A therapeutic pulpotomy treatment with Biodentine™ (Septodont) was performed through the overlay. The access cavity was filled with composite resin, and the occlusion was checked.Treatment lasted a total of 7 weeks (one session per week) and was followed by two additional check-up appointments to respond to the patient’s grievances.One month after the end of treatment, an increase in the GOHAI score, compared to the initial situation, was noted, with a score of 54/60. This final score, which corresponds to an adequate oral quality of life, was qualified according to the GOHAI scale. The Hue index (SD Hue) allowed the evaluation of masticatory function  and showThe patient’s new smile is shown in The GOHAI was performed at 8 months after the end of treatment, and the score obtained was 59/60, indicating an excellent oral quality of life. An orthopantomogram of the patient at 8 months after the end of the dental treatment is presented in For this treatment, it was deliberately chosen not to restore the maxillary and mandibular canines at the anterior level. Indeed, the lack of space led us to prioritize the smile by reconstituting the eight incisors in order to obtain the best aesthetic result considering the clinical situation of the subject. Additionally, for aesthetic reasons, we chose to perform vestibular preparations on the posterior teeth. Following this treatment, the lower left second molar and lower right first molar underwent a therapeutic pulpotomy ,12 that There has been a growing interest in glass-ceramic systems with good aesthetics, high fracture resistance and bonding durability, and simplified fabrication techniques using CAD/CAM. The choice of material for full-mouth restoration in this case was lithium disilicate glass-ceramic . Lithium disilicate glass-ceramic has better mechanical and excellent optical properties than conventional dental porcelain. It has become popular in a monolithic configuration ,16. SingThis case report shows the interest of a CAD/CAM procedure during oral rehabilitation in a patient with agenesis by raising the vertical dimension and being minimally invasive. The interest of the CAD/CAM with the same procedure has already been shown in extensive erosive lesions ,21 and tThe long-term prognosis is favorable and conditioned upon good oral health hygiene. According to Abduo et al. and Aldegheishem et al., this kind of restoration has a high survival rate at 5 years ,27.This clinical case is an example of functional and aesthetic rehabilitation in a patient with generalized small teeth size and multiple agenesia. The use of CAD/CAM systems has allowed time saving, tissue saving, better communication and aesthetic management with the patient during a complex prosthetic treatment.