Extended Kalman filter (EKF) is one of the most widely used Bayesian estimation methods in the optimal control area. Recent works on mobile robot control and transportation systems have applied various EKF methods, especially for localization. However, it is difficult to obtain adequate and reliable process-noise and measurement-noise models due to the complex and dynamic surrounding environments and sensor uncertainty. Generally, the default noise values of the sensors are provided by the manufacturer, but the values may frequently change depending on the environment. Thus, this paper mainly focuses on designing a highly accurate trainable EKF-based localization framework using inertial measurement units (IMUs) for the autonomous ground vehicle (AGV) with dead reckoning, with the goal of fusing it with a laser imaging, detection, and ranging (LiDAR) sensor-based simultaneous localization and mapping (SLAM) estimation for enhancing the performance. Convolution neural networks (CNNs), backward propagation algorithms, and gradient descent methods are implemented in the system to optimize the parameters in our framework. Furthermore, we develop a unique cost function for training the models to improve EKF accuracy. The proposed work is general and applicable to diverse IMU-aided robot localization models. Over the past years, localization has become one of the challenging issues for autonomous ground vehicles (AGVs). In particular, the most difficult issue in navigation is estimating the accurate and stable position and orientation of the robot through data obtained from sensors and other navigation systems . RecentlHowever, the simultaneous localization and mapping (SLAM) methods may fail to function correctly in certain complicated situations due to the physical characteristics of sensors. The laser imaging, detection, and ranging (LiDAR) sensor, for example, is a sensor system that measures distance by transmitting light into spaces and receiving reflected signals from a target . HoweverThe inertial measurement unit (IMU), a combination of a gyroscope, accelerometers, and sometimes magnetometers, could provide an efficient approach to solve this problem. Specifically, the IMU is one of the solutions that can independently measure the state of the body without any external feedback. The accelerometer and gyroscope components measure linear acceleration and angular velocity, which represent the movement of the body to which the sensor is attached. Additionally, the magnetometer component measures orientation based on the Earth’s magnetic field, which is available almost everywhere .Q, and the measurement-noise covariance matrix, R, are constructed with a priori constant values determined by the characteristics of sensors and environments in traditional KF systems, which assume that they remain constant throughout the whole navigation operation. EKF can achieve optimal results if the process noise is well defined. However, depending on external factors, such as complex environments or sensor limitations , sensor noise values can change, and it is difficult to recognize the exact error and in situ information of when and how the change occurs [Nevertheless, there is one drawback that could degrade the performance of an IMU-based localization, which is the accumulated drift. To reliably utilize the IMU, it is imperative to eliminate the accumulated drift . In the e occurs .In this work, we propose a novel approach to improve the accuracy of EKF-based IMU localization with a convolutional neural network (CNN) architecture. Specifically, we design a stable training method that can find the optimal parameters of the system and the observation-noise covariance in real time by reducing the error in each iteration. Furthermore, the system is designed and tested for online training, unlike many other approaches, such as , where tOur proposed CNN module consists of multi-head attention (MHA) layers to model the cross-modal fusion of different sources of modalities . The MHA was initially proposed to address the problems of natural language processing (NLP) , and it We conducted extensive experiments using an actual robotic platform to assess the effectiveness of our proposed method in the real world (a factory environment in our case). We designed real-world scenarios for the online training, where the SLAM might fail in some cases and only the IMU(s) can provide sensory information for the EKF-based localization module. The algorithm is also trained continuously while the robot is online and navigating.The following is a list of the major contributions of this study:Q and R at each step of the EKF process to improve the state estimation accuracy. In addition, Hu et al. [Studies have been conducted recently on Kalman filter (KF)-based localization technology with adaptive noise-covariance estimation. One previous study proposed by Akhlaghi et al.  introducu et al.  proposedIt is known that the artificial neural network has the capability to approximate nonlinear functions . Haarnojn to the body frame b, can be represented by s and c, respectively):The rotation matrix g, so the inclination angles When the IMU is stationary or moving at a constant speed, the acceleration in the navigation frame should be equal to the gravity constant Equation  as in [2The IMU coordinate frame is assumed as shown in t, which can be calculated using The continuous-time relationships among position From the continuous-time model, the dynamics of position, velocity, and orientation in discrete time are given by Equations –8), as , as 8), The state variable We linearize Equation  at the cfunction  with resSince the accelerometer measures the local gravity vector when an AGV is moving at a constant speed or is stationary, it can provide information about the inclination of the sensor . Then, tThus, we can obtain the measurement function mapping the state space to the measurement space as follows:Then, we obtain the measurement matrix Therefore, the measurement residual quations  and 20)yt and ths, the covariance matrices could be changed indirectly.It is well known that the EKF is a model-based optimal filter, which requires exact knowledge of process and measurement models as well as process- and measurement-noise statistics. However, it is difficult to model the dynamic noise changes over time . Thus, wParameters Equation . Once thv and pseudo-velocity Although the noise components of the velocity variance . A CNN lx,y,z). Using a single channel, where the overall input dimension becomes The MHA mechanism was initially proposed in the field of natural language processing (NLP) . Later, Following the definition of , QA∈Rd×dEquation . In thisSLAM works well unless a sensor fails , which iFurthermore, the iterative EKF estimation process in consecutive time steps is a kind of Markov decision process, which means that the current state is only related to the previous one, so our method focuses on optimizing each EKF estimation process. When the estimation performance of each EKF iteration is high, it will show high estimation accuracy in the entire iterative process. In order to evaluate the performance of each EKF estimation, the loss function is designed as follows:The initial state of each EKF iteration should be the same as the state of SLAM at the previous timestamp, so that the loss function can effectively express the error generated by each EKF iteration. Therefore, the input of trainable EKF that estimates the state at the next timestamp should be the state from SLAM at the current timestamp during training as follows:However, the frequency of SLAM is different from the frequency of IMU in real-time, so we cannot guarantee that the estimated state of each EKF iteration corresponds to the output state of each SLAM at the same time, which means that it is unable to regard the ground truth of EKF as SLAM at that time. Additionally, the frequency of IMU is usually higher than the frequency of SLAM, so to synchronize the output of EKF and SLAM, multiple EKF iterations can be trained by performing backpropagation  at once This section introduces the settings and the implementation details of our algorithm. The whole self-adjusting method is implemented in Python with the PyTorch library  for trais are defined as The initial parameters of the EKF were set as follows prior to training. The initial system error covariance  m/s2 in , σω = 0.ad/s2 in , σv = 0.5 m/s in . The adj= 3 from  to 26),,P0 = I10l values .The Adam optimizer  was applIn this section, we present the robotic platform that was used for collecting data and testing a real-time SLAM failure scenario. Three evaluation metrics were used for evaluating the performance. The performance of our proposed localization architecture, the trainable quaternion-based EKF, was evaluated by comparing it with the EKF model without training. Additionally, various deep learning model architectures were compared to determine the most effective model.The data collected to evaluate the proposed algorithm are based on the usage of a proprietary omniwheel robot platform of dimensions of Experiments were conducted across four different trajectories with total lengths of 66.46, 145.39, 103.42, and 78.62 meters, respectively. The trajectory path collected as a result of SLAM was used as the ground truth. For this work, the visual–LiDAR–inertial SLAM algorithm in  was usedSince the EKF state estimation was calculated at the origin of the IMU, all data points (SLAM and velocity) were transformed to the IMU’s origin. Furthermore, all data had to be transformed to the body frame coordinates of the omniwheel robot, where the x-axis runs along the length of the robot in the forward direction, the y-axis is 90 degrees anticlockwise, and the z-axis runs upward.To evaluate the improvement of the proposed algorithm, the localization output of a traditional EKF was used as the experimental control data.Our proposed system architecture was designed with the intention that the algorithm would be trained continuously while the robot is online; therefore, the data are collected in real time, and each data point is unique and independent. Taking this into account, we trained the algorithm on three sequence runs and performed the inference test on the fourth unseen sequence, with the data of each sequence only seen once during training and inference. This process was performed four times to run the inference test on all available sequences.The EKF state estimation was then fused with the SLAM state during the inference mode while the system considered the SLAM positional data reliable, which is determined by the covariance values provided by the SLAM module. The time range when the SLAM positional data were unreliable was considered the SLAM failure period, at which point the SLAM state was no longer fused with the EKF estimation, and the output was purely relying on EKF.To simulate a real-time SLAM failure scenario, 100 s time intervals within each individual trajectory were selected as the SLAM failure period. Both approaches, the traditional EKF and the proposed trainable EKF, had the same start and end times for the SLAM failure.In The ratio of the position error to the total path length when SLAM fails.Azimuth angle error relative to the total path length when SLAM fails.The overall average of the squares of the errors between ground truths (SLAM) and object to be assessed (estimated position and orientation) when SLAM fails.To evaluate the performance of our proposed algorithms, we considered the following three evaluation metrics for our experiments:We present the comparison results of our proposed algorithm with those of the traditional method . From the trajectory comparisons in Our trainable EKF CNN with MHA showed improved results over the traditional EKF estimation due to optimization of process and measurement errors through a combination of CNN inference, backpropagation, and gradient descent.In We demonstrated the advantages of our proposed trainable quaternion-based EKF with CNN and MHA based on the conducted experiments and analysis. Our proposed method could achieve 1.24765% in However, the online training cost of the proposed method can limit the overall performance. Furthermore, the time lag introduced by the online model training can be accumulated and cause deviations at the endpoints because of the limitations of the embedded computing system. Nonetheless, this issue can be solved by transmitting data to a more powerful server for online training. Furthermore, the future development of the AI-embedded platform will provide more power to achieve better online training performance. In conclusion, our results demonstrate that the deep learning model can be trained and provide predictions in an online training manner in a local integrated system.In this paper, we designed a trainable and adjustable quaternion-based EKF algorithm with CNN and MHA for the sensor fusion of IMU-based localization and visual–LiDAR–inertial SLAM. Specifically, we developed an approach where the EKF-based localization system can provide a more accurate position estimation when SLAM failure occurs during a short time period. This was performed by tuning the process- and measurement-covariance matrices trained by CNN through backpropagation and further adjusting the velocity measurement covariances according to real-time IMU data through network inference. The approach leveraged the SLAM data as ground truths to compute the mean squared error of position and orientation estimated by the EKF while training.For training and estimation, we designed a tandem EKF structure to adapt to the situation where real-time data from different sources were fused at different frequencies. Our proposed trainable EKF will be effective in dead-reckoning as a complementary process of SLAM when SLAM fails, which will enhance accuracy and stability in localization in complex and dynamic environments. Our future steps will focus on enhancing the structure of our proposed EKF CNN with MHA by performing multi-IMU fusion with multiple EKF modules, where each EKF leverages a unique IMU source.


Spectral-domain resonances for cavities formed by two distributed Bragg reflectors (DBRs) were analyzed theoretically and experimentally. We model the reflectance and transmittance spectra of the cavity at the normal incidence of light when DBRs are represented by a one-dimensional photonic crystal (1DPhC) comprising six bilayers of TiO Complex dielectric structures  such as The BSWs are electromagnetic waves that can be localized at the interface between a homogeneous medium to be analyzed and a truncated periodic dielectric medium 1DPhC) decay exponentially along the normal direction, away from the surface into surrounding media. 1DPhCs can be tuned to support BSWs in any desired wavelength range by varying the dispersion and geometry of the periodic dielectric media. Additionally, the integration of PhC technology can further improve light–matter interaction and, thus, the device’s performance [DPhC decas- and p-polarized waves [1DPhCs exhibit photonic band gaps (PBGs) in which the propagation of electromagnetic waves is forbidden inside the crystal for a specific direction of the incident waves. PBGs can support BSWs for which their excitation is possible by employing a coupling prism to fulfill the resonance condition for the surface wave, in which a strong confinement of light is attained. The excitation of BSWs is similar to that of surface plasmon polaritons (SPPs), but BSWs have several advantages compared to SPPs ,35. By sed waves  at any wed waves . Howevered waves ,39, incled waves  that outed waves .Most importantly, a direct free-space excitation of BSWs is not possible, contrary to TPs , as statIn this paper, cavities formed by two DBRs and their spectral-domain resonances at the normal incidence of light are analyzed theoretically and experimentally. A new approach based on the reference reflectance of a single DBR from the substrate side is used to model the reflectance ratio as the function of both the cavity thickness and its refractive index. For a 1DPhC comprising six bilayers of TiOTwo DBRs under consideration that form the cavity are represented by a multilayer structure (1DPhC) for which its normal incidence band gap in the reflection spectrum is approximately 280 nm wide (from 590 to 870 nm). 1DPhC was fabricated  by a sputtering thin film deposition technique, and exact parameters such as layer thicknesses and dispersions were provided by techniques of scanning electron microscopy (SEM) and spectral ellipsometry. An SEM image see a captureus paper . The SEMified in .N dielectric layers, the total transfer matrix j-th layer.In the theoretical analysis, the normal incidence of light is considered; to obtain the spectral reflectance of cavity ic media :(1)M(λ)=Spectral reflectance To model the reflectance spectrum x component of the electric field of the incident wave, as show in The reflectance spectrum 3.4×10−4 , respectCavity mode undergoing significant change bt varying the cavity’s medium, which has the potential to be used in sensing of various analytes. However, the response can be deteriorated by the source spectrum so it is desirable to use a new approach. The proposed method is based on the reference measurement of the spectral reflectance Au layer , but it Au layer . Among tUsually, the performance of the resonance-based sensor is also characterized by a full width at half maximum (FWHM) of the resonance dip or figure of merit (FOM), defined as follows: smaller . Alterna smaller  and caviNext, we consider the two DBRs separated by a cavity thickness of  sensors ,30,31,32Along with the reflectance spectrum, it is crucial to analyze the transmittance spectrum, which is important from the point of view of the spectral filtering. If we assume that the absorption losses of a dielectric multilayer structure are negligible, the spectral transmittance As an example, in relation .(9)Ts was used in the second setup shown in The spectral transmittance of air cavity formed by two DBRs was measured in the experimental setup shown in In the first step, the experimental setup shown in The mutual shift of the resonance dips in a spectral region near a long-wavelength edge of the band gap is nearly constant, as seen from the inset of To increase both the contrast of the spectral fringes and the depth of resonance dips in the reflectance measurements, the second setup see  employinOut of the band gap, the Fabry–Perot resonances are revealed and they show up as spectral interference fringes with the contrast (visibility) dependent on the reflectance measurement technique. The measurement results obtained in the first setup in a short wavelength region are shown In the next step, the first setup see  was usedchniques ,52 utiliFinally, the spectral power transmitted through the two DBRs with air cavity has been measured in the experimental setup shown in The resonance peaks near the long-wavelength edge shown in In both filtered spectra, some power is also transmitted by another cavity mode because the LED spectrum is wider than the wavelength spacing  of the cavity modes. To obtain a single-mode operation of a spectral filter, the cavity thickness needs to be as small as possible in terms of what is attainable by mirrors with substrates of a high flatness, such as horlabs) . GeneralIn this paper, DBR-based cavity resonances have been analyzed theoretically and experimentally in the spectral-domain. The reflectance and transmittance spectra of the cavity have been modeled at the normal incidence of light, and for DBRs represented by a 1DPhC comprising six bilayers of TiOThe theoretical results are confirmed by experimental ones and very high sensitivity and FOM of the narrow dips to variations of RH of moist air, namely 0.156 nm/%RH and 0.047 %RHThe proposed sensing concept, based on the use of the dielectric DBRs that are mechanically and chemically robust, offers the possibility of operation in both VIS and NIR spectral regions  and in a


In recent years, deep convolutional neural network (CNN)-based image enhancement has shown outstanding performance. However, due to the problems of uneven illumination and low contrast existing in endoscopic images, the implementation of medical endoscopic image enhancement using CNN is still an exploratory and challenging task. An endoscopic image enhancement network (EIEN) based on the Retinex theory is proposed in this paper to solve these problems. The structure consists of three parts: decomposition network, illumination correction network, and reflection component enhancement algorithm. First, the decomposition network model of pre-trained Retinex-Net is retrained on the endoscopic image dataset, and then the images are decomposed into illumination and reflection components by this decomposition network. Second, the illumination components are corrected by the proposed self-attention guided multi-scale pyramid structure. The pyramid structure is used to capture the multi-scale information of the image. The self-attention mechanism is based on the imaging nature of the endoscopic image, and the inverse image of the illumination component is fused with the features of the green and blue channels of the image to be enhanced to generate a weight map that reassigns weights to the spatial dimension of the feature map, to avoid the loss of details in the process of multi-scale feature fusion and image reconstruction by the network. The reflection component enhancement is achieved by sub-channel stretching and weighted fusion, which is used to enhance the vascular information and image contrast. Finally, the enhanced illumination and reflection components are multiplied to obtain the reconstructed image. We compare the results of the proposed method with six other methods on a test set. The experimental results show that EIEN enhances the brightness and contrast of endoscopic images and highlights vascular and tissue information. At the same time, the method in this paper obtained the best results in terms of visual perception and objective evaluation. Medical endoscopic images suffer from serious degradation problems such as uneven illumination and low contrast due to the complexity of the internal structures of the human body and the limitations of imaging technology . These dHistogram equalization (HE)  is used Secondly, Gamma Correction (GC) is also a popular method of contrast enhancement in the pixel domain, which is suitable for processing images of different luminance but requires manual setting of the appropriate gamma value. For this reason, the researchers who proposed adaptive gamma correction  (AGC) weOther traditional algorithms are based on the Retinex theory approach, which is a model of human color perception and assumes that the observed image can be decomposed into two components: the reflectance component and the illumination component, and generally, only the illumination component needs to be processed to achieve enhancement. Earlier single-scale Retinex  (SSR) anIn recent years, many CNN-based methods have shown outstanding performance in image denoising, image super-resolution, and low-light image enhancement. They have contributed significantly to the development of CNN in image enhancement. For example, a depth autoencoder is proposed in LLNet  to enhanAlthough these existing methods achieve superior results in solving the problem of partial image degradation, they still have some drawbacks when applied to endoscopic images, as shown in The main contributions of this paper are as follows:We use a migration learning approach to retrain a pre-trained Retinex-Net decomposition network model on an endoscopic image dataset to fine-tune the decomposition network weights. The generalizability of the decomposition network in endoscopic images is improved, and suitable decomposition images are provided for illumination correction and reflection component enhancement.We propose a self-attention guided multi-scale pyramid network to implement illumination correction. It can extract image features at different scales and guide the network to generate illumination components with balanced illumination and rich details through a self-attention mechanism.We combine the imaging characteristics of endoscopic images, stretch the green channel and blue channel in the reflection component, and fuse the stretched reflection component with the original reflection component by weighting to achieve image fidelity while highlighting the contrast of blood vessels and tissues.Pyramid network structure: Image pyramids are a structure that presents images in multiple resolutions and with conceptual simplicity. The pyramid structure is divided into the Gaussian pyramid and the Laplace pyramid, which are combined by two ways of image downsampling and upsampling, respectively. A typical application of them is image fusion, where the fusion process can achieve the effect of highlighting features and details. In recent years, pyramidal structures have played an essential role in convolutional neural network image enhancement. Li et al. [i et al.  proposedi et al.  used coni et al.  introducAttention Mechanism: In recent years, many attention modules have been proposed. Their development can be divided into two directions, namely, enhanced feature aggregation and channel and spatial attention combination, whose function is to suppress redundant features while emphasizing informative features. For example, Hu et al. [u et al.  proposedu et al.  uses twou et al.  used nonu et al.  proposedu et al.  used theu et al.  of the eThis section illustrates the details of the endoscopic image enhancement network (EIEN) we designed. Inspired by Retinex theory and endoscopic imaging properties, the framework of EIEN is shown in The classical Retinex theory is the modeling of human color perception. This theory assumes that the observed image can be decomposed into two components: reflectance, and illumination, as shown in Equation (1):inex-Net . In this1.Invariable Reflectance Loss: According to the Retinex theory, it is known that the reflection components are invariant by the nature of the object itself, so the reflection components of the abnormal/normal illumination images are similar. The abnormal illumination image contains both low-light and high-light images. The decomposition network uses pairs of images as input. It imposes reflection consistency constraints between the two images to guide the optimization of the decomposition network, whose loss is calculated as shown in Equation (2):2.Illumination Smoothness Loss: Abnormal and normal image decomposition does not have the same illumination components but should highlight the structure of the image and local details while maintaining overall smoothness. The direct use of TV minimization  [ization)  as a los3.Reconstruction Loss: To further constrain the decomposition of the network, we need to focus not only on the similarity between the reconstructed results of the reflection and illumination components of the image’s own decomposition and the original image. It is also necessary to pay attention to the similarity between the combined reconstruction results of the reflection component and the illuminance component of the paired image decomposition and the abnormal/normal image. Reconstruction loss is shown in Equation (4):Therefore, the total loss of the decomposition network is defined as follows:After decomposing the network to decompose the reflection and illumination components, the next step should be illumination correction. Since we have obtained two illuminance components, one corresponds to the normal illumination endoscopic image, and the other corresponds to the abnormal illumination endoscopic image. Therefore, the design idea of our illuminance correction network is to make it similar to the illuminance component of the normal image by learning the features and detailed information of the illuminance component in the abnormal image, combined with feature fusion and attention guidance.1 is adjusted to two downsampled versions, which are 1/2 and 1/4 times of the original size, named I2 and I3, respectively. These three inputs first start feature extraction from the lowest resolution image, then multiply the extracted final feature map with the attention weight map, and finally perform channel stitching with the features extracted from the branch of the previous layer, and repeat the above operation in the branch of the original size image for image reconstruction to get the normal illumination component. Among them, due to the small resolution of the bottom two branches, they can have a high perceptual field without special convolution processing. Therefore, the lower resolution branches consist of a small number of 3 × 3 convolution layers and residual structures. The residual structure is shown in Specifically, the illumination correction is implemented through a self-attention-guided pyramid module, as shown in f) of the illumination component obtained from the decomposition network and the G and B channels containing rich detail information in the endoscope to get a three-channel feature map. Then, three atrous convolutions with convolution kernels of 3 × 3 and expansion rates of 1, 2, and 5 are used to process this feature map simultaneously to obtain three single-channel feature maps. Finally, the mean values of the three feature maps are processed by the sigmoid activation function to obtain the attention weight map, which is implemented as shown in Self-attention mechanism: Although the pyramid network model can achieve illumination correction, the image may be over-enhanced, and details may be lost. For example, regions with high pixel values in higher exposures may appear over-brightened by network correction. Areas with lower exposures may not be noticed by the network during correction and cannot achieve good correction results. One possible solution is to process the image in chunks to avoid the drawbacks of global processing, but it increases processing complexity and may also cause block effects. Instead, inspired by Lee et al. , we stitÎ is the corrected light component. The total loss of the light component is shown in Equation (8).In addition, the corrected illuminance component needs to be considered not only for its illuminance smoothness but also for the similarity of the reconstructed result with the normal illuminance image after elemental multiplication of the corrected illuminance component and the reflected component. Therefore, we use the combination of illuminance smoothing loss and reconstruction loss as the total loss of the illuminance correction network, where the illuminance smoothing loss :Although the stretched reflection component significantly enhances vascular and tissue contrast, reconstructing the image color produces severe color distortion leading to visual discomfort. To solve this problem, we retain only part of the detailed information of the stretched reflection component and make it weighted with the original reflection component to obtain a reflection component with great image fidelity details. The formula is as follows:The experiments were conducted on a dedicated facility in our lab, which was configured with an Intel(R) Core(TM) i7-7700K CPU@4.20GHz and equipped with a GTX1080Ti graphics card with 11 GB of video memory. We implemented the model in the PyTorch  frameworIn this section, some experimental results are given. Specifically, we first present the dataset used for training and testing. Then, we perform a subjective and objective analysis of the proposed EIEN method and several classical image enhancement methods. Finally, we perform an ablation study to verify the effectiveness of each component in EIEN.To our knowledge, there is no human endoscopic image dataset specifically for neural network enhancement due to constraints involving privacy and difficulty of acquisition. Based on this consideration, we acquired images with a CMOS  image sensor dedicated to endoscopy and an endoscopy system developed by Hefei Deming Electronics Co. A total of 192 low and normal exposure images of various parts of the chicken, such as lungs, kidneys, and intestines, were captured by varying the exposure time and sensitivity (ISO) with other configurations fixed. In order to further constrain the decomposition ability of the decomposition network and prevent the drawbacks caused by lens dithering, we generate images with higher luminance by gamma-correcting the luminance channel (Y channel) of the Ycrcb color space of some normally illuminated images, where the gamma value is 0.5. The final dataset we used for training was 292 images, i.e., 146 pairs of images; part of the training set is shown in In this section, we compare the results of our proposed EIEN method with the results of six representative image enhancement methods, including MSRCR , AGCWD , LIME 11, Wang eThe quality of images captured by medical endoscopes seriously affects the accuracy of doctors’ diagnosis of early lesions inside the human body and the safety of operations in minimally invasive surgery, where natural color and uniform illumination are more conducive to human eye observation. Therefore, endoscopic image enhancement should adjust image brightness and contrast while preserving image naturalness and edge details, thus highlighting blood vessels, tissues, and lesions to assist physicians. As shown in MSRCR prevents color distortion in images by introducing a color recovery factor, but does not entirely prevent color distortion generated by multi-scale feature processing and fusion.AGCWD enhanced endoscopic images can appear over-enhanced in localized areas, and vascular details in dark areas can be blurred. We can clearly see through the two sets of LIME achieves good results in brightness correction, and dark areas in the image are visible. Still, the excessive tendency to brightness enhancement does not improve the contrast of endoscopic images well, resulting in tiny blood vessels and tissues becoming blurred, which is not conducive to the doctor’s diagnosis and operation in minimally invasive surgery.Wang et al. showed significant improvement in both luminance enhancement and contrast enhancement, and its shortcoming may be due to the lack of universality in the treatment of illuminance components. These cases are visible in the three sets of Retinex-Net has a significant effect on the correction of the illumination component. Still, it only denoises the reflection component, which leads to a severe decrease in the contrast of the endoscopic image and blurs the details of some blood vessels.Zero-DCE processes each of the three channels of the image separately using curves estimated by the depth network to be in the appropriate dynamic range, which often results in color distortion in endoscopic images. As shown in From the comparison in Objective metrics efficiently evaluate the quality merits of enhanced images and quantitatively discriminate between good and poor image enhancement methods. We chose three commonly used reference evaluation metrics: PSNR  , SSIM  is designed by combining the imaging characteristics of endoscopic images and the advantages of deep learning methods. The method consists of three parts: a decomposition network, illumination correction network, and reflection component enhancement algorithm. Inspired by Retinex theory, the image is decomposed into an illumination component and a reflection component by the decomposition network. Then, the illuminance correction is performed by a designed self-attention guided multiscale pyramid structure, while the reflection component is enhanced and the enhanced image is reconstructed. In this paper, the method’s performance is verified on different endoscopic images, and the necessity of the steps and components in the method is analyzed. Compared with other image enhancement algorithms, experimental results show that the technique can enhance tiny blood vessels and tissue clarity while maintaining image naturalness and solving uneven illumination and low contrast in medical endoscopic images more effectively. The method will significantly improve the accuracy of doctors’ diagnosis of diseases and the safety of minimally invasive surgical operations.The algorithm proposed in this study also has some shortcomings, and has large room for improvement. Future research will consider creating unsupervised network models using physical and generative models, and designing and introducing more appropriate image quality losses to guide network optimization.


Bearing is a key part of rotating machinery. Accurate prediction of bearing life can avoid serious failures. To address the current problem of low accuracy and poor predictability of bearing life prediction, a bearing life prediction method based on digital twins is proposed. Firstly, the vibration signals of rolling bearings are collected, and the time-domain and frequency-domain features of the actual data set are extracted to construct the feature matrix. Then unsupervised classification and feature selection are carried out by improving the self-organizing feature mapping method. Using sensitive features to construct a twin dataset framework and using the integrated learning CatBoost method to supplement the missing data sets, a complete digital twin dataset is formed. Secondly, important information is extracted through macro and micro attention mechanisms to achieve weight amplification. The life prediction of rolling bearing is realized by using fusion features. Finally, the proposed method is verified by experiments. The experimental results show that this method can predict the bearing life with a limited amount of measured data, which is superior to other prediction methods and can provide a new idea for the health prediction and management of mechanical components. Rolling bearing is a key component in rotating machinery, which has been widely used in modern industry ,2. AccurAmong the methods commonly used for bearing life prediction, there are often physical models and data-based methods. Physical models need to be built taking into account the influence of the complex surrounding environment and usually exhibit a weak generalization capability. Data-driven approaches, however, avoid the need for detailed modeling of complex environments and have better generalization capabilities by building models such as statistical extrapolation from historical data, and are one of the most popular research directions in the field of health prediction recently. Data-based methods often require manual extraction of features, construction of a health indicator (HI), determination of the health stage (HS), and determination of the first predicting time (FPT) before the final life expectancy can be predicted ,7,8.Machine learning, as a typical data-driven approach, constructs approximate models to approximate the real situation and build predictive models based on real-time, historical, and relational data. On this basis, Wang et al. proposed a residual service life prediction method for rolling bearings based on PCA and multi-dimensional feature fusion, aiming at the low reliability of bearing single feature characterization . The lifIn recent years, digital twin technology has developed rapidly and has been applied in a number of practical projects ,16,17. TThis paper combines digital twin with bearing life prediction and proposes a bearing residual life prediction method driven by macro and micro attention bi-directional long short-term memory (MMA-BiLSTM). Signal features are extracted from actual signals, a feature matrix is constructed, and feature selection is carried out by improving the self-organizing feature mapping method. The twin dataset framework is constructed by using sensitive features, and the missing data set is supplemented by the integrated learning CatBoost method to form a complete digital twin dataset. A new database is built to provide a qualitative analysis basis for the prediction of bearing residual life. The MMA-BiLSTM model is used for training to obtain the final residual life prediction results. The main contributions of this paper are as follows.(1) An improved self-organizing feature mapping method is proposed, which can achieve automatic extraction of sensitive features by calculating the corresponding probability density interval of feature values;(2) A twin data construction method is proposed to use sensitive data in the original data as a digital twin framework, and CatBoost is used to learn the remaining features and generate new digital twins;(3) An MMA-BiLSTM neural network is proposed to extract important information through macro and micro attention mechanisms to achieve weight amplification and improve the accuracy of remaining useful life prediction.Self-organizing mapping (SOM) is an unsupervised learning method that can be clustered and visualized in high dimensions. On the basis of unsupervised advantage, this method can also provide the change of feature weight in the connection layer after classification. By iteratively updating the feature weight in the network structure, you can easily observe the change in the feature weight, obtain the sensitive features of the dataset classification, and provide the classification basis. SOM converts the input data into discrete low-dimensional data, which is then represented as active points in local areas or networks. After the initialization step is completed, the following three important learning processes are competition, collaboration, and adaptation.In SOM, each neuron of the competition layer is connected according to the input N-dimensional feature vector The cooperative process is that only the optimal solution of the competitive process and its neighboring neurons learn from the provided input data. In order to form a map more sensitively for similar features in the competitive hierarchy, the “optimal” neuron determines the adjacent neurons according to a fixed function, and the corresponding weight of this neuron will be updated.The adaptive process refers to the adaptive activation function, which makes the optimal neuron and neighboring neurons more sensitive to specific input values, and also updates the corresponding weights. Through this process, the neurons adjacent to the optimal neuron will be more adaptive than those far away. The size of adaptation is controlled by the learning rate, which decreases with the learning time, and plays a role in reducing the convergence rate of SOM.The algorithm flow of SOM is as follows:(1) Initialization weight (2) Give a eigenvector (3) Calculate the distance (4) Update the connection weights (5) Input another feature vector into the network and return to step (3) until all the feature vectors are traversed;(6) Return to step (2) by making The CatBoost algorithm is a model based on the decision tree. It does not need a large number of samples as the training data and can adapt to the training of small-scale samples and high-precision diagnosis. The CatBoost algorithm belongs to the Boosting algorithm family and is a new machine learning algorithm framework based on a gradient boosting decision tree (GBDT). The GBDT algorithm is an algorithm for regression and classification proposed by Friedman in 2000 , which cIn CatBoost, the target statistics (TS) method is usually used to process categorical features target statistic (TS) method refers to replacing category features with calculated values. Representing the The commonly used TS value calculation method is the Greedy TS method, which can be smoothed by using the average of the target variable However, due to the duplication in the use of the training set and test set, this method will lead to condition deviation and result in overfitting. Based on this situation, CatBoost uses a method to improve the category feature processing and uses the sorting principle to solve the problem of condition shift and overfitting. In the Ordered TS, a random sequence For multi-dimensional feature data sets, the relationship between the actual value of most features and the prediction is often nonlinear, which brings great difficulties to the analysis of feature changes. To solve this problem, the feature intersection approach is proposed, which combines different features to form new cross-features to fit the changing relationship of data set features. If all features of the dataset are crossed, the exponential dimension will grow exponentially, thus increasing the computational complexity. Based on this problem, CatBoost adopts a greedy strategy to deal with it and does not cross features at the previous node of the gradient lifting tree. Instead, the features divided in front of the node and the features within the node are considered as two groups of features to cross, and one pot vector method is used for feature fusion. The acquisition method of cross-features is expressed by the formula:Where Where CatBoost uses a symmetric tree structure in the decision tree structure. The advantage of the symmetric tree structure is that it is not easy to overfit, and it is much faster than the gradient lifting algorithm, such as XGBoost. In addition, the CatBoost algorithm can realize multiple graphics processing unit (GPU) operations. The distributed learning tree enables CatBoost to perform parallel computing, thus improving its overall computing speed.LSTM network is an improvement in recurrent neural network (RNN). The key to the LSTM network is the cell state. The information in the cell state is updated and deleted through the forgetting gate, update gate, and output gate. The structure of LSTM is shown in Because the next moment prediction output of unidirectional LSTM is only affected by the previous multiple time inputs, in many cases, the prediction will be affected by the previous and subsequent multiple time inputs at the same time. In order to fully extract the correlation between before and after features and obtain better prediction results. The BiLSTM network was introduced to calculate the front and back information from two opposite directions  is used to select features, calculate the numerical probability density intervals of features corresponding to sensitive features, determine the optimal number of sensitive features, and form a feature framework. The feature framework is combined with existing data to form an interactive dataset with missing data, and CatBoost integrated learning algorithm is introduced. The missing eigenvalues are taken as the feature learning objectives of CatBoost, respectively, and their regression operation characteristics are used to complement the interactive dataset, thus forming a complete twin dataset. Finally, the macro and micro attention mechanisms are combined with BiLSTM to form MMA-BiLSTM. The weight of MMA-BiLSTM is amplified in the whole time dimension and each time dimension to realize the residual life prediction of bearings.The classical SOM method needs to preset the number of nodes in the output layer. Therefore, it is necessary to improve this method to make it have the ability to adaptively select the number of nodes in the output layer. In this paper, a generation-by-generation node processing method is proposed to remove nonsensitive features in the neighborhood during SOM’s selection of sensitive features. Both the number of output nodes of SOM can be adaptively obtained, and the nonsensitive feature removal strategy can also improve the feature selection efficiency of SOM. The proposed ISOFM method mainly selects nodes by introducing a learning rate The derivation of BiLSTM based on macro and micro attention mechanisms is as follows: macro and micro attention mechanisms refer to the operation of the attention mechanism on the whole time dimension of input data and data on each time dimension. Specifically, firstly, the data matrix generated by digital twins is processed, and its macro and micro attention coefficients are calculated using MMA. In the prediction process, the input dataset of the whole time dimension is imension .The formula for calculating macro and micro attention coefficients:Where Where According to the corresponding macro and micro attention coefficients, the associated input data weights and recursive data weights are magnified at multiple levels.Wherein, According to the amplification of input data weight and recursive number weight, the corresponding calculation results are obtained:Wherein, The bearing vibration data are collected separately as samples, and the data samples are twin expanded. Finally, different machine learning methods are used to compare the prediction accuracy between the original sample and the interactive dataset. The specific process and structure of the proposed method can be shown in (1) Set up a test platform to collect vibration signals of bearings from normal operation to fault status;(2) Extraction of time-domain and frequency-domain features of vibration signals from the original signal;(3) Use ISOFM to determine the number of sensitive features and select features from the acquired feature data set, and extract the main features in the feature set that can determine the signal category;(4) The probability density distribution models of sensitive features in feature data sets are constructed, respectively; determine the feature frame and the selection range of its feature values;(5) The feature data frame generated is combined with the feature data set extracted from the initial samples interactively, and the data at the nonsensitive features are represented by missing values;(6) The CatBoost regression algorithm is used to fill in the missing values in the interactive dataset containing missing values. Sorted according to importance, the missing value is used as the prediction target to fill the characteristic value. During the filling process, the missing values of other features are filled with the feature mean value;(7) An interactive dataset with a complete data structure is obtained, i.e., a twin feature dataset that expresses vibration signal fault information obtained from a small amount of data. The dataset is normalized to fit the health indicators of the bearing;(8) Use the first k health indicators of the bearing as network input to predict the health value at moment k + 1;(9) Repeat step 8 a certain number of times, and when these output values are less than 0, the inverse normalization of the sampled points results in RUL.The experimental design was carried out according to the research idea shown in Corresponding to the data sets in In constructing the digital twin data feature framework, the optimal number of sensitive features is first determined, and the feature framework is formed. The feature framework is combined with existing data to form an interactive dataset with missing data. The CatBoost integrated learning algorithm is introduced, and the missing feature values are used as the feature learning targets for CatBoost, respectively. The interaction dataset is complemented by using its regression operation properties to form a complete twin dataset. In this paper, the rolling bearing vibration signals are used as samples, the data samples are twinned and expanded, and finally, the prediction errors of the original samples are compared with the interaction dataset.After generating the digital twin interaction dataset, the initial data and the digital twin-generated data were used separately for bearing life prediction. The dataset was validated for prediction by an LSTM network, and the first 80% of the normalized sample points were used as the training set for predicting the RUL. The current sample points and true RULs for Dataset 1, Dataset 2, Dataset 3, and Dataset 4 are shown in It can be seen from To improve the prediction performance of LSTM networks. The macro-microscopic attention mechanism is combined with BiLSTM to propose MMA-BiLSTM. The method is compared with BiLSTM, LSTM, and GRU for experiments. The results are presented in From The prediction performance of the methods proposed in this paper has been improved, and the predicted values of RUL for bearings in different working conditions are closer to the actual values than other methods, which can provide an effective way to predict bearing life.For the bearing life prediction problem, this paper proposes a bearing life prediction method combining digital twin and MMA-BiLSTM network. Firstly, an extracted sensitive feature matrix is constructed to build the digital twin framework; the data set is supplemented by the integrated learning CatBoost method for missing data to form a complete digital twin data set. The MMA-BiLSTM network is proposed for life prediction. Finally, the accuracy of the proposed approach was verified by building a bearing life prediction test bench. The method can be further extended and applied to other condition parameters of gearboxes to provide data closer to the true value for predicting the RUL of machinery.


As an extension of optical coherence tomography (OCT), optical coherence tomographic angiography (OCTA) provides information on the blood flow status at the microlevel and is sensitive to changes in the fundus vessels. However, due to the distinct imaging mechanism of OCTA, existing models, which are primarily used for analyzing fundus images, do not work well on OCTA images. Effectively extracting and analyzing the information in OCTA images remains challenging. To this end, a deep learning framework that fuses multilevel information in OCTA images is proposed in this study. The effectiveness of the proposed model was demonstrated in the task of diabetic retinopathy (DR) classification.  First, a U-Net-based segmentation model was proposed to label the boundaries of large retinal vessels and the foveal avascular zone (FAZ) in OCTA images. Then, we designed an isolated concatenated block (ICB) structure to extract and fuse information from the original OCTA images and segmentation results at different fusion levels. Significance. The visualization results show that the FAZ and the vascular region close to the FAZ provide more information for the model than the farther surrounding area. Furthermore, this study demonstrates that a clinically sophisticated designed deep learning model is not only able to effectively assist in the diagnosis but also help to locate new indicators for certain illnesses. The experiments were conducted on 301 OCTA images. Of these images, 244 were labeled by ophthalmologists as normal images, and 57 were labeled as DR images. An accuracy of 93.1% and a mean intersection over union (mIOU) of 77.1% were achieved using the proposed large vessel and FAZ segmentation model. In the ablation experiment with 6-fold validation, the proposed deep learning framework that combines the proposed isolated and concatenated convolution process significantly improved the DR diagnosis accuracy. Moreover, inputting the merged images of the original OCTA images and segmentation results further improved the model performance. Finally, a DR diagnosis accuracy of 88.1% (95%CI ± 3.6%) and an area under the curve (AUC) of 0.92 were achieved using our proposed classification model, which significantly outperforms the state-of-the-art classification models. As a comparison, an accuracy of 83.7 (95%CI ± 1.5%) and AUC of 0.76 were obtained using EfficientNet.  Diabetic retinopathy (DR) is a microvascular impairment of the fundus caused by diabetes and is one of the leading causes of blindness and visual impairment . DR has Optical coherence tomography (OCT) is a new noninvasive imaging technique that can be used to effectively observe subtle changes in the superficial and deep capillary plexus of the human retinal microvasculature and has become popular in recent years . As an eCompared with traditional machine learning algorithms, deep learning shows higher performance in analyzing medical images –15. The In this paper, we proposed a deep learning framework that extracts and analyzes the multilevel information in OCTA images and demonstrated its advantage in DR diagnosis. We presented a segmentation model based on U-Net to segment the boundaries of vessels and the foveal avascular zone (FAZ) in OCTA images. Then, a new deep learning framework was proposed to predict the class of OCTA images based on fusing the original OCTA image and the segmentation results. A visualization method was used to indicate the regions of interest (ROIs) in the CNN model to locate the key lesion areas that are focused on by the prediction model to provide guidance for researchers on the key features for DR diagnosis.The OCTA-500 dataset compiled by Li et al.  was the The dataset contains two subsets with different fields of view (FOVs). As shown in To acquire the labeled image that indicates the vessels and FAZ, we proposed a segmentation method based on the U-Net  architecH denotes the height of the input image, W denotes the width, ym,n denotes the true label of the sample, and fm,n(x) denotes the output of the segmentation model.The loss function is designed as follows:The labeled images that indicate the vessels and FAZ can be used as expert opinions , which provide more effective information to the deep learning model. Therefore, the labeled images are input with the OCTA images as additional domain knowledge. These two types of images are single-channel images. Therefore, they can be concatenated by the channel dimension to obtain double-channel images. In other words, the labeled image and OCTA image are two independent channels of the merged image. The concatenation process is shown in The proposed deep learning framework for DR diagnosis in this study is designed based on the ResNet50  architecThen, the feature maps are processed by the following convolution process. More specifically, the convolution process is composed of four-stage residual convolutional blocks according to ResNet50. These blocks are made up of a convolutional block and several identity residual blocks. These block types are shown in Figures c denotes the current class, D denotes all classes of the whole dataset, tc denotes the true label of the sample, and fc(x) denotes the output of the classification model.The loss function is designed as follows:In this study, we used gradient-weighted class activation mapping (Grad-CAM)  to visuaPC is calculated by the following algorithm:Ml denotes the lth channel of the output feature map, fC denotes the output of the sigmoid function of the current class, N denotes the number of feature points in the feature map, σ denotes the activation function, and a rectified linear unit (ReLU) is used in this study.The class activation map A total of 301 images were used for the training and testing process in the cross-validation. Among these images, the ophthalmologists labeled 57 images as DR and 244 as normal (the ground truth).An accuracy of 93.1% and a mean intersection over union (mIOU) of 77.1% were achieved using our segmentation model. These values were calculated as the average of three classes. As shown in An ablation experiment was conducted to verify the performance of the proposed classification model. Three models were compared in the ablation test, namely, a model with only an isolated convolution process, with only a concatenated convolution process, and with both isolated and concatenated convolution processes. Obviously, the model using only the concatenated convolution process is equivalent to ResNet50, and the model using both isolated and concatenated convolution processes is the proposed model. Moreover, images with only segmentation results, only OCTA images, and merged images were taken as the input of the above three classification models. As shown in We compared our model with other existing models on the DR classification task for the same dataset with 6-fold cross-validation. In this comparison, the inputs for every model were merged images. As shown in The receiver operating characteristic (ROC) curves, which more comprehensively represent the performance of the classification model, of the above models are shown in The class activation maps of the last feature map were generated by Grad-CAM. The weight heatmaps were added to the merged images to indicate the ROIs for the classification model, as shown in We performed an experiment on another OCTA dataset, and the results are shown in The proposed deep learning framework with isolated and concatenated convolution processes significantly improved the accuracy of DR classificationFusing the information from the original OCTA image and labeled images that indicate the FAZ and vessel parts, which work as domain knowledge, provided more features and helped to lessen the DR classification errorsVisualization analysis confirmed that the FAZ and the vascular region around it contain more useful information, such as the shape of the FAZ and the density of vessels around it, than the surrounding areas to distinguish DR samples from normal samplesDeep learning can be used to analyze OCTA images by combining multilevel information and domain knowledge. The key discoveries in this study can be summarized as follows:The proposed analysis not only demonstrated the effectiveness of the deep learning algorithm and multilevel information fusion on DR diagnosis but also highlighted a potential indicator for DR in OCTA images. Hence, it was found that images with a larger FAZ area or a smaller density of vessels around the FAZ may be highly associated with the risk of DR in fundus screening. In the future, a study will be conducted on larger multicenter datasets, and the potential of the proposed deep learning framework in other related biomedical image analysis applications will also be explored.


According to the research status of Software Defined Network (SDN) control layer traffic scheduling, we find the current common problems, including single path, easy congestion, Quality of Service (QoS) requirements and high delay. To solve these four problems, we design and implement a QoS-oriented global multi-path traffic scheduling algorithm for SDN, referred to as QOGMP. First, we propose a link weight calculation algorithm based on the idea of traction links and deep reinforcement learning, and conduct experimental verifications related to traction links. The algorithm considers QoS requirements and alleviates the problems of easy congestion and high delay. Then, we propose a traffic scheduling algorithm based on link weight and multi-path scheme, which also considers QoS requirements and solves the problem of single path. Finally, we combined the link weight calculation algorithm and the traffic scheduling algorithm to implement QOGMP, and carried out comparative experiments in the built simulation environment. The experimental results show that QOGMP is better than the two comparison algorithms in terms of delay and rescheduling rate. Traffic scheduling is the process of assigning concurrent request packets of massive users to server program instances of different IP addresses according to a specific strategy.With the continuous expansion of network scale and continuous updating of network technology, high availability and high performance requirements are put forward to the network. In order to maintain network availability and improve network performance, it is necessary to effectively allocate network resources and reasonably schedule network traffic2. Under the new situation of rapid network development and continuous emergence of network applications, traffic scheduling based on the traditional network has limitations: (1) It challenges the efficiency of traffic scheduling. The development of cloud computing has made the demand for large-scale data centers more and more obvious. In the process of network traffic scheduling, network resources should be used more effectively to reduce costs and improve the overall performance of the network3. However, the traditional network traffic scheduling technology has problems such as low performance and high overhead when facing large-scale networks. (2) The network is prone to congestion. In traditional network traffic scheduling, only local information can be used. The network node calculates the path to the destination node based on itself. It does not have a global network view, so it is difficult to perform good traffic scheduling4, which makes the probability of congestion and other problems in the network continue to increase. (3) Traffic scheduling is not flexible. In the face of rapid network changes and user demand, it is very weak. After traffic scheduling according to network application requirements, if the demand changes, the configuration of the corresponding network equipment  needs to be revised in the traditional network5, so as to re-schedule the traffic, which is a very tedious process.With the continuous increase in the number and types of devices connected to the network, in the face of the rapidly changing business environment of the Internet, the drawbacks of the traditional network have gradually become apparent6, breaking the closed mode of integration of software and hardware of traditional network equipment, and separating the control level of network equipment from the data forwarding level.The birth of Software Defined Network (SDN) can solve these limitations. In 2009, Professor Mckeown formally proposed SDN, a new network architecture in the literature8, thereby effectively solving the disadvantages of the traditional IP network in maintenance, expansion, and experimental innovation. The typical architecture of SDN is divided into three layers9. The top layer is the application layer, including various services and applications. The middle layer is the control layer, which is mainly responsible for processing the data resource arrangement and maintaining information such as network topology and status. The main body of the control layer is a logically centralized and programmable controller that can master global network information, which is convenient for operators and scientific researchers to manage and configure the network and deploy new protocols. The bottom layer is the data layer, which is responsible for data processing, forwarding, and status collection based on the flow table. The main body of the data layer is a lot of dumb switches . These switches only provide simple data forwarding functions and can quickly process matched data packets to meet the increasing demand of traffic. An open unified interface (such as OpenFlow10) is used to interact between the control layer and the data layer. The controller issues unified standard rules to the switch through the standard interface, and the switch only needs to perform corresponding actions in accordance with standard rules.The design concept of SDN is to separate the control layer and data layer of the network while realizing programmable control, which can provide centralized management and dynamic maintenance capabilities for distributed networks11, the control layer can use the global network view and dynamic rule configuration capabilities provided by SDN to perform load balancing and flexible traffic scheduling12. This solves the limitations of traffic scheduling based on the traditional network to a large extent, thereby maintaining network availability and improving network performance. Therefore, it is of great significance to carry out research on SDN-based control layer traffic scheduling methods.Different from the “slice” management of the traditional network17 are all multi-path transmission. For example, literature13 proposed an equal-cost multi-path (ECMP) scheme, which is currently widely used. However, this type of solution has two major problems: one is that the multi-path transmission scheme implemented under specific conditions lacks versatility in the SDN environment; the other is that traffic scheduling can only use local information, which is likely to cause congestion problems.Single path. The means to solve the single-path problem in literature23 include processing elephant flows and using SDN global link load information. These methods introduce additional overheads such as query detection and congestion calculation, which increase the forwarding delay to a certain extent.Easy to congest. The traffic in the network has shown explosive growth. The traditional network architecture cannot achieve flexible, fast and effective scheduling of network traffic. In addition, it is difficult to know the load status of the path. Congestion problems are prone to occur, resulting in low link utilization. The means to solve the problem of easy congestion in literature24, while ensuring the efficient operation of the network. At present, SDN can provide QoS guarantee through mechanisms such as flow control and bandwidth reservation, but it is difficult to meet the increasing demand for QoS of business applications25. A real QoS-oriented traffic scheduling scheme is needed. The main means to solve the QoS problem in literature29 is to introduce user-defined constraints and comprehensively consider the link occupancy rate and the size of the business flow. These methods also introduce additional storage overhead such as packet loss rate measurement and link occupancy calculation and the delay caused by this. There is no widely used standard for user-defined constraints, and even the SDN northbound interface (between the application layer and the control layer) as the basis for its realization has not yet a unified or recognized standard. Although its research space is relatively large, its current practicability and research value are not great.QoS requirements. When the network is overloaded or congested, QoS can ensure that important services are not affected by delay or packet loss during the transmission process30 proposed an SDN global multi-path traffic scheduling algorithm based on reinforcement learning. The algorithm uses the link bandwidth information provided by the SDN data layer to update the link weights, and selects k shortest paths as the forwarding paths according to the weights. Such methods based on reinforcement learning are difficult to classify complex traffic characteristics. Literature31 proposed an SDN traffic scheduling algorithm based on the deep neural network. Such methods are based on deep learning. Deep learning can make up for the shortcomings of reinforcement learning but requires a large amount of labeled data to train neural networks.High latency. The solutions to the above three problems will introduce considerable delay pressure. In addition, in the face of large-scale networks, the current routing algorithms are obviously weak. It is necessary to consider an efficient forwarding path calculation algorithm to deal with it. Machine learning algorithms can usually extract traffic characteristics automatically, and do not rely on expert experience, so they are more efficient than traditional solutions in solving traffic scheduling problems. LiteratureIn recent years, many scholars have devoted themselves to the study of SDN-based control-layer traffic scheduling methods. These researches are dedicated to solving different problems, including four problems such as single path, easy congestion, QoS requirements, and high latency. Based on the above research status, in view of the four existing problems in the SDN control layer traffic scheduling research, we consider the following solutions: (1) Using the multi-path scheduling method of the ECMP scheme to solve the single path problem. (2) Using the global information provided by the centralized controller of SDN to alleviate the congestion problem. (3) Calculating traffic forwarding rules by integrating multiple link parameters  and the size of business traffic to provide QoS guarantee. (4) Using deep reinforcement learning algorithms to alleviate the high latency problem. The forwarding path calculation problem is suitable for deep reinforcement learning, and deep reinforcement learning algorithms can overcome the deficiencies of reinforcement learning and deep learning.32. In the traditional single-path transmission mode, all QoS-related network parameters can be used for link weight calculation and the optimal path can be selected. However, when we adopt multi-path transmission, we need to balance multiple paths, considering the efficiency of the path selection scheme and the success rate of scheduling. Therefore, in order to ensure the efficiency of the path selection scheme, we combine the deep reinforcement learning algorithm to use the packet loss and delay parameters for link weight calculation. In order to improve the scheduling success rate of the scheme, we use the link capacity parameter to calculate the traffic forwarding path.QoS refers to the service capability that a network can provide for network communication tasks. For different communication tasks, QoS needs to achieve different indicators, including packet loss tolerance, delay tolerance, link capacity and other metricsThe main research content is as shown in Fig. We design a link weight calculation algorithm based on the idea of traction link and deep reinforcement learning.P defaults to 1, the state of the environment x. The quality of the strategy can be measured by the value function.We use deep reinforcement learning algorithms to calculate link weights. The agent of the reinforcement learning system is set as a neural network (strategy generation network), and its interaction with the environment is modeled as a Markov process. The Markov process is represented by a four-tuple The strategy generation network is implemented by the neural network The value function of the strategy is expressed as Eq. .1\documeThe reward function of the strategy is expressed as Eq. .2\documeThe parameters of the strategy value network and the strategy generation network need to be continuously adjusted based on the error of value and reward.The parameter update of the strategic value network is in accordance with Eq. .3\documeThe parameter update of the strategy generation network is in accordance with Eq. .4\documeThe input of the strategy generation network is the traffic view and reward, and the output is the link weight value we need. The traffic view is a summary of the link information collected and calculated by the data layer, including information such as nodes, links, and the cost of each link. Since different QoS requirements have different requirements for packet loss and delay, we express the link cost as Eq. .5\documeIn Eq. , \documeThe flow of link weight calculation algorithm based on traction link and deep reinforcement learning is shown in Algoritnm 1.n, the maximum number of optional links increases exponentially, as shown in Eq. . The second is to simply allocate according to the margin ratio. The definition of the margin ratio is the capacity of each path to be allocated divided by the total capacity of all paths to be allocated. The capacity of each path is the minimum value of the capacity of all links on the path.We update the link weight according to the output of Algorithm 1. If the weight is not updated, the default is 1. After calculating the Pycharm can not only run python algorithms, but also create graphical interfaces. We use the editor pycharm-community-2019.1.1 to implement the algorithm and create a simulation environment.We combine the algorithms in section two and section three to implement the complete flow scheduling algorithm QOGMP, which is implemented in the order of Algorithm 2, Algorithm 1, and Algorithm 3. We perform a simplified simulation of the SDN network system.The simplified network system is divided into two layers: the control layer and the data layer. The controller of the control layer has the function of receiving information from the data layer and the calculation function of the traffic scheduling scheme. The data layer has the function of data collection and flow forwarding. Data transmission is allowed between the two layers.V, link E and link parameters . (2) The function of the controller to receive information from the data layer: The controller obtains the traffic view G fed back from the data layer. (3) The function of the controller to calculate the flow scheduling scheme: We embed the traffic scheduling algorithm into the controller as the main algorithm of the controller. We take the traffic views G1 , G2 and user requirements (including business flow and delay tolerance) as input. After the main algorithm is executed, the traffic scheduling scheme is output. (4) Flow forwarding function of the data layer: The data layer receives the scheme generated by the controller and forwards the flow according to it .In the order of execution, the specific functional design ideas are as follows: (1) Data collection function of the data layer: The data layer collects network information, including switch We conduct related experiments to verify the effectiveness of traction link. After implementing Algorithm 2, we record the amount of calculation saved after applying the traction link algorithm to verify whether the extraction of the traction link can greatly reduce the number of links.n nodes in V, the input format of the link graph  is a i and node j, i and node j. The data used in the experiment is randomly generated. We record the number of links in the input and output link graphs (the links between the same nodes are not recorded repeatedly), and the results are shown in Table We conduct experiments on 20 link graphs It can be obtained from Table Therefore, the use of the traction link algorithm can greatly reduce the number of links that need to be processed without affecting the subsequent path selection results. It can save a large amount of calculation and improve the efficiency of the link weight calculation algorithm.29. The machine learning algorithm with the smallest delay is the SDN global multi-path traffic scheduling algorithm based on reinforcement learning proposed in literature30.At present, the traditional method with the best performance (delay and rescheduling rate) is the QoS-oriented SDN global traffic scheduling algorithm proposed in literature29, QOGMP algorithm considers multi-path scheduling and has a higher link utilization rate. It uses machine learning algorithms to speed up the calculation of weights, which is suitable for big data environments. In contrast, QOGMP algorithm has obvious advantages, so it is no longer verified by comparison experiments.Compared with the algorithm proposed in literature30 (hereinafter referred to as GMPRL).We evaluate the performance of QOGMP on the built simulation system. Indicators for performance evaluation include delay and rescheduling rate. We compare QOGMP with the traffic scheduling algorithm that does not use traction links  and the algorithm proposed literatureDelay here refers to the algorithm running time. We carry out comparative experiments on the three algorithms. In order to reduce the experimental error, each experiment needs to be measured multiple times to record the shortest running time.For the delay indicator, the comparative experiment is completed on 20 different traffic views, and the experimental result is shown in Table Plot Table It can be seen from the experimental results that QOGMP is better than pre-QOGMP and is almost consistent with GMPRL in terms of delay indicator. (1) Since both QOGMP and GMPRL use machine learning algorithms, the delay of QOGMP is not much different from that of GMPRL; (2) QOGMP increases the step of pulling link extraction which leads to the increase of running time. But at the same time, the introduction of traction link greatly saves the amount of computation. So QOGMP outperforms pre-QOGMP in delay.Since the three algorithms are all multi-path algorithms, the criteria used for rescheduling judgment are the same, that is, the sum of all path traffic in the solution generated by the algorithm is less than the service traffic or the transmission delay of a single path exceeds the delay tolerance.For the indicator of rescheduling rate, we conduct 20 groups of comparative experiments, each of which was completed on 20–100 different data sets. The experimental result is shown in Table Plot Table It can be seen from the experimental results that QOGMP is better than GMPRL and is almost consistent with pre-QOGMP in terms of rescheduling rate indicator. (1) GMPRL does not consider QoS requirements, resulting in an increased probability that the traffic scheduling scheme is not suitable for service traffic, so QOGMP outperforms GMPRL in re-scheduling rate; (2) Since the introduction of traction links will not have a great impact on the final scheme, so QOGMP is almost identical to pre-QOGMP in rescheduling rate.To sum up, compared with pre-QOGMP, QOGMP has lower delay and almost the same rescheduling rate; compared with GMPRL, QOGMP has lower rescheduling rate and almost the same delay. Therefore, our proposed QOGMP is better than GMPRL and pre-QOGMP for delay and rescheduling rate indicators.We design and implement a QoS-oriented SDN global multi-path traffic scheduling algorithm, referred to as QOGMP. Aiming at the four problems currently existing in the research of SDN control layer traffic scheduling, QOGMP has adopted solutions: (1) QoS problem is solved by using the three QoS-related network parameters of packet loss, delay, and link capacity to generate traffic scheduling scheme; (2) The easy congestion problem is alleviated by utilizing the controller global view; (3) The high latency problem is alleviated by introducing traction links and deep reinforcement learning; (4) The single path problem is solved by multi-path scheduling. We carry out comparative experiments in the built simulation environment. The experimental results show that QOGMP has better performance than the two comparison algorithms in terms of delay and rescheduling rate.We only considered the three common QoS parameters of delay, packet loss and link capacity. However, the complex network environment also contains other QoS parameters such as jitter. We can conduct in-depth research on QoS parameters and improve the algorithm proposed in this article in order to further cope with the complex network environment and business requirements.34.As the neural network is not the innovative point and focus of the QOGMP algorithm, the neural network used in this article is the most basic. It can be replaced with an improved neural network. For example, in order to improve the update stability of neural network, we can use the target network method proposed in literature35.The delay in the comparative experiment is up to 276 s, which is caused by hardware limitations. Later, the delay can be shortened to meet practical application requirements through Brax accelerator hardwareHowever, we still have room for improvement in terms of application scenario expansion, application algorithm improvement, and system details reproduction. The details are as follows.