In view of the limitations of existing rotating machine fault diagnosis methods in single-scale signal analysis, a fault diagnosis method based on multi-scale permutation entropy (MPE) and multi-channel fusion convolutional neural networks (MCFCNN) is proposed. First, MPE quantitatively analyzes the vibration signals of rotating machine at different scales, and obtains permutation entropy (PE) to construct feature vector sets. Then, considering the structure and spatial information between different sensor measurement points, MCFCNN constructs multiple channels in the input layer according to the number of sensors, and each channel corresponds to the MPE feature sets of different monitored points. MCFCNN uses convolutional kernels to learn the features of each channel in an unsupervised way, and fuses the features of each channel into a new feature map. At last, multi-layer perceptron is applied to fuse multi-channel features and identify faults. Through the health monitoring experiment of planetary gearbox and rolling bearing, and compared with single channel convolutional neural networks (CNN) and existing CNN based fusion methods, the proposed method based on MPE and MCFCNN model can diagnose faults with high accuracy, stability, and speed. Rotating machinery is widely used in many fields of heavy industry and plays an important role in modern industrial production . Gears aVibration signal analysis is the most commonly used diagnostic method in mechanical fault diagnosis at present . In receIn order to solve the above drawbacks, convolutional neural networks (CNN) may provide effective solutions for intelligent information fusion and fault diagnosis. CNN is a model of deep learning, which uses end-to-end processing technology to achieve feature extraction and fault classification . In receMulti-channel CNN is widely used in the fields of image processing and speech recognition. In these fields, the input can be naturally divided into different channels, such as the color channel of the image and the wavelength of the speech . Liu et Nevertheless, the structure of mechanical equipment is very complex, and the interaction and coupling effect between the components make the vibration signal contain many inherent oscillation modes on different time scales . TraditiIn MCFCNN, multiple independent channels are established in the input layer, and each channel corresponds to the MPE feature set of each measuring point sensor. Each convolutional kernel further learns the deep features of each channel separately, and fuses the learned feature maps of each channel into a new feature map . MCFCNN The remainder of this paper is organized as follows. In Aziz et al.  proposedStep 1: Constructing the coarse-grained time series. Given a time series Step 2: Calculating PE value for coarse-grained time series Convolutional neural networks (CNN) is based on multi-layer supervised learning and it requires a large amount of labeled data to complete model training by repeating forward propagation and backward propagation. CNN is usually composed of feature extractor and multilayer perceptron (MLP). The feature extractor consists of multiple alternating pooling and convolutional layers. The convolutional layer is to extract the features from input data, whereas the pooling layer is to decrease the feature dimension . The fulm and n, respectively, represent the sizes of the pixels in the height direction and the width direction of the input image; l convolutional layer; l convolutional layer; the l layer; l convolutional layer.The convolutional layer is responsible for feature extraction through a convolution kernel, which is essentially a weight matrix. The convolutional layer contains multiple convolution kernels with different weights. The convolutional layer slides different convolution kernels on the input data by sliding the window. The convolutional kernel and the corresponding area of the sliding window are convoluted to obtain local perception and the matrix obtained by the convolution operation after sliding is called feature mapping. The convolution operation between input neuron and convolution kernels is:k input feature map in the l pooling layer; k output feature map at l pooling layer. In the pooling layer, the numbers of input and output feature maps are the same, but the size of output feature maps is usually reduced to by 50% compared to that of input feature maps. The pooling operations generally includes mean pooling operation and maximum pooling operation. The mean pooling operation is to calculate the average value of each subregion, whereas the max pooling operation is to calculate the max value of each subregion as the output feature.The pooling layer is also called the sub-sampling layer because it involves the division of the input feature map into many small areas of the same length (greater than 1), the calculation of a value for each area, the acquisition of the local optimal value, the arrangement of the calculated values, and the output of a new feature map . The purAfter the multistage convolutional layer and pooling layer, several fully connected layers are connected. The fully connected layer is used to integrate features from multiple feature maps and map them into a fixed length feature vector. At last, the fully connected layer and the output layer are connected by the classifier, and the learned features are mapped into the sample label space to complete the classification task. Similar to the training process of traditional neural networks, CNN uses back propagation (BP) algorithm to fine tune network parameters.The proposed overall MCFCNN framework is shown in Step 1: Condition monitoring at different locations of mechanical equipment through multiple sensors. Assuming that there are Step 2: For each group of vibration signals, sample division is performed, that is, discrete points of vibration signals of length l layer.Step 3: The training sample MPE feature of the k feature mapping of the In image classification, the experimental effect of max pooling is better than that of average pooling ,37. Max Step 4: MCFCNN is trained with the training set until the loss function converges to a certain degree or the number of iterations is satisfied.Step 5: The trained MCFCNN is validated with the testing set, and the average testing accuracy and standard deviation are calculated.In order to verify the effectiveness of the proposed method, which was compared with single channel CNN and existing CNN based fusion methods in two fault diagnosis experiments of planetary gearbox and bearing data of Case Western Reserve University (CWRU) . Where sThe planetary gearbox test rig is composed of operating console, a variable speed drive motor, flexible coupling, helical gearbox, planetary gearbox, a magnetic powder brake, and isolation floor. The motor is connected with the helical gearbox through the flexible coupling . The plaMost of the previous studies focused on fault diagnosis of fixed-axis transmission or relative stationary components such as sun gear, inner gear rings and planetary carriers , but conIn the experiment, each sample contains 1024 data points, and the faulty planetary gear has no mesh within the sample length. Therefore, two 3-axis accelerometers and three 1-axis accelerometers were used to monitor vibration signals in nine directions, and the total number of samples for each state was 5400. Raw vibration signals of the planet gear with atw monitored in nine directions under a load of 0.5 hp are shown in The number of channels in the input layer of MCFCNN model can be set according to the number of sensors used in the experiment. In the planetary gearbox experiment, five groups of sensors were used to monitor vibration signals in nine directions, so the number of channels in the input layer is 9. There were 10 convolutional kernels and 20 convolutional kernels in the two convolutional layers. The convolutional kernels of two convolution layers were set to be 1 × 3. The moving step was set to be 1 and the ratio of max pooling was set to be 1 × 2. At the fully connected layer, the number of neurons was set to be 200 and Softmax regression was adopted as a classifier. Details of the parameters of the MCFCNN model are given in To verify MCFCNN in gearbox fault diagnosis, the proposed method was compared with single-channel CNN. The two methods were tested through ten trials with three datasets under three loads, respectively. The diagnostic accuracy of each experiment is shown in To further assess the classification performance of MCFCNN, the diagnostic reliability of two diagnostic methods for each fault state of planetary gear was obtained with the confusion matrix. The confusion matrices of five fault states of MCFCNN and single-channel CNN are shown in To verify the feature learning ability of the method, the features in the fully connected layers of MCFCNN and single-channel CNN were visualized by t-distributed stochastic neighbor embedding (t-SNE). When MCFCNN and single-channel CNN were used to diagnose the testing datasets A, B, and C, the features obtained by the fully connected layer were reduced to 2D by the T-SNE technique to observe the classification effect. As shown in In terms of the feature distribution of the fully connected layer, MCFCNN had the good feature separation capability and classification performance. Multi-channel data fusion and feature fusion could be well clustered into categories, and not adversely affected by bad signals. It can be judged that MCFCNN with multi-sensor data as the input can effectively delete redundant information in the process of feature learning and improve the classification accuracy. Single-channel CNN showed the good feature separation capability under testing datasets A and C, but the features of two tooth worn state and three tooth worn state were mixed together. In the testing dataset B, except that the feature of all tooth worn state was clustered together, the features of three tooth worn state and two tooth worn state overlapped and the features of normal state and single teeth worn state overlapped, it was difficult to differentiate the fault of two tooth worn from the fault of three tooth wear or the normal state from the fault of single teeth worn. In addition, fault size or other properties such as load may lead to the feature overlapping.In order to further prove the stability and superiority of MCFCNN, it is compared with other fusion methods ,14,16,17Under three datasets, MCFCNN had the better classification performance than other methods . Our forIn practical engineering application, the load of rotating machinery always changes, and the fault diagnosis model needs to adapt to different load conditions. In order to verify the load adaptability of the proposed method, the training datasets A, B, and C are used to train the model, and the other two testing datasets different from the training set are used to test the trained model. For example, the model is trained on training dataset A, and tested on testing dataset B, which is represented by symbol “A→ B”. Therefore, there are six combinations between the training set and the test set, each training dataset contains 480 samples and the test dataset contains 120 samples. It can be observed from Compared with It is difficult to verify the performance of the proposed method with a single dataset, so the proposed method is also tested on CWRU’s rolling bearing fault dataset. CWRU dataset is the world recognized standard dataset for bearing fault diagnosis. In the field of fault diagnosis based on deep learning, the test data of the two most cited papers ,44 are aAs shown in Due to the different number of sensors, this paper studies 6 classes of faults with different severity on the rolling element and outer raceway of the drive end rolling bearing when the motor load is 0, 2, 3 horsepower (hp). Three groups of sensors were used to monitor the vibration signals in three directions in the rolling bearing experiment of CWRU. Therefore, the number of channels in the input layer of MCFCNN model is 3, and other parameters are the same as those in Experiment 1.MCFCNN model and single channel CNN model are used for rolling bearing fault diagnosis. MCFCNN model simultaneously takes the MPE value of DE, FE, and SBP sensor data as input, and single channel CNN takes PE value of single sensor as input. The diagnostic accuracy of the two models in ten experiments on three datasets under three loads are shown in The model used for comparison is the same as the planetary gearbox fault diagnosis experiment, and the experimental results are shown in The load adaptability of the proposed method is verified on the rolling bearing data. During the experiment, the models trained on different training datasets A, B, and C are tested on different testing datasets. Each training dataset and testing dataset contains 94 and 24 samples, It can be seen from the results that under different loads, the proposed method achieves higher accuracy than other methods, but compared with In this paper, a fault diagnosis method based on MCFCNN and MPE is proposed for the fault diagnosis of rotating machinery. Firstly, the permutation entropy across 25 scales is extracted from the vibration signal to represent the fault information of the signals on multiple timescales. The extracted MPE features are then input to MCFCNN for information fusion and fault identification. The novelty of MCFCNN is that several channels are built for the input layer, which correspond to different sensors. Sharing a CNN structure between channels. MCFCNN realizes end-to-end multi-source information fusion and classification without human intervention. The proposed method was verified by the experimental data of the planetary gearbox and rolling bearing. Experimental results show that the proposed method can effectively identify faults with different severity of gearbox and rolling bearing. Through comparative research with single-channel CNN and other CNN-based fusion methods, the proposed method has obvious superiority in learning ability, loss function convergence speed, training time, fault recognition, and load adaptability.In the next work, we will test the proposed method under more conditions. Furthermore, in terms of load adaptation, there is still the possibility of improving the fault identification rate, so we will consider using different types of sensors to fuse and optimize the network structure, so as to achieve better fault diagnosis performance.


Due to the existence of marine environmental noise, coupled with the instability of underwater acoustic channel, ship-radiated noise (SRN) signals detected by sensors tend to suffer noise pollution as well as distortion caused by the transmission medium, making the denoising of the raw detected signals the new focus in the field of underwater acoustic target recognition. In view of this, this paper presents a novel hybrid feature extraction scheme integrating improved variational mode decomposition (IVMD), normalized maximal information coefficient (norMIC) and permutation entropy (PE) for SRN signals. Firstly, the IVMD method is employed to decompose the SRN signals into a number of finite intrinsic mode functions (IMFs). The noise IMFs are then filtered out by a denoising method before PE extraction. Next, the MIC between each retained IMF and the raw SRN signal and PE of retained IMFs are calculated, respectively. After this, the norMICs are used to weigh the PE values of the retained IMFs and the sum of the weighted PE results is regarded as the classification parameter. Finally, the feature vectors are fed into the particle swarm optimization-based support vector machine multi-class classifier (PSO-SVM) to identify different types of SRN samples. The experimental results have indicated that the classification accuracy of the proposed method is as high as 99.1667%, which is much higher than that of other currently existing methods. Hence, the method proposed in this paper is more suitable for feature extraction of SRN signals in practical application. Signal analysis of ship-radiated noise (SRN) is important, not only for military operations but also for environmental protection ,2. FeatuPrevious studies have already demonstrated the non-linear properties of SRN ,12. As oDespite the importance of entropy methods, they suffer in the presence of marine environmental noise. One way to suppress the marine environmental noise is through signal decomposition algorithms. A large number of signal decomposition methods have emerged to further improve the performance of feature extraction, such as empirical mode decomposition (EMD) , ensemblA denoising processing always includes removal of noise IMFs, reconstruction of signal-dominant IMFs, or selection of one or more sensitive IMFs. In recent years, considerable denoising techniques combining signal decomposition and entropy methods have been continuously emerging with satisfactory results. Study in , a groupK is set to 5–15, and the quadratic penalty term K and On this basis, this paper proposes a new feature extraction method for SRN signals using improved VMD (IVMD), normalized maximal information coefficient (norMIC), and PE. For the calculation of parameters in IVMD method, the range of mode number IVMD overcomes the blindness of parameter selection in VMD such that it is employed to process SRN signals in this paper. As a pointer to correlation, MIC is first introduced to measure underwater acoustic signals. RWPE is a newly proposed uncertainty metric for time series, and it is also utilized for the first time to denoising of noisy signals. The scheme proposed in this paper comprehensively covers the steps of underwater acoustic target recognition, including signal decomposition, noise reduction, feature extraction and classification, which can serve as a supplement to the field of underwater acoustic signal processing, and also provide a reference for the basis of further research.The structure of the paper is organized as follows: kPh(t) and instantaneous amplitude kAm (t) given as:As an adaptive signal decomposition algorithm, variational mode decomposition (VMD)  can decos(t), partial derivative function kI(t) and its center frequency kf. The optimization problem can be expressed as follows:In the VMD algorithm, given the target signal ɛ is the updated parameter. The VMD stop condition is given by:a represents the convergence accuracy.The constraint problem in (2) can be converted into a new form by using Lagrange multipliers: Since permutation entropy (PE)  only comm and the time delay Given the embedding dimension iX is rearranged in an increasing order as:iX are equal, like:Vector m! symbol sequences can be calculated in total, where S(g) is one symbol sequences in phase space, In this case, a symbol sequences can be obtained as:The probability distribution of all the symbol sequences is calculated as pH means higher complexity of the time series; a smaller pH means lower uncertainty of the time series.The PE value ranges from 0 to 1. A larger m and time delay iX can be denoted as:The main defect of PE lies in the neglect of amplitude with only consideration of neighboring values. The weighted permutation entropy (WPE) proposed in  has addrThe weighted relative frequency can be calculated as:RWPE measures the distance between the observed signal and the Gaussian white noise, in which case the RWPE is defined as follows:Maximal information coefficient (MIC)  is a gooStep 1: The set of data points with two attributes is distributed in two-dimensional space. Step 2: Calculate the MI of random variables X and Y. As there are more than one way to divide the data points by a grid of k denotes the number of retained IMFs, K denotes the mode number of IVMD, and S is the raw SRN signal.The noise IMFs by IVMD will be removed after a denoising processing. In order to weigh the PE values of the retained IMFs, the normalized MIC (norMIC) corresponding to each retained IMF is calculated as:This paper proposes a hybrid feature extraction method for SRN signals integrating IVMD, norMIC and PE. The flow chart of the proposed method is presented in K is set to 5–15, and the quadratic penalty term Step 1: The mode number Step 2: The combination of Step 3: Decompose SRN signals using IVMD and a set of IMFs can be extracted.Step 4: Extract the RWPE for each IMF and the sensitive IMFs are retained  between IMFs and simulation signals for EMD, EEMD, and IVMD are calculated. The results are listed in Combined with the analysis in Some components are noise and they should be excluded before feature extraction. Therefore, the simulation signals in 9. Intuitively, the denoising method is feasible to suppress the in-band noise hidden in the raw SRN signals. To further validate the effectiveness of the method, the IMFs with RWPE values greater than 0.1 are selected to reconstruct new signals. We can easily observe from In order to explore the effect of data length on PE, the standard Gaussian white noise and colored noise series with length of 5000 are constructed, respectively. Then we calculate the PE, AE, and SE of the noise with a step size of 100. In this way, each entropy has a total of 50 values. The calculation results of entropy are presented in K and quadratic penalty term K and In this paper, three types of SRN signals are randomly selected from a real measured data set in , namely,To demonstrate the necessity of the denoising method, fist, the selected IMFs in As seen from After this, 60 randomly selected samples from each class are used to train the PSO-SVM multi-class classifier, and the remaining 40 ones are used as the test data. The MATLAB codes of PSO-SVM multi-class classifier can be downloaded from . The claIn order to extract the inherent features that can characterize ship-radiated noise, a technique fully integrating IVMD, norMIC and PE is presented in this paper. The IVMD addressed the mode number, as well as the quadratic penalty term issue for VMD. Thus, it is employed to decompose the SRN signals for the first time. As a kind of correlation metric, MIC is more accurate than MI in measuring the correlation between paired variables. PE enjoys the merits of being conceptually simple and computationally fast, and has better stability than AE and SE. The proposed technique inherits the advantages of IVMD, MIC coupled with PE in a proper manner. In stark contrast to other methods, the proposed technique employs RWPE to screen sensitive IMFs, thereby paving the way for the subsequent PE extraction. Furthermore, the norMICs are used to weigh the PE results according to the degree of correlation between the sensitive IMFs and the raw SRN signal. The excellent performance of the proposed method has been reasonably verified by the classification experiment of ships. The experimental results reveal that the proposed method has a 99.1667% classification accuracy, which is much higher than other schemes. On this basis, the proposed technique is more reliable in terms of feature extraction and classification in practice. In future work, we will try to explore other parameter selection methods for VMD and denoising algorithms to further improve the performance of the system.


Image quality assessment (IQA) is a fundamental technology for image applications that can help correct low-quality images during the capture process. The ability to expand distorted images and create human visual system (HVS)-aware labels for training is the key to performing IQA tasks using deep neural networks (DNNs), and image quality is highly sensitive to changes in entropy. Therefore, a new data expansion method based on entropy and guided by saliency and distortion is proposed in this paper. We introduce saliency into a large-scale expansion strategy for the first time. We regionally add distortion to a set of original images to obtain a distorted image database and label the distorted images using entropy. The careful design of the distorted images and the entropy-based labels fully reflects the influences of both saliency and distortion on quality. The expanded database plays an important role in the application of a DNN for IQA. Experimental results on IQA databases demonstrate the effectiveness of the expansion method, and the network’s prediction effect on the IQA databases is found to be improved compared with its predecessor algorithm. Therefore, we conclude that a data expansion approach that fully reflects HVS-aware quality factors is beneficial for IQA. This study presents a novel method for incorporating saliency into IQA, namely, representing it as regional distortion. With the current state of development of multimedia technology, a large number of videos and images are being generated and processed every day, which are often subject to quality degradation. As a fundamental technology for various image applications, image quality assessment (IQA) has always been an important issue. The aim of IQA is to automatically estimate image quality to assist in the handling of low-quality images during the capture process. IQA methods can be divided into three major classes, namely, full-reference IQA (FR-IQA) ,2, reducHowever, attempts to use DNNs for the BIQA task were limited due to the conflicting characteristics of DNNs and IQA . DNNs reBased on the baseline datasets considered for expansion, the current DNN-based BIQA methods can be divided into two general approaches. The first approach is to use the images in an existing IQA dataset as the parent samples; we call this approach small-scale expansion. In this case, the goal of expansion is achieved by dividing the distorted images from the IQA dataset into small patches and assigning to each patch a separate quality label that conforms to human visual perception. The second strategy is to expand the number of distorted images by using another, non-IQA dataset as the parent dataset; we call this approach large-scale expansion. In this approach, nondistorted images from outside the IQA dataset are first selected; then, distortion is added to these images based on the types of distortion present in the IQA dataset to construct new distorted images on a large scale. Then, the newly generated distorted images are simply labeled with different values that reflect their ranking in terms of human visual perception quality to achieve the goal of expansion.The small-scale expansion strategy relies on division. The initial algorithm  assigns To avoid dividing the images in the IQA database while still not requiring human labeling, the large-scale expansion strategy instead involves creating new distorted images by adding distortion to a large number of high-definition images obtained from outside the IQA database. Separate values that reflect the overall quality level are assigned to each distorted image obtained from each original parent image. Because the labels of the newly generated images are not direct quality scores, the expanded database is used only to pretrain the DNN, which is then fine-tuned on the IQA database. This approach alleviates the training pressure placed on the small IQA dataset and successfully avoids the drawbacks of division encountered in the small-scale strategy because the number of labeled training images is expanded by a large amount, increasing the diversity of the training sample content. Such unrestricted, large-scale expansion also makes it possible to use deeper networks; in fact, a deep model pretrained on an image recognition task could also be used to further enhance the effect. This large-scale expansion approach was developed over the past two years, and it showed a much better effect than small-scale expansion algorithms. However, large-scale expansion also has some significant shortcomings. Although the newly added images with quality-level labels are consistent with human perception, they reflect only HVS-aware quality factors; distortion and the joint effects of saliency and distortion are not considered. Moreover, large-scale expanded datasets are typically prepared to assist in specific IQA tasks. The more similar the extended pretraining dataset is to the original IQA dataset for the target task, the more effectively it can support the IQA task. In this case, a “similar” dataset is an expanded dataset that fully reflects the influences of the HVS-related perceptual factors  as embodied in the IQA task of interest. The current algorithms ,20 that In this paper, we introduce saliency into the large-scale expansion method, with the aim of constructing DNN-based BIQA models that will be effective in various cases where image quality is crucial. The objective is to be able to automatically estimate image quality to assist in handling low-quality images during the capture process. Moreover, by virtue of the introduction of saliency, our proposed model can achieve better prediction accuracy for large-aperture images (with clear foregrounds and blurred backgrounds), which are currently popular. We propose a new approach for incorporating saliency into BIQA that is perfectly compatible with the large-scale data expansion approach to ensure the full consideration of HVS-related factors in the mapping process. Specifically, we introduce saliency factors through regional distortion, thereby conveniently combining saliency and distortion factors during the expansion of each image to generate a series of distorted image versions. Then, we use the information entropy to rank these images based on their quality to complete the labeling process. By constructing a more efficient pretraining tool for DNN-based BIQA, we improve the prediction performance of the final model. We use our generated large-scale dataset to pretrain a DNN (VGG-16) and then use the original small IQA dataset to fine-tune the pretrained model. Extensive experimental results obtained by applying the final model to four IQA databases demonstrate that compared with existing BIQA models, our proposed BIQA method achieves state-of-the-art performance, and it is effective on both synthetic and authentic distorted images. Therefore, we conclude that a data expansion approach that fully reflects HVS-aware quality factors is beneficial for IQA. This study presents a novel method for incorporating saliency into IQA tasks, namely, representing it as regional distortion.Our contributions can be summarized as follows: (1) We introduce saliency into the large-scale expansion method in a manner that fully reflects the influence of HVS-aware factors on image quality, representing a new means of considering saliency in IQA. With the incorporation of the saliency factor, the proposed data expansion method overcomes the main drawback of its predecessor algorithm, RankIQA , which eThe remainder of this paper is organized as follows. As stated above, the main requirement for the expanded dataset is that it should be as similar as possible to the original IQA dataset. Therefore, to identify some features of good BIQA model design, we analyzed the influence of the three functional aspects of the HVS on human visual IQA. To improve the reliability of the results, all images considered below were taken from the IQA dataset, which consists of distorted images and subjective quality score labels that are often used as criteria based on the human visual perception mechanism.As previously discussed, saliency is an important factor that influences image quality because when people observe an image, they tend to focus on the regions that contain the most relevant information in the visual scene. Previous HVS evaluation experiments with eye trackers ,12,13,21To conduct a detailed analysis of the substantial impact of saliency on quality, we analyzed several images with different visual quality scores. The images shown in panels (a) and (b) of By contrast, the effect in nonsalient regions is the opposite. As shown in We will now discuss the crucial impact of content on IQA. We present detailed figures for observation. Among the existing IQA databases, LIVE is the most commonly used. Its 29 reference images were distorted using five types of distortion: JPEG2000 (JP2K), JPEG, white noise in the RGB components (WN), Gaussian blur (GB), and transmission errors in the JPEG2000 bit stream using a fast-fading Rayleigh channel model (FF). Moreover, different levels of distortion were added to each reference image using the same distortion type to ensure that the quality of the distorted images of the same distortion type covers the entire quality range. To draw our conclusions, we selected 4 of the 29 reference images  in the public IQA database. This phenomenon is clearly reflected in The degree of distortion seriously affects the image quality. This conclusion is obvious and beyond doubt for all four distortion types displayed in The above analysis provides the following inspiration. When a DNN learns a mapping between distorted images and quality scores, it is actually learning different curves for different image contents. This suggests that it should be possible to construct an expanded training database to improve the DNN-based prediction performance on BIQA tasks by simply adding synthetic distortions to a baseline database containing a large number of original images, exactly as was done to construct the LIVE database. As discussed above, the original external content is not subject to any rating biases; therefore, we should find a parent database that consists of multiple types of images with no distortion and then distort them using several distortion types. For each type of distortion, we could generate a series of distorted images of different qualities for each original image such that the generated distorted images would reflect the joint effects of saliency and distortion on image quality. Then, we could apply a reasonable two-stage training method. First, pairs of images of different qualities from the expanded dataset would be sent to the DNN to pretrain the network to learn the quality ranking of distorted images with the same content. Then, the smaller original IQA database could be used to fine-tune the DNN, which would already be trained to perform quality ranking, to refine the mapping of distorted images to quality scores for each type of content. Then, the DNN should be able to output high-precision score prediction results. The authors of the RankIQA algorithm  accompliAs mentioned above, our main goal is to construct a newly expanded dataset to support DNN-based BIQA tasks. We introduce saliency into the large-scale expansion strategy for the first time by creating distorted images based on the joint consideration of both saliency and distortion. Finally, we label the images based on the information entropy. The degradation of image quality in our new expanded dataset not only is related to the distortion level (as in RankIQA ) but alsIn this section, we present a detailed description of our method, which is divided into two main stages: dataset expansion and the use of the expanded dataset. First, we introduce our novel method of incorporating saliency into the large-scale dataset expansion process for IQA. Then, we describe the dataset generation process: image expansion based on saliency and distortion and image labeling guided by the information entropy. Finally, we describe how the expanded dataset is used in the IQA task, which involves a two-step training process to ensure that the DNN fully learns how HVS-aware factors influence image quality.The incorporation of saliency into the expansion procedure is a key step because we want to consciously capture the influences of both saliency and distortion when generating distorted images. Previous algorithms ,17,18,19On the one hand, the characteristics of the large-scale expansion strategy are as follows: the time-consuming psychometric approach is not employed to obtain subjective score labels, and each distorted image derived from a given image by applying a given type of distortion has only a simple numerical label that represents its level of quality. On the other hand, To implement the approach proposed above, we performed two preparatory steps. First, we needed to choose a saliency model. From among the many possible saliency models, we selected  because We selected the Waterloo database , which iAs an example, we chose one original parent image (“shrimp” from the Waterloo database), and the image shown in panel (b) is its saliency map, generated as described in . Due to Our expanded set of distorted images fully reflects the influence of HVS-aware quality factors. The nine distorted image versions generated from each parent image contain different levels of distortion across the entire image region, thus representing the influence of the overall distortion level on quality. In addition, some distorted images have different levels of distortion in the salient and nonsalient regions, thus representing the joint influence of saliency and distortion on quality. We ranked the nine distorted images of the same distortion type generated from each original image separately. The corresponding distorted image versions of decreasing quality can fully reflect the quality degradation caused by various HVS-aware factors.H represents the information entropy of the image and i in the grayscale version of the image. The ordering of the information entropy values reflects the quality ranking of a group of images. We used this formula to calculate the information entropy of each of the nine distorted images in one group and ranked these nine images in order of their information entropy values. Accordingly, labels 1–9 were assigned to represent the image quality ranking. As mentioned above, there are a total of 4744 × 4 groups in our expanded dataset. We use letters c to k to denote the distorted image versions generated to compose each group . For each of these nine distorted image versions, we calculated the average entropy for the corresponding 4744 × 4 images, as shown in c to k range from 1 to 9, representing their sequentially decreasing quality. For example, for the nine images in d  and e  are reversed. However, we still sort image e below image d in quality to emphasize the importance of the salient region.After generating the distorted images, we next assigned quality labels to them. We know that each image in an IQA database will have an assigned quality score generated through a time-consuming psychometric experiment, an option that is unavailable to us, and is, in fact, unnecessary. Labels that simply reflect the quality ranking are sufficient to create the needed effect  and the Pearson linear correlation coefficient (PLCC). Given N input images, the SROCC is calculated as follows:In We compared the performance of our method on several IQA databases with the performance of various state-of-the-art FR-IQA and NR-IQA methods, including the FR-IQA methods PSNR, SSIM  and FSIMFirst, it can be clearly seen that our proposed model achieves the highest PLCC and SROCC scores on almost all tested databases, indicating that the proposed data expansion method for DNN-based BIQA has the best overall effect for both synthetic and authentic distortion databases and is largely consistent with the subjective judgments made by humans. Moreover, we can see that compared with its predecessor algorithm, RankIQA, our method achieves better results on all of the datasets listed, especially on LIVEC, because the introduction of the saliency factor causes the model to somewhat depend on the distortion type consistency between the expanded dataset and the original IQA database. The performance improvement on CSIQ is also considerable, possibly because the reference images in this dataset include many examples with clear foregrounds but blurred backgrounds.Because the SROCC and PLCC results were averaged over ten runs in our experiment, we also present the standard deviations of these results to illustrate the stability of our model’s predictive performance. y-axis represents the predicted quality scores obtained with our method. Because the four databases use different subjective score labels , there are two different x-axis ranges in x- and y-axis scales range from 0 to 1. For the other three databases, unified scales from 0 to 100 are used.To further visualize the consistency between our method’s final predicted scores and the subjective human perception scores in the IQA databases, we show scatter plots of the scores predicted by our model (pretrained on the expanded dataset and fine-tuned on the corresponding IQA database) versus the ground-truth labels (DMOSs/MOSs) in The output of our pretrained model is not a direct image quality score. Only when multiple different images are input and their output values are obtained can the order of these values reflect the quality ranking of the images. To further evaluate the contribution of our expanded dataset and more accurately evaluate the contribution of the incorporation of saliency during the pretraining stage, we applied our pretrained model to various images and compared its predicted outputs to evaluate whether it could precisely rank images by quality. We compared our model with the pretrained RankIQA model, for which only images with whole-region distortion are considered during pretraining. Five image examples from the CSIQ database are shown in In this section, we study the effects of using different parent databases for expansion. To confirm the effectiveness of our selected parent database, we performed tests using different databases as parents for data expansion, including the Waterloo database, which consists of images with rich scene contents, and MSRA-B , anotherAs mentioned in In this paper, we have proposed a new approach for considering saliency in IQA. In this approach, we expand a large-scale distorted image dataset with HVS-aware labels to assist in training a DNN model to more effectively address IQA tasks. The novel feature of the proposed method is that this is the first time that a saliency factor was incorporated into the large-scale expansion strategy by representing saliency the form of a regional distortion. Then, by using the information entropy to rank the generated images by quality, we ensure that the labels in the newly expanded dataset are highly consistent with human perception. The ability to fully consider the various factors affecting image quality also solves the overfitting problem. Specifically, the introduction of saliency not only improves the applicability and versatility of the overall model but also overcomes the heavy reliance of the predecessor to our algorithm on the degree of similarity between the distortion types in the expanded dataset and the original IQA database. The final experimental results demonstrate the effectiveness of the proposed method, which outperforms other advanced BIQA methods on several IQA databases.


Many countries advocate sports for all to cultivate people’s interest in sports. In cities, cross-industry alliances between sports and tourism are one of the common practices. The following two important issues need to be discussed, namely, what factors should be paid attention to in the development of sports tourism, and what are the mutual influential relationships among these factors. This study proposes a novel two-stage multi-criteria decision-making (MCDM) model to incorporate the concept of sustainable development into sports tourism. First, the Bayesian best–worst method (Bayesian BWM) is used to screen out important criteria. Bayesian BWM solves the problem of expert opinion integration of conventional BWM. It is based on the statistical probability to estimate the optimal group criteria weights. Secondly, the rough decision making trial and evaluation laboratory (rough DEMATEL) technique is used to map out complex influential relationships. The introduction of DEMATEL from the rough set theory has better practicality. In the calculation program, interval types are used to replace crisp values in order to retain more expert information. A city in central Taiwan was used to demonstrate the effectiveness of the model. The results show that the quality of urban security, government marketing, business sponsorship and mass transit planning are the most important criteria. In addition, in conjunction with local festivals is the most influential factor for the overall evaluation system. Including some sports activities or watching sports events in the tourist itinerary has become one of the major development projects of the tourism industry . Sports Due to abnormal climate change and frequent natural disasters, many international organizations , European Union (EU) and World Trade Organization (WTO)) have called on all industries to pay attention to “sustainable development” and formulated many regulations and agreements on environmental protection . TherefoIt is an important task to develop an effective urban tourism development evaluation model ,9. Multi(i)The traditional sustainability concept revolves around social (S), environmental (B) and economic (C). The addition of “institutional sustainability (I)” makes the evaluation structure more complete;(ii)Bayesian best–worst method (Bayesian BWM)  is used (iii)In the calculation of DEMATEL, it combines rough set theory to optimize the applicability of the conventional DEMATEL;(iv)The mutual influential relationships among the criteria are identified using rough DEMATEL to support decision-makers in developing urban sports tourism development strategies;(v)The proposed methodology is not limited to any industry and various industries can imitate and develop their own decision-making systems.According to the literature review, the evaluation system of urban sustainable sports tourism for cities has not been established yet. The purpose of this study is to propose a novel two-stage MCDM model to establish the evaluation criteria for cities to develop sustainable sports tourism and to explore the mutual influential relationships among the criteria. The evaluation framework is new, and the proposed hybrid methodology has not appeared. In summary, this model brings some contributions and innovations to sustainable tourism development for the cities:The rest of this article is organized as follows. Many countries promote sports tourism by joining sports, setting up specialized sports tourism agencies whether in large cities or local towns ,18. SporThis study proposes a novel evaluation framework to determine the evaluation criteria and their mutual influential relationships. For cities to develop sustainable sports tourism, they must receive support from local governments and the tourism industry. First, important criteria should be fully integrated into the evaluation system to reflect the characteristics and connotation of sports tourism. The initial criteria review was based on relevant academic literature ,18,19,24The chosen case is Taichung City, Taiwan. The Taichung City Government actively promotes sports infrastructure and promotes the correct sports concept to implement “sports for all ages”. In December 2019, the Taichung City Sports Bureau decided to organize marathons to connect the sports events with local specialties in order to serve the purpose of marketing the city and promoting culture. In 2020 alone, Taichung City has already prepared at least 35 marathon events. However, building an image of a sports city is a difficult and complex project; many factors and restrictions must be considered, including economic feasibility, social development, environmental awareness and policy support. Only through continuous review and improvement can we move towards the vision of urban sports for all ages. At present, there has not been a sustainable sports tourism evaluation system developed specifically for the cities. In addition, most studies have not examined the mutual influential relationships among criteria. Which evaluation criteria are the main factors that affect the success or failure of urban sports tourism? How do these criteria affect other criteria? These two issues are the focal points of this study.In the study, the decision-making team consisted of ten experts, including tourism managers, members of the Ministry of Tourism and academics. These ten experts have at least 8 years of qualifications in sports events or tourism-related jobs; their current jobs are highly relevant to the development of sports tourism. The proposed evaluation framework is presented in Next, we describes the method used and its detailed calculation process. In the first stage, preliminary evaluation criteria were established based on the literature discussions on sports tourism and sustainable tourism. Due to the large number of evaluation criteria, screening must be performed to retain key criteria. Based on the interview data of several experts, the Bayesian BWM was used to calculate the weight of criteria and select the key criteria. Bayesian BWM—proposed by Mohammadi and Rezaei —effectivBayesian BWM effectively integrates the opinions of multiple experts to generate a set of optimal group criteria weights. Its survey process is simple and intuitive. Experts are asked to choose the most important and least important criteria; then they are compared pairwise with other criteria to form a structured set of two vectors. Based on the concept of statistical distribution, the optimal group criteria weights are estimated. The detailed Bayesian BWM derivation and proof can be found in the study of Mohammadi and Rezaei . The impThe evaluation criteria Based on the set of criteria, each expert chooses what s/he considers the most important and least important criteria.B relative to criterion j.Each expert evaluates the relative importance of the most important criteria to other criteria to generate the BO vector. The ratings of BWM are shown in j relative to the least important criterion W.Similar to Step 3, each expert evaluates the relative importance of the other criteria to the least important criteria to generate the OW vector.Each expert follows Step 1 to Step 4 to get multiple sets of BO and OW vectors. According to the MATLAB program software provided by Mohammadi and Rezaei , the evaic and jc and use the concept of Credal Ranking to test their confidence. Then the probability that ic is better than jc isI is the condition parameter, which can be calculated when Q obtained by it is used to calculate its average confidence level.qi is more important than criterion j and the probability presented is the confidence level. In addition, the total probability is 1, After the weights are obtained, it must be checked whether the ranking of the weight is consistent. Assume that the two criteria in the criteria set are n criteria in the criteria set, The This step can distinguish the relatively important and relatively unimportant criteria groups. We retain the rules that are larger than A are 4, 4, 3, 2 and 2, respectively, then lower and upper bounds of rough numbers . This is because AHP and BWM use arithmetic averages when integrating experts’ opinions. This method is vulnerable to the influence of extreme values, resulting in the loss of some information. In contrast, Bayesian BWM, which pays extra consideration for G8 and I2, makes the influential relationship system of the criteria more complete. It must be noted here that I2 and G8 are important affecting and affected factors in the analysis of rough DEMATEL.The implementation process of rough DEMATEL is explained in n is the number of samples, p is the number of experts and t is the evaluation value in the matrix. Based on this index, the average gap of the 10 experts is 4.8%, which means the confidence level is 95.2%, indicating that these experts have a high degree of consensus.The consensus degree of the experts can be viewed by average sample gap index , increasing the number of visits to the attractions in the city (E7), marketing and promotion by local governments (I7), sponsorship and support by local businesses (E5) and maintenance of the urban tourism website (I2). These criteria will affect the performance of other criteria. This result echoes the management implications of many studies, including Pouder et al. , Huang eIn summary, the two-stage evaluation model proposed in this study provides a complete and systematic method, providing the management implications of the development of sports tourism in the cities. This effective soft calculation method can reduce the subjectivity of management decisions. The academia has not yet studied and explored the mutual influential relationships among the criteria for sustainable sports tourism. Our model integrates several state-of-the-art methods and takes into account a variety of realistic factors, including the consideration of message uncertainty and the introduction of the concept of rough set theory.It is well known that sports bring many benefits to people’s physiology and psychology. The spirit of sustainability has brought into the sports tourism industry the purpose of accelerating the expansion of the “sports for all”. This study proves the effectiveness and reliability of the proposed model. It should bring several benefits to practitioners and sports-related sectors: (i) identifying the most important and influential criteria; (ii) providing an improved basis for urban development sports tourism; (iii) helping decision-makers in the decision-making process to be more systematic.In the future, researchers can further investigate the quantitative data of the actual assessment, making the evaluation results more accurate. Beyond that, using Bayesian BWM for cross-dimensional criteria comparison can entail more discussion.


Though selective laser melting (SLM) has a rapidly increasing market these years, the quality of the SLM-fabricated part is extremely dependent on the process parameters. However, the current metallographic examination method to find the parameter window is time-consuming and involves subjective assessments of the experimenters. Here, we proposed a supervised machine learning (ML) method to detect the track defect and predict the printability of material in SLM intelligently. The printed tracks were classified into five types based on the measured surface morphologies and characteristics. The classification results were used as the target output of the ML model. Four indicators had been calculated to evaluate the quality of the tracks quantitatively, serving as input variables of the model. The data-driven model can determine the defect-free process parameter combination, which significantly improves the efficiency in searching the process parameter window and has great potential for the application in the unmanned factory in the future. Selective laser melting (SLM), a powder-based additive manufacturing (AM) which is capable of fabricating components with complex shape, has now been increasingly used to produce metallic parts especially in the aerospace field . BesidesFor the single track, laser power and scan speed during the process is of great importance. Various efforts have been made to obtain the suitable combinations of them. Yadroitsev et al. ,9 obtainMachine learning (ML), a data-driven method with the superiority of learning from the given data and making the decision automatically by itself ,16, is cHere, we propose a supervised machine learning (ML) method to detect the track defect and predict the printability of material in SLM intelligently, which can figure out the suitable parameter combinations to fabricate the defect-free track and therefore the part with excellent performance. Thirty-three single tracks under different parameter combinations of laser power and scan speed were fabricated. Then the formed tracks were classified according to their surface morphologies and characteristics. Based on the analysis of the surface morphology, four indicators had been calculated to evaluate the quality of the tracks quantitatively. A backpropagation-based neural network model, one of the ML methods, was then developed. The classification results were used as target output while the four indicators serve as input variables. Finally, defect detection and printability prediction can be realized by using the trained neural network. Taking advantage of the automatic feature-learning capacity of machine learning, this approach has great potential to determine the printable parameter window intelligently in the unmanned factory.Extracting data from the experimental results: The formed tracks are sorted according to their surface quality, followed by calculating the value of the four evaluation indicators. Evaluation indicators and classification results are used as input variables and target output, respectively, of the following neural network model.Training the ML model: The samples are randomly selected for the training and testing process. A backpropagation-based neural network model was set up for the prediction task in this work. The factors which could affect the prediction accuracy includes the size of the database, the learning algorithm, and the network structure.Practical prediction to guide the SLM process: After inputting the related evaluation indicators of the parameters to be examined, a value representing the track’s possibility of having defects will be returned, which could help to guide the SLM process.The workflow started from the SLM experiments and classification of the formed tracks. The printed tracks are classified into five types based on the measured surface morphologies and characteristics. The classification results are used as the target output of the ML model. Four indicators have been calculated to evaluate the quality of the tracks quantitatively, serving as input variables of the model. Specifically, a backpropagation-based neural network model was set up for the prediction in this work. The newly developed prediction method consists of three steps:2/Al composites through the exothermic reaction via mixture salts of K2TiF6 and KBF4 and obtaining as-cast ingots by adding high purity ingots of alloying elements [®DMP200, 3D Systems, Inc., Rock Hill, SC, USA) employed in this research was equipped with a fiber laser with a wavelength of 1070 nm and a spot diameter of 75 μm. Before the printing experiment, pure argon gas was delivered into the chamber until the oxygen content detected by the built-in oxygen sensor of the SLM machine reached the range between 0 and 1 ppm, aiming to mitigate the oxidation content in melt pool during the fabrication process. A powder layer thickness of 30 μm was set and the continuous laser mode was used during the fabrication. Then thirty-three parameter sets were used to produce single tracks whose length were all 1 cm on a bare plate of 5083 Al alloy . The effect of scan speed (V) in the range of 200 to 2200 mm/s was investigated, and three laser powers (P)  were adopted.The homemade in-situ 6 wt.% nano-TiB2 reinforced AlSi10Mg composite was used in this study. The as-cast composite was synthesized by an two-step in-situ mixed salt reaction method which were preparing TiBelements . Then thThe surface morphologies of the thirty-three single tracks were obtained through Laser Scanning Confocal Microscope . Simultaneously, for each single track, a 3D matrix file with the dimensions of X, Y, and Z, respectively, containing the microscope information, was also exported. X and Y referred to the directions in the measurement plane while Z referred to the relative height to the standard plate. We used the MATLAB software  to process the data, providing information about the outline of the track’s cross-sections perpendicular to the laser beam travel direction. The number of cross-sections extracted from a track was related to the enlargement factor and lateral resolution of the measurement equipment during the scanning process. In this work, 2048 cross-sections were extracted from a track.Subsequently data of the width and height of each cross-section were extracted for each track. The average and standard deviation values of both width and height were then calculated and used to obtain the ratios for each track,  follows :(1)x¯=∑iAmong all the ML models, neural network (NN) was selected in this work. The neural network is composed of many interconnected nodes which are arranged in layers , and a transfer function is used to introduce nonlinear relationship between the layers. The nodes receive input from an external source or nodes in the previous layer and produce an output with the transfer function. Each connection between two adjacent layers has an associated weight  to repreFor the training process, we used a backpropagation algorithm called gradient descent with momentum (GDM). GDM algorithm is an optimization of the standard gradient descent approach, which considers the gradient of both the current and the previous step when updating the weights and bias terms ,35. The −6.We used the MATLAB software and its neural network toolbox to create our GDM-based neural network model. The model consisted of three layers: an input layer with four nodes, a hidden layer with six nodes, and an output layer with one node. The input variables were connected to the output variable via the neural network. We used a hyperbolic tangent as the transfer function, and the output of a node was updated with the following equations ,34:(6)X=In this research, we studied the effect of laser power and scan speed on the quality of the tracks by analyzing the surface morphologies and characteristics of the printed single tracks. Five types of tracks were observed as depicted in During the SLM process, the energy input increases with laser power and decreases with scan speed. When laser power was low and scan speed was high, experimental results showed that the printed single track of type I was discontinuous and badly bonded to the substrate. The energy input was not sufficient to fully melt the powder layer in the circumstances. As scan speed decreased slightly, the energy input melt through the powder layer but balling phenomenon happened. Balling phenomenon was the manifestation of Plateau-Rayleigh capillary instability in nature’s way ,4, whichFor each track, 2048 cross-sections were obtained and data of the width and height of them were extracted. The average and standard deviation values of both the width and height were then calculated and used to obtain the ratios, Then we confirmed four evaluation indicators i effect ,37, and The results of these indicators were plotted in Among all the ML models, neural network (NN) model was selected in this work. The classification results were used as target output while the four indicators serve as input variables of the NN model. The structure of the model was described in We processed a normalization of the data for each indicator by dividing each variable by its maximum value. A dot distribution map showing the normalized results of the four evaluation indicators, which were served as the input variables of the NN model, is displayed in After the training process, the NN model was tested by inputting the normalized results of the evaluation indicators of nine parameter combinations of laser power and scan speed. The predicted results from the trained NN model and the corresponding target outputs of the nine parameter combinations were demonstrated in Taking advantage of the automatic feature-learning capacity of machine learning, this method can obtain a parameter window for single track intelligently and satisfy the demand of the unmanned factory. And the method explored in this paper can be applied to any other materials. It needs to be mentioned that the tracks will interact with the bare plate only at the first layer process during the practical SLM fabrication. In most cases, the tracks will be deposited in the SLM-fabricated layers, which needs further study in the future.As SLM is a track-by-track and layer-by-layer manufacturing process, the suitable combinations of the laser power and scan speed for a defect-free track is the firm foundation for obtaining a SLM-fabricated part with excellent performance. Multi-track and multi-layer SLM experiments, which take the other process parameters like layer thickness, hatch space and scan strategy into consideration, should be conducted to have a more comprehensive understanding of the key factors that affecting the fabrication quality of the realistic feature in the future. The prediction method developed in this work can also be extended by extracting indicators related to the multi-track and multi-layer SLM experiments and used for the decision-making processes of the other process parameters.A prediction method for selective laser melting using machine learning model was developed, which could detect the defect track and predict the printable parameter intelligently.The printed single tracks were classified into five types based on the measured surface morphologies. The classification results were used as target output for the ML model.Four evaluation indicators were determined to evaluate the quality of the tracks quantitatively. They were highly correlated to the surface morphology and key geometrical characteristics of the printed single track.This approach with a backpropagation-based neural network model was successfully used to predict the process parameter window (laser power and scan speed) for TiB2 reinforced AlSi10Mg composite. The feasibility of this prediction method had been proved by experiment.We proposed a supervised machine learning (ML) method to detect the track defect and predict the printability of material in SLM for the application in the unmanned factory. The developed method helps to figure out the suitable parameter combinations for defect-free printing intelligently. The morphology of the track could be measured by a 3D microscope, which has the potential to be integrated into the selective laser melting system for in-situ measurement. The specific findings are as follows:


In contrast, the average gain in the MFR decreased from EC to EO/SI  and further to EO/CI (p = 0.0002). In the HFR, all three visual conditions produced, similarly, high gain levels. A single inverted pendulum (SIP) model controlling center of mass (COM) balancing about the ankle joints formally described the EC response as being strongly shaped by a resonance phenomenon arising primarily from the control’s proprioceptive feedback loop. The effect of adding visual information in these simulations lies in a reduction of the resonance, similar as in the experiments. Extending the model to a double inverted pendulum (DIP) suggested in addition a biomechanical damping effective from trunk sway in the hip joints on the resonance.Vision is known to improve human postural responses to external perturbations. This study investigates the role of vision for the responses to continuous pseudorandom support surface translations in the body sagittal plane in three visual conditions: with the eyes closed (EC), in stroboscopic illumination  and with eyes open in continuous illumination  with the room as static visual scene . In the frequency spectrum of the translation stimulus we distinguished on the basis of the response patterns between a low-frequency, mid-frequency, and high-frequency range . With EC, subjects’ mean sway response gain was very low in the LFR. On average it increased with EO/SI (although not to a significant degree  Humans use mainly vestibular, proprioceptive, and visual cues for their balancing of upright stance  and adju1 condition to vestibular loss (VL) and vestibular-able (VA) subjects. Interestingly, to improve the balancing capability of the VA subjects’ the visual tilt magnitude had to exceed the value of their vestibular threshold.Studies emphasizing the importance of vision for balancing control often relied on tests of patients with a degraded vestibular or somatosensory sense . Also, tIn earlier work of 1). By this, the IC model allows replicating sway responses observed in human tilt experiments using a simple sensory feedback control scheme that linearly combines weighted sensory cues.Formal descriptions of the involved sensorimotor mechanisms are still a topic of ongoing research. The present study investigates the postural response to support surface translation in the body sagittal plane with the aim to formally describe the effects of vision on the response using different visual viewing conditions. For comparison, for the formal description of tilt responses, 2 and in that it contains non-linear elements (detection thresholds). But it is also more general in that it covers other external perturbations such as support surface translation  model . The DECnslation  and in tnslation  and coornslation .Generally, model description can help to better understand biological functions. Conceptually, visual improvements of postural control have in the past often been attributed to a reduction of sensory noise in the system, presumed to stem foremost from the vestibular signal. To formally describe the phenomenon, multi-sensory integration methods such as Kalman filtering or the Bayesian estimation theory have been used . The noiIn the present context of support surface translation stimuli, however, vestibular noise appears to play no major role. As emphasized in the study of The quality of the visual stimulus used in such studies is, however, often of relevance. In the experiments of Taken together, the present study aims to formally describe the role of vision for human postural responses to support surface translations such that model and robot simulation results approximately match the experimental results. With the focus on the role of vision for the translation response, we distinguished between the effects of visual position versus velocity plus position cues by using stroboscopic (flashed) versus normal (continuous) illumination. Using for support surface translation stimulus a continuous pseudorandom ternary sequences (PRTS) translation profile allowed us to cover a broad band of frequencies of the postural responses. Our modeling of the responses suggests that these were shaped in terms of an amplitude enhancement to a large extent by a resonance of the proprioceptive feedback loop controlling the ankle joints, and that vision of a stationary scene is damping this resonance together with a biomechanical effect from hip bending.The experiments were performed in the posture control laboratory of the Neurological Clinic, University of Freiburg, Germany. The study was approved by the Ethics Committee of the Freiburg Clinics and was conducted in accordance with the 1964 Declaration of Helsinki in its latest revision.76.8 ± 7.5 kg and 1.73 ± 0.08 m, respectively. All subjects were without a history of balance impairment and epilepsy . Each subject performed the experiments twice . The sensitivity of the performed tests (ANOVA) with the given number of subjects was analyzed. Setting the power to β = 0.80 and the significance level α = 0.005 the minimum detectable effect size was 81% of the common standard deviation within the groups.Seven subjects  participated in the study after giving written informed consent. Subjects’ mean mass and height were Subjects stood on a motion platform holding in each hand a handle of a safety rope, which gave no support or spatial orientation cues with flexed arms during trials but supported the body when the arms were stretched compare . Body kiEn-bloc translation of cabin and platform . Overall, seven experimental stimulus conditions were used (The experiments were performed with the eyes closed (EC) or open (EO). With eyes open, we used for the visual scene two conditions. In one set up condition, the visual scene was represented by the interior of the illuminated laboratory room . In this situation, the visual stimulus arose from the motion of the body on its translating support with respect to the lit interior of the stationary walls of the laboratory room while looking straight ahead with a distance of approximately 3 m to the front and to each side wall . In the second set up, the visual scene consisted of the interior of a lightweight cabin . Using the cabin, two visual motion conditions were used (translation amplitudes as before pp. 9.6 cm): (a) Translation of the cabin in space, with the subject standing on stationary external support and viewing translation of the cabin’s interior visual scene  ere used . Strobosere used , 2015. Tere used  with 242The amplitude spectra of support and/or scene displacement, velocity, and acceleration of the stimulus are given in In pilot experiments, we familiarized subjects with the experimental setup and the stimuli. In these experiments, we observed that the sway response to the translation stimulus involved a considerable upper body  sway on the hip joints .All analysis steps orient on the methods proposed by Recorded sway kinematics were exported to Matlab  for analysis. Sway kinematics of the leg and the HAT  segment were obtained from the hip and shoulder marker translations and the manually measured marker heights using trigonometric functions. Angular sway of the body center of mass  was obtained from these measures and the anthropometric data of The frequency-domain representation of the variables  was obtained using the fast Fourier transform implemented in the Matlab function ‘fft’. After averaging the spectra across stimulus cycles, frequency response functions (FRFs) were calculated dividing output  by input (translation stimulus spectrum) for each frequency point and expressing the result in terms of gain  and phase  model  A For the robot simulations, we used the humanoid robot Posturob II which operated with the DEC control . In the high-frequency range (HFR), gain levels off to a value slightly below unity. The phase passes in the transition from the mid to high-frequency range a lag of 180° (corresponding to a lean of the body COM counter to the stimulus).The FRF results obtained for the seven visual stimulus conditions listed in p = 0.078), but gain is slightly increased in the low-frequency range and reduced in the mid-frequency range. These changes became enhanced when allowing additionally for visual velocity information in continuous illumination . Noticeably, however, in the high-frequency range, gain reached a similar value close to unity with all three illumination conditions (note arrows at the endpoint of the gain FRFs in the three panels).With stroboscopic illumination of the laboratory (EO/SI), providing visual position information on self-motion, the patterns of the phase and the gain FRFs  basicallAs described in Introduction, viewing a moving visual scene may evoke body sway responses on the background of self-motion illusions. Here, the cabin was translated with the PRTS stimulus profile while the subjects stood on the stationary body support viewing the moving cabin’s interior visual scene. This visual stimulus-evoked only very small sway responses mainly in the low-frequency range, this with stroboscopic illumination EO/SI;  and onlyThe results obtained in the experiments with the cabin fixed on the translating body support (Cabin&Body) are shown in Using the DEC model shown in Considering the relative contributions of the DEC for compensation of the gravity effect versus that for the acceleration force, we found that the contribution of the latter was relatively small. Generally, its gain requires a downgrading to avoid control instability in face of the second derivative term (acceleration) in the sensory feedback  in the human laboratory using the same stimuli and analyses as described for the human subjects. The results for the body COM sway obtained in the absence (“EC”) and the presences (“EO/CI”) of visual information are given in This study’s aim is to formally describe how humans adjust their sensorimotor control when they use their joint proprioception combined with visual cues for balancing of erect stance during horizontal support surface translations in the body sagittal plane. As a means to shed light on this control from different perspectives, we used the three viewing conditions EC, EO/SI, and EO/CI. The DEC model  was usedresonance phenomenon, which is damped when visual cues became available. This implies (B) the question in which way the visual cues were reducing the resonance tendency and what the role of involving hip motion in this respect may be. (C) Finally, we focus on more general considerations such as the comparisons with earlier work that used sinusoidal and thus predictable  translation stimuli and (D) on biomechanical aspects.In the following, we address: (A) A concept on how to interpret the sequence of a reduced sway in the LFR followed by a pronounced gain peak in the MFR in the EC condition, with both effects showing a diminished manifestation in the EO conditions. Our hypothesis is that the gain peak represents a Our hypothesis is that the gain peak in the mid-frequency range in the EC condition  results In one set of simulations, we focused on changes of the loop gain . In the absence of the translation stimulus, noise alone produced no specific response. In contrast, noise injection in combination with the presentation of the translation stimulus led to a sway response that built up over repetitions. It showed features comparable to our experimental results with respect to the gain peak, the phase, and the coherence . We takeA straightforward hypothesis related to the above resonance concept is that the visual information, which we provided in the EO conditions, was damping the resonance. We dealt here with two visual signals , both of which we assume to antagonize in our experiments the resonance evoked by the translation stimulus .As detailed in our full SIP model , we attrWith the visual-only stimuli in our experiments, the small sway responses we obtained differed remarkably from the pronounced sway in the model simulation . The reaCompared to these findings, there exists a seeming contrast to several studies reporting that postural sway can readily be evoked by visual field motion (see section “Introduction”; en bloc with the support surface. At higher frequencies, the upper body tended to sway less or remain almost stationary, while the lower body on the support below was swinging. With the eyes closed, the evoked sway changed and resembled generally more that of an inverted pendulum. Conceivably, using periodic stimuli might have allowed those subjects to involve prediction in their balancing. Yet, although our subjects reported on request the PRTS stimuli as unpredictable, there exist some similarities in the response patterns with the previous data in the sense that also our subjects showed with EC in the LFR a “riding the platform” pattern, while they were producing sway with a gain close to unity in the upper MFR and in the HFR (see EC responses in As mentioned in Introduction, previous studies  reportedA remarkable feature in both, the experimental data of The reason to performed the latter experiments with the en-bloc translation of both cabin and body support originally was to dissociate between the visual effects of the two disturbance estimators in the DEC model, i.e., one for the support surface translation and another for the evoked body lean and gravity effect (compare model in As underlined by mechanical damping of the excursion of the body COM in space. Interestingly, the study of In the study of This study suggests that humans, when balancing support surface translations in the earth-horizontal plane, stabilize upright body posture primarily on the basis of proprioceptive feedback control mechanisms in the ankle and the hip joints. Per se, this control is, due to time delays in the sensorimotor feedback control, prone to resonance effects, which tends to enhance the disturbance-related body lean effects. Our study indicates that vision as well as biomechanical effects arising with hip bending reduce the resonance tendency and thus help the postural body stabilization.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.The studies involving human participants were reviewed and approved by Ethics Committee of the Freiburg Clinics. The patients/participants provided their written informed consent to participate in this study.All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Dear Editor,The newly confirmed coronavirus disease 2019 (COVID-19) cases are still increasing strikingly in many countries according to the data reported by the World Health Organization. Liu et al. suggested that reconstructing an existing hospital into an infectious disease hospital (IDH) is an important strategy to prepare for the epidemic [Wuhan Red Cross Hospital is a secondary general hospital with 500 beds. According to the instruction of the Chinese government, the hospital became a designated hospital which only received the febrile patients since January 22. The hospital responds to it very quickly. First, all the uninfected patients were transferred to other hospitals. Second, the sixteen-story hospital building was redesigned, and the changes are shown in Fig.