K-means algorithm and noise cancellation in CIR of time domain with K-nearest neighbor (KNN) algorithm into an integrated process. Complexity analysis and simulation results showed that the proposed scheme has relatively lower complexity and the performance is proven better than traditional schemes, which meets the requirements of CPSS in complex electromagnetic environments.With social characteristics integrated into cyber-physical systems (CPS), the wireless channel has been a complex electromagnetic environment due to the subjectivity of human behaviour. For the low-power and resource-constrained nodes in cyber-physical-social systems (CPSS), minimum research is available focusing on conquering the issues of computational complexity, external interference and transmission fading simultaneously. This study aims to explore channel estimation with interference suppression based on machine learning. A novel channel estimation scheme is proposed, which combined interference suppression in channel impulse response (CIR) of frequency domain with  The cyber-physical system (CPS) was coined to describe the mapping, interaction and collaboration between computational and physical resources . With thChannel estimation is used for the recognition of fading characteristics in communication channel. For broadband communication, channel estimation is often realized by inserting training sequences of signals in time domain and frequency domain. The transform domain signal processing theory has achieved a good compromise between performance and complexity for channel estimation, which can then be applied in broadband communication . StudiesThere are two typical strategies for anti-interference. One is to pre-process before transmitting, such as transmission power allocation, spectrum sharing or reusing ,14,15. PDue to the interference in suppression and channel estimation aimed at different objects, the technologies evolved through relatively independent roadmap. While the channel estimation and interference suppression are implemented separately, the computational complexity is linear stacking in the signal detection, which increases the processing delay and the cost of hardware. Although there is some research named joint interference suppression and channel estimation, the technical relevance of conquering the two issues is not remarkable ,19,20. FThe rest of this manuscript is organized as follows: NBI is partial band jamming, which consists of a simple unmodulated carrier . Figure K is the number of interfered subcarrier.Thereby, the signal with interference at the receiver can be written as:To estimate CIR Equation  should bWe perform LS estimation to Equation , and thelth symbol in kth carrier: CIR, interference and noise. The existence of interference is uncertain, and the signal feature is unknown. System performance will be deteriorated with the interference incorrectly cancelled as AWGN [It can be seen from Equation  that the as AWGN . It is nK-means algorithm [K-means algorithm is to start from the data to be measured, first randomly generate K clusters, and then calculate the similarity between the data to be measured and the cluster center, and continuously update both the cluster center and the cluster in the iterative calculation until the convergence of the cluster center point is completed. For interference clustering, the set of subcarriers with no interference is denoted as cluster Usually, NBI is added to partial subcarriers by high-powered signals. Thus, the receive signal power of the subcarriers that are interfered with will be higher than the others. As in CIR, the modulus of the interfered subcarriers will be higher than the others. On the basis of the feature, we introduced machine learning techniques into interference cluster. Because the interference does not necessarily exist in CIR, we proposed using the clustering of interfered subcarriers based on the global lgorithm . The maiK-means interference clustering are as follows:The steps of Step 1. The data to be measured is denoted as Step 2. Euclidean distance is used to measure the similarity between Equation :(6)di,t=Step 3. The cluster center of Equation . The disEquation :(7)di,s=Step 4.quations  and 7)::Step 4.mStep 5. The cluster center Q is denoted as the number of samples in Equation :(9)ms=1QQ is not much less than N, the corresponding subcarriers are not interfered with or the interference is not significant to the subcarriers. Nevertheless, the subcarriers may be subject to non-narrowband interference and it is not the scope of this study.According to the interference cluster scheme, the modulus of samples in It is necessary to suppress interference for a is the normal carrier at the left of the interfered subcarriers, and b is the normal carrier at the right of the interfered subcarriers. Both a, b will be at the left/right of the interfered subcarriers, which are at the far right/far left.The CIR in interfered subcarriers can not reflect the fading characteristics of the subcarriers. It is assumed that the channel is stationary for a continuous spectrum transmission. Thereby, the difference between CIR in the interfered subcarriers and the adjacent subcarriers will be small. Hence, we propose to suppress interference by linear interpolation of the CIR of frequency domain. The interpolation algorithm is presented as Equation :(10)HH^LSIt must be noted that linear interpolation on the interfered subcarriers is to improve the accuracy of channel estimation, which can not restore original data on interfered subcarriers.K-means algorithm can achieve the local optimization in unsupervised learning [K-means algorithm will be an effective scheme for interference detection. However, even if the signal component is misjudged as interference, the line interpolation over adjacent subcarriers will not have a serious negative impact on CIR.The learning . On the The noise term in Equation  can be cEquation :(12)h^LSBecause the IFFT of Equation  can be rK-nearest neighbor (KNN) algorithm [K-nearest neighbors which have the greatest similarity with the data to be measured from all the training sample sets. The data to be measured will be classified into the category that is the majority of its neighbors.Because the envelop feature of lgorithm . The meaAccording to the envelop feature of The other set is denoted as P samples in For the noise components in According to the principle of It can be seen from Equations  and 17)17) that The data to be measured is denoted as Step 1. Calculate the standardized Euclidean distance between Step 2. Sort k elements. k elements, which mean that the k nearest neighbor of Step 3. Update Reiterate the three steps until all the samples in k is the number of nearest neighbors.Signal components and noise components are independent of CIR in the time domain. The classifying error of KNN in two categories is presented in :(21)RNN∗k is big enough.It can be seen from Equation  that noiK-means algorithm and noise cancellation in the CIR of time domain by the KNN algorithm. Although the original data on the interfered subcarriers can not be restored by the accurate channel estimation, the original data on the interfered subcarriers could be partially restored by channel decoding. Soft bits demodulated from interfered subcarriers should be identified in channel decoder in order to reduce the weight of these soft bits.We have achieved interference suppression in the CIR of frequency domain by the The complete process of channel estimation with interference suppression is described in In this section, we will evaluate the computational complexity and performance of the proposed channel estimation with traditional channel estimation. For easy description, the proposed channel estimation scheme is abbreviated as Discrete Fourier transform-KNN (DFT-KNN). One of the comparative schemes is the frequency domain windowed scheme, which is a typical scheme of traditional transform domain channel estimation, and abbreviated as DFT-WF . The othWe evaluate the computational complexity of DFT-KNN with others by multiply-accumulate (MAC) and multiply-add (MA) times. The computation of the algorithms modules are shown in According to the process shown in The sum of computation of interference detection and interference suppression in DFT-KNN is less than 125 MMAC. DFT-KNN achieves interference suppression with amazing low computation load. The computational complexity of DFT-KNN than that of DFT-WF increased by 12.7%, and the computational complexity of DFT-KNN than that of DFT-DA increased by 3.7%.The current processors can provide 100 GMAC (giga MAC) computing power, such as ZYNQ, C66x, in which DFT-KNN can be easily implemented . TherefoWe evaluate the performance of DFT-KNN with others by simulation comparison. The communication system parameters of simulation are shown in Mean square errors (MSE) of the channel estimation schemes are shown in In fact, all of these channel estimation schemes find the noise components by the modulus of the CIR of time domain. According to the envelop feature of CIR being known to the receiver, the signal and noise components are completely independent, and DFT-KNN that is based on a supervised learning algorithm can achieve Bayes minimum error. The frequency domain window has improved resolution of CIR of the time domain, but a lot of extra noise will be introduced to CIR by a filtering window function. It makes the performance gap of MSE between DFT-WF and DFT-KNN 5 dB more or less. DFT-DA is based on the Gaussian distribution of signal and noise components in CIR, but the signal and noise components obey the Rician distribution in the simulation, which makes the performance gap of MSE between DFT-DA and DFT-KNN by 0.5 dB more or less.The system bit error rate (BER) based on different channel estimation schemes are shown in The system BER in different interference suppression schemes are shown in The system BER in different interference styles is shown in As an important component in CPSS, data serve as a carrier system connecting cyberspace, physical world and artificial world. Depending on the communication network, data in both the physical and artificial world are transmitted to cyberspace, which are analyzed and calculated in the cyberspace. The analytic results and decision making then give feedback to the physical and artificial world through the communication network. Thus, the stability and reliability of the communication network is the prerequisite for efficient CPSS operation.When applied in telemedicine, based on various wearable device sensors, CPSS can continuously monitor physiological vital signs and transmit them for remote diagnosis. Interference in communication may cause distortion of data . When prDue to the diversity of communication systems in CPSS, frequency spectrum coordination is becoming increasingly difficult. That interference suppression provides an alternative option for anti-interference.We have applied the proposed scheme to an intelligent connected vehicle (ICV). As shown in There are multiple wireless communication systems in such a small vehicle, and the wireless channel will be a complex electromagnetic environment of mixed radio signals. In our experimental ICV, RFID and SON happened to be in the same frequency spectrum . The mutThis study proposed a novel channel estimation with interference suppression, which used machine learning to meet the requirements of CPSS in complex electromagnetic environments.K-means algorithm was used for interference detection in CIR of frequency domain, and suppressed the interference with linear interpolation of interfered subcarriers. The KNN algorithm was used for noise cancellation in the CIR of time domain, which is close to Bayes probability of error. Both K-means in CIR of frequency domain and KNN in CIR of time domain have relatively lower complexity in channel estimation, which are compliant for nodes in CPSS. Using extensive simulations, the performance of the proposed scheme is proven superior to the traditional scheme. The proposed scheme is proven to work effectively in an interference scene and the negative effects of transmission in interference are under control.The Our future work will focus on the implementation and optimization of the scheme, hopefully helping to improve the transmission stability and reliability in CPSS.


The physiological monitoring of older people using wearable sensors has shown great potential in improving their quality of life and preventing undesired events related to their health status. Nevertheless, creating robust predictive models from data collected unobtrusively in home environments can be challenging, especially for vulnerable ageing population. Under that premise, we propose an activity recognition scheme for older people exploiting feature extraction and machine learning, along with heuristic computational solutions to address the challenges due to inconsistent measurements in non-standardized environments. In addition, we compare the customized pipeline with deep learning architectures, such as convolutional neural networks, applied to raw sensor data without any pre- or post-processing adjustments. The results demonstrate that the generalizable deep architectures can compensate for inconsistencies during data acquisition providing a valuable alternative. The monitoring of daily activity patterns may support people seeking to enhance their personal fitness, promote patient engagement in the management of chronic diseases, or enable providers and patients to gain insights into the progression and impact of illnesses . A greatMost of the technological solutions previously reported in the literature for the recognition of activities of daily living (ADL) use a variety of wearable and non-wearable sensors. In this paper we present an activity classification scheme for detecting movement patterns of older people and focus on the necessary model reconfigurations for resolving challenges imposed by inconsistent measurements due to the type and misplacement of the sensors. Two approaches will be presented. The first is based on standard machine learning techniques coupled with necessary pre- and postprocessing steps to tackle data inconsistencies , which iOne of the earliest and well cited studies on sensor-based ADL classification used accelerometers in multiple locations of the body of young individuals and performed decision tree classification . The resAlthough many frameworks have been reported in the literature for activity monitoring of older people, most of them have been tested on data from young and healthy participants ,12,14, oComparable works that use wearable sensors on older people for activity recognition aim at providing the means for promoting active and healthy ageing through the development of assisted living systems or prediction of adverse events ,16,17,18Deep learning architectures, such as convolutional neural networks (CNNs), have also been previously exploited for ADL recognition ,20,21,22The work most related to ours is the CNN proposed by Yang et al. . This evFurthermore, Ordóñez and Roggen  used conMoving from a laboratory environment to real-life experiments, researchers have to deal with numerous obstacles that they must overcome, concerning mostly the devices used to monitor older people. A first challenge appears in case the device is not placed with standard orientation, causing rotation of axes. For sensors such as the accelerometer, orientation plays a significant role in recognizing the subject’s activity. Thus, a misplacement of a wearable device could easily disorientate the data analysis. It should be noted here that research has been conducted in the past regarding the sensor displacement problem, e.g., in  the authAnother issue occurs when a different type of sensor is used across individuals , or the sensors are placed in different locations on the body. This is possible when data from different clinical centers are combined or for example, after updating hardware and software components, since it causes lack of uniformity in the dataset adding barriers on inference and modeling. When activity annotations are used for model construction , additional challenges come from the inter- and intra-rater variability, the rater’s subjectivity and the interactive nature of the annotation process, especially when frail individuals are instructed and monitored. A relevant work that aimed at integrating new sensors to an existing activity recognition system is reported in , where aA SVM-based classification method is developed and assessed exclusively on older people’s recordings from a wearable sensor in everyday life conditions.Variations of the basic model are proposed to address device-relative issues that are due to data acquisition based on two different wearable devices, as well as their possible misplacement, during monitoring of physiological activity.The subject-specific prediction models of our previous work  are replTemporal consistency criteria are enforced to improve the predictions’ robustness.Three different convolutional neural network architectures are developed and applied to the same ADL recognition problem improving classification accuracy over our standard approach .Advanced Bayesian optimization is exploited for efficient hyper-parameter tuning.The current study builds upon and extends our previous work ,12. The The remainder of this article is structured as follows. In The main device used to record the physiological signals of older people is a wearable solution that takes its origin from a previously developed product of Smartex  , with a The purpose of the classification procedure was to discriminate basic movement activities, such as sit/stand, laying, walking, walking upstairs/downstairs, and transitional state, and store them in the user profile , in ordeThe classification scheme is based on previously reported work  but is aThe preprocessing procedure involves the separation of body acceleration from gravity acceleration, as reported in . SpecifiSubsequently, the features were normalized, in order to avoid skewing the analysis by specific features’ scale. This was achieved by applying the widely used z-score normalization (standardization), that centers the features at zero and scales them to have unit variance. The parameters that used to scale the training features were stored in a data structure to be subsequently used for the standardization of the test features.As discussed previously, performing experiments with sensors of slightly different technology, or with a different sensor placement prohibits the use of a uniform classification model. In particular for the data used in this study, we explored whether any differences occurred between recordings from the two incorporated devices—a wearable wellness system (WWS) and a wearable WBan system (WWBS)—by asking two young volunteers to perform daily activities while simultaneously wearing the two devices used in the study placed at the center and laterally of the chest, respectively. It was revealed that a scaling difference occurred between the two types of measurements. To address this issue, we define a reference space and normalize all recordings with regards to the reference measurements using a baseline correction technique. The baseline was defined as the mean value of time segments with small standard deviation for each axis. Correction was performed by aligning the baseline of each new recording to the reference baseline of the corresponding channel. After baseline correction, the classification model to be used is selected according to the rules identifying errors in sensor’s orientation, as described in the next section.To overcome the challenge of axes rotation, we investigated the possibility to automatically identify mis-oriented device’s data and map them back to a reference space. Although some heuristic rules helped us to recognize mis-orientations in most cases, no robust automatic technique was found to be always successful. Thus, we decided to extract also rotation-invariant features and build a substitute classification model. To achieve this, each triplet of features extracted from the three axes, X, Y and Z, was reduced to only one feature corresponding to the magnitude of the three-dimensional vector. These features are insensitive to the orientation, and thus to the misplacement of sensors.Although the rotation-invariant features are preferable, in the case of inconsistent data they are expected to have lower predictive value, because some activities, such as walking, are strongly related to a particular axis. Therefore, it would not be the best practice to replace the detailed model by a less complex one overall. Accordingly, we had to deal with the problem of automatically recognizing whether the wearable device had been misplaced or not, and accordingly select the most appropriate model. The problem was addressed by learning the distribution of measurements in the reference space (correct orientation) defined by the training set. A two-step heuristic approach was introduced for that purpose. The first rule determines whether the vertical axis (axis X), which is the most prominent axis due to gravity, coincides with the one used during training. This was achieved by outlier detection assuming that the largest amount of measurements (80th percentile) of test and reference data should be in the same range. If so, then the axis-dependent model is selected for classification, otherwise the second rule is examined. θ), the axis-dependent model is selected, otherwise classification is based on the axis-invariant scheme. The threshold θ was determined by examining the normal variation, i.e., it was equal to the maximum within-training-subjects’ distance. The chosen distance metric was a 3-D version of the Kolmogorov–Smirnov distance [The 2nd rule is used to examine more thoroughly cases in which the signal has a shifted baseline value, and therefore might fall outside the predefined range. The signature for the recording is based in this case on the overall distribution of measurements along all three axes and specifically on the 3D histogram of raw values . Outlier detection is performed by computing the pairwise distance between the test signal and each of the reference signals (from the training subjects). If the minimum distance is smaller than a threshold (distance . rbf) kernel from the LIBSVM library [C and gamma of the rbf kernel was performed, in order to achieve optimum cross-validation accuracy on the training set. To deal with imbalanced classes (since the number of samples that corresponded to each class was different), we used weights, inverse proportional to the class size, in the classification function.The normalized feature vectors of the training samples were used to train a classification model that can be later used to classify new instances. A SVM classifier with radial basis function  than deep networks, due to their limited dependencies, small computational cost and reduced requirements of large numbers of annotated data for training. However, deeper architectures offer prospective of further exploitation when databases get richer and also provide advantages of seamless analysis avoiding the tedious step of dedicated feature extraction, which if proven successful, accelerates the deployment phase.We investigated three convolutional neural networks that are optimized and applied for discriminating the basic movement patterns of older people. The CNNs take as input overlapping time windows of the multi-channel recordings. Depending on the dimensionality of the convolutional filter and the multi-dimensional signal representation, various architectures are defined that process the multi-channel information in a different way, thereby affecting the networks’ performance. However, the common aspect in all three architectures is the extraction of translation-invariant patterns by applying the processing units in CNN along the temporal dimension and by sharing the units among multiple sensors . The timwidth, height and depth  layer, which affects the number of computed feature maps, as detailed below.CNN1: 1D convolution is performed on the input data along the temporal dimension only, with a convolutional kernel of size CNN2: 2D convolution is performed with a CNN3: Following the idea of Jiang and Yin [ and Yin , we creaThe three deep network architectures are relatively shallow since the available training samples were not enough to train deep architectures efficiently without jeopardizing their generalization ability. The layers of a CNN have neurons arranged in three dimensions, that are denoted as The three CNNs are illustrated in The exploration and choice of the networks hyperparameters was aided by the Bayesian hyperparameter optimization platform SigOpt . BayesiaThe CNNs were implemented in TensorFlow  using KeStanding for 1 min;Sitting for 1 min;Walking for 1 min;Walking upstairs for 30 s;Walking downstairs for 30 s;Laying for 30 s.The recordings were obtained from older people age: 70–92 years) who participated in the FrailSafe project [2 years wAll participants performed all activities except of upstairs and downstairs walking which was only performed if stairs were accessible by the subjects in their residence. Times were recorded by the medical instructors while the participants performed the ADL protocol. A prediction model was trained using part of the annotated data and assessed on the left-out data. Four-fold cross-validation on the training set was used to optimize the parameters for both classification models. The recordings of the subjects selected for training had the same orientation, which was used as reference space . An example of the raw acceleration signals during different activities is illustrated in sitting and standing were merged into one class, as well as walking upstairs and walking downstairs. This was performed based on previously reported work that suggests that these classes are not easily separable [sitting/standing, laying, walking, walking upstairs/downstairs, transition state.Although six activity classes were initially defined, the classes eparable . AdditioThe classification framework was evaluated on accelerometer signals of 20 older individuals , categorized according to Fried’s criteria  in 10 noA grid search with internal four-fold cross-validation was performed on the training set to optimize the SVM hyper-parameters  percentage of correctly classified validation instances, was 89.46%. The rotation-invariant model on the same subjects obtained a smaller cross-validation accuracy (75.56%), as expected. Since the recordings used for training did not show any visible rotation errors, the removal of orientation cues, critical in the case of sensor inconsistencies, led to information loss and, therefore, to less-sensitive solutions. Based on this observation, the necessity of a rule to automatically identify the orientation and accordingly select the most appropriate model, became evident.Furthermore, in order to obtain a final prediction model, we used all 8 training subjects to retrain the SVM classifier using the previously selected features. Τhe overall accuracy of the end-to-end pipeline was assessed on the 12 test subjects (not used during training) coming from different countries and whose recordings were acquired by wearing any one of the two devices, with the sensors placed correctly or rotated, and by letting the algorithm decide which model to use. The average accuracy was 71.8%. The accuracy drop might be either due to failure to recognize the orientation of sensors and thereby to select the most suitable model, or it might be due to limitations of the classification mechanism itself.In the following, in order to better appreciate the effect of the different steps in the pipeline, we first selected 8 (out of the 12) test subjects wearing the latest of the sensor devices (WWBS) and whose recordings were acquired by placing the device according to the reference orientation. These subjects were from different clinical centers than the ones used in the training set. The classification accuracy of this independent test set reached 81.7%. It is evident that the effect of orientation errors is quite significant since it causes an accuracy drop of ~10%. The mean confusion matrix is illustrated in It appears that sit/stand is the most easily predictable activity by the model, since it is correctly classified in almost all cases. On the opposite the activity of walking on stairs (up/down) was completely misclassified as walking forward/backward, which might be explained by the very small size of walking up/down samples (one subject was available for training and one for testing). It might also indicate high similarity of walking patterns of the elderly, on the floor or in stairs, both characterized by slow speed and possibly instability. Class walking has some false negatives identified mostly as transition between activities, which is not surprising since a transition state is not a class with a standard pattern. Similarly, the class transition is intermixed with all other classes. This might be also attributed to imperfect annotations of the transition samples, extracted within constant time windows (located at the beginning of each activity) without any visual inspection or individualized correction. Furthermore, in order to assess the effect of using an alternative, rotation-invariant model, we used the recordings of the 2 test subjects using the WWBS device with orientation errors during acquisition, and also manually induced rotation in several ways on 6 other test subjects that were also using the WWBS. The mean classification accuracy for these 8 subjects was 70.31%. To better interpret the rotation-invariant models’ performance, we compare the classification accuracy of these inconsistent recordings when applying the orientation sensitive model and the current surrogate model. The results are depicted in Finally, regarding the evaluation of the baseline correction technique for data acquired by different devices, we used recordings acquired with the WWS device from 4 participants  and assessed the classification. The mean accuracy was initially 37.5% with the orientation-sensitive model (trained with data from the newer device), while after performing baseline correction it was increased to 61.7%. The results highlight the importance of baseline correction as data standardization step.The deep learning approach aimed to examine an alternative way of ADL classification that circumvents the need for dedicated feature extraction. Since the scope here was not to test the requirement of appropriate data manipulation techniques handling inconsistencies, we excluded data from the outdated device (WWS) and manually rotated the axes of the sensors whenever a misplacement was identified by visual inspection. Moreover, we used more data that were not available by the time of the development of the standard approach constructing a dataset of 23 subjects in total. We used 16 of them for training, in order to approximate the deep learning requirements of large number of annotations, and 7 subjects for testing. The recordings in both groups (training and testing) included recordings from different clinical centers in order to capture the data variability. The distribution of the different class labels is illustrated in The hyper-parameters were tuned by performing 4-fold stratified cross-validation on the training set using Bayesian optimization. The obtained values and their contribution in the classification performance are shown in A direct comparison with other studies is not feasible due to differences in the experimental setup including the type of activity, as well as the use of different classification performance metrics. Nevertheless, the different approaches reported in this paper are compared in Other works report higher accuracy than the current work, but there can be many reasons for this. First of all, most studies use data from younger participants, selected to be in good physical health. This results to a more homogeneous group and allows the collection of a larger number of samples. Since our work is targeted to the ageing population, functional status and age variability are confounding factors when building a monitoring system for older people. Second, we use a sensor in a single location (sternum), whereas the combination of more sensors could affect the classification performance, especially if located in different parts of the body. Third, the data of this study were acquired in each participant’s house using a prototype research platform (FrailSafe system) based on different versions of sensors and with possible measurement errors, whereas most of the studies report accuracies in controlled, simulated or laboratory settings with consistent data. Appropriate processing steps, such as orientation recognition and classifier adaptation, were introduced to mitigate the effects of misplacement, mis-orientation and device-related variations in the data, reducing the measurement bias and automating the analysis within this end-to-end platform. Regarding the problem of increased data variation which is present in our dataset, multiple techniques have been proposed in the literature to address this issue. More specifically, transfer-learning methods can be used in order to reduce the requirement of a large number of labeled data, as reported in . The autFinally and more importantly, our methods were assessed by cross-validation on the subjects, i.e., the model was trained using measurements from subjects not used during testing. This gives us an estimate on the method’s accuracy when applied on recordings of new subjects. By contrast, other methods either do not report cross-validation results ,16,17, oIn this paper we presented a system recognizing basic physical activities from wearable sensors, with respect to challenges arising from device-generated or human-related parameters. Data from older participants with different levels of frailty and functional conditions were used to train and assess the end-to-end modeling framework developed to address those challenges. Classification was performed by standard machine learning, as well as deep learning techniques, demonstrating a slight advantage of the latter. Overall promising results support the use of the proposed activity recognition scheme for unobtrusive monitoring of older people. Future work includes the investigation of data augmentation and transfer learning techniques to allow the exploitation of available databases from younger people, and thereby to improve the performance of the deep networks.


To evaluate the efficacy and safety of three treatment approaches to applying Botulinum Toxin Type A (BoNTA) for crow's Feet. in vivo study. They were randomly divided into three groups, including the local intramuscular, intradermal microdroplet injection, and nanomicroneedle delivered with BoNTA therapy group. After one session, evaluations were done at the time points of weeks one, four, and twelve after the treatment. The assessments included subjective satisfaction, blinded clinical assessment, and the biophysical parameters . Thirty female subjects with moderate-to-severe crow's feet were included in this comparative For dynamic wrinkles, intramuscular injection and intradermal microdroplets injection were more effective than nanomicroneedles. For static wrinkles, nanomicroneedles and intradermal microdroplets injection were more effective. However, the intramuscular injection had no significant effect on static wrinkles. At one week and four weeks after the treatment, the skin elasticity, collagen content, and hydration of nanomicroneedle group and intradermal microdroplet group increased more significantly than those of the intramuscular injection group; at twelve weeks after the treatment, the skin elasticity, collagen content, and hydration of intradermal microdroplet group were higher than those of other two groups. However we observed no statistically significant difference in sebum content between the three groups before and after the treatment. BoNTA delivered through nanomicroneedles and intradermal microdroplets injection can effectively treat crow's feet. This trial is registered with [2016]KY018-01, registered 16 Feb 2016. The periorbital region is an area most vulnerable to aging, usually attacked by crow's feet or lateral canthal lines. Rhytides are caused by the gradual loss and disorganization of underlying collagen and elastin fibers that support the skin . RepeateBotulinum Toxin Type A (BoNTA) is a neurotoxin protein secreted by clostridium botulinum, which is an anaerobic, Gram-positive bacterium. It blocks the transmission of nerve impulses to the targeted muscle by selectively preventing the release of the neurotransmitter acetylcholine at the neuromuscular junction, a process that can temporarily prevent muscle contraction . The abi in vivo study to compare the efficacy of three approaches to treat crow's feet: the local intramuscular, intradermal microdroplet injection, and nanomicroneedle delivered with BoNTA therapy.An assumption is that large-size molecules hamper the penetration through the skin barrier. Interestingly, a delivery method using a roller studded with fine needles, termed as microneedle therapy, has been invented and used successfully in improving scars and wrinkles and rejuvenating faces through collagen induction . In addiA total of 30 females  with moderate-to-severe crow's feet wrinkles were enrolled. All subjects provided written informed consent, and this study was approved by the Institutional Review Board of the Affiliated Changzhou No. 2 People's Hospital of Nanjing Medical University. All subjects' conditions were classified into grade 3 and grade 4 according to the Global Assessment of Lateral Canthal Lines scale . The first injection was made 1.5 cm lateral to the outer canthus (dose of 3U), and the other two injection points (2U per injection) were made 1 cm above and below the primary injection point [on point .Intradermal Microdroplet Injection Group. Ten subjects received intradermal microdroplets of BoNTA into seven injection points in the crow's feet area on both sides of the face . The intradermal microdroplet technique targeted the outermost muscle fibers in the dermis. To achieve this, a needle was advanced slowly with a bevel parallel to the skin and the plunger was pressed until a small bleb appeared from the skin.Nanomicroneedle Therapy Group. The nanomicroneedle consists of a rechargeable hand-piece with disposable needles driven by a motor. The nanomicroneedles are composed of an array of nanochips attached to the plastic crystal head .Specifications for 0.25 mm nanotips  were as The device was applied three times back and forth, and then 1.4 mL of BoNTA solution  was immediately topically applied on each side for 20 min.Efficacy was assessed by both investigators and subjects. The primary endpoint was an improvement in physician-rated crow's feet severity both at rest and maximum expression. The outcome was assessed based on the subjective satisfaction, scores calculated by blinded investigators (using standardized photography), and biophysical results. The assessment was conducted at the time point of baseline, one, four, and twelve weeks after the treatment.Each subject completed a self-assessment questionnaire and rated their condition with a score from 0 (aggravated) to 4 (much improved). After the treatment, all the subjects graded their intraprocedural pain on a 10 cm visual analog scale (VAS), with the end points designated as 0 (no pain) and 10 (the intolerable pain). The investigator subjectively graded the edema severity after treatment on a scale from 0 to 4 . The duration of erythema and crusting was investigated through interviews. Any adverse events and complications were recorded during the treatment and the follow-up.Standardized photographs were obtained at baseline and weeks one, four, and twelve after the treatment. Standardized close-up photographs were taken using a high-resolution digital camera . Two dermatologists who were blind to the treatment group evaluated the serial photographs independently and performed clinical assessments on the crow's feet using the IGA-LCL scale.The parameters  were measured using CBS-Medicai skin analysis system . The CBS skin analysis system is based on the principle of spectrum color gradation recognition and texture scanning, supplemented by statistical foundations, to obtain data and results of the corresponding test items . All meaP < 0.05 was considered statistically significant.The results were analyzed with the single factor variance analysis using SPSS 15.0 software . When the crow's feet severity was assessed in a dynamic state, all the three groups showed a statistically significant difference from baseline to one, four, or twelve weeks after the treatment. However, the nanomicroneedle group showed lower efficacy than the intramuscular injection group and the intradermal microdroplet injection group –7.When the crow's feet severity was assessed in a static condition, a statistically significant difference was detected from baseline to one, four, or twelve weeks after the treatment on both the nanomicroneedle group and the intradermal microdroplet group; however, no statistically significant difference was detected on the intramuscular injection group before and after treatment. The difference at weeks one, four, and twelve was not statistically significant between the nanomicroneedle group and the intradermal microdroplet group –7.The subjective satisfaction at week one, four, or twelve after the treatment showed no statistically significant difference between the three groups .The pain scores of the three groups showed statistically significant differences. The nanomicroneedle group showed lower pain score than the intramuscular injections group and the intradermal microdroplet group.The intramuscular injections group showed severer erythema and edema than the nanomicroneedle group and intradermal microdroplet group .Throughout the study, no serious or persistent adverse effects occurred and no one withdrew from the study because of adverse events. No potential infection or no brow ptosis was observed.One week after the treatment, the skin elasticity of the three groups increased, but showed no significant between-group difference. Four weeks after the treatment, the skin elasticity in the nanomicroneedle group and the intradermal microdroplet group increased and became higher than that in the intramuscular injection group; then twelve weeks after the treatment, the skin elasticity in the intradermal microdroplet group was higher than that in the intramuscular injection group, and that in the intramuscular injection group was higher than that in the nanomicroneedle group; however, we observed that the skin elasticity of all three groups insistently increased at weeks one, four, and twelve after the treatment .Skin hydration is an index to evaluate the function of skin barrier. As shown in It is well known that the collagen in the dermis is mainly composed of collagen I(80%~85%) and collagen III(10%~15%), which are responsible for the elasticity and intensity of skin and for producing a healthier skin texture . The CBSSebum is a sticky liquid mixture of nonpolar lipids. The composition of sebum is relatively constant, and its changes may entail some skin diseases. Production of sebum depends on individual characteristics, environmental factors, and the density, location, and activity of sebaceous glands.No statistically significant difference was detected from the baseline to week one, four, or twelve after the treatment among the three groups .In our study, when treating dynamic wrinkles, intramuscular injection and intradermal microdroplets group showed more significant effectiveness than the nanomicroneedle group. The underlying mechanism is that nanomicroneedle can create microscopic columns in the epidermis and dermis with the expectation that this would permit molecules of BoNTA to reach the superficial orbicularis oculi in the lower dermis. And the transcutaneous penetration doses of BoNTA by nanomicroneedle may be partially reduced.In our study, we found that half of the patients saw the improved condition of crow's feet wrinkles within twenty-four hours after treatment, with a median onset time of twelve to twenty hours, and all the patients reported the improvement six days after the treatment. These results are consistent with those of the previous studies , 20. In When treating static wrinkles, nanomicroneedle therapy and intradermal microdroplets technique showed effectiveness at weeks one, four, and twelve after the treatment. However, the intramuscular injection had no significant effect on static wrinkles. We also observed that the nanomicroneedle therapy and the intradermal microdroplet technique can improve skin texture in the treatment area. The performance of the skin in the treatment area is more delicate and smooth. Besides that, we observed that the skin elasticity and collagen I content of all three groups increased at weeks one, four, and twelve after the treatment. Four weeks after the treatment, the skin elasticity and collagen I content of the nanomicroneedle group and the intradermal microdroplet group increased more significantly than the intramuscular injection group; twelve weeks after the treatment, the intradermal microdroplet group had higher elasticity and collagen I content than the intramuscular injection group, and that in the intramuscular injection group was higher than that in the nanomicroneedle group. in vivo study further confirmed the antiphotoaging role of BoNTA by improving skin collagen I production and elasticity, especially in nanomicroneedle group and intradermal microdroplet group. However, we found that this effect lasted less than three months in the nanomicroneedle group, shorter than that in the intradermal microdroplet group.Our preliminary study  demonstrSkin hydration is also an index to evaluate the function of skin barrier. Skin hydration of the three groups showed a statistically significant increase at week one, four, or twelve after the treatment. One week and four weeks after the treatment, the skin hydration in the nanomicroneedle group and the intradermal microdroplet group increased more significantly than that in the intramuscular injection group; then twelve weeks after the treatment, the skin hydration in the intradermal microdroplet group was higher than that in the intramuscular injection group and nanomicroneedle group. It can be inferred that better skin barrier function can be achieved by BoNTA topical treatment, especially in nanomicroneedle group and the intradermal microdroplet group.Sebum is a sticky liquid mixture of nonpolar lipids, with a relatively stable composition. The production of sebum is dynamically affected by individual characteristics and environmental factors, like density, location, and activity of sebaceous glands mainly regulated by androgen. It has been reported –25 that As to erythema and edema being complicated with the treatment, in our study we found that the Botulinum Toxin A-induced swelling in all subjects subsided within 48-72 hours. the intramuscular injection group showed a higher incidence than the nanomicroneedle group and the intradermal microdroplet group. The pain score of the nanomicroneedle group was lower than that of the intramuscular injection group and intradermal microdroplet group.Comparing with intramuscular injection technique, the intradermal microdroplet technique prevents diffusion of BoNTA into deeper muscles, which can lead to a “frozen” appearance. It is proposed that the microdroplet injections can smooth and tighten the skin by inducing bulk atrophy of the sweat and sebaceous glands, weakening the superficial muscle fibers in the skin, and thereafter reducing the pulling and tethering force of the facial muscles that form fine lines and wrinkles. We found that microdroplet injections technique could reduce dynamic wrinkles and static wrinkles and improve skin barrier function. Its treatment effects usually last for 3 to 4 months.BoNTA, a type of large molecule, has a low cutaneous bioavailability—simply applying its solution to intact skin is not effective . CombinaOne major limitation of this study is its small sample size. Larger sample randomized controlled clinical trials are needed to verify the results of this study. To the best of our knowledge, our study is the first to report the efficacy of the nanomicroneedle therapy delivered with BoNTA in treating crow's feet and compare it with conventional injections. We conclude that BoNTA delivered through nanomicroneedle and intradermal microdroplets injection can effectively treat crow's feet. Its exact mechanism of action and how to optimize its clinical effects are still worth further researches.


Registry-based randomized controlled trials (RCTs) are presumed to include a less-selected patient population and thus to have enhanced generalizability compared to conventional RCTs. However, this view disregards the levels of patient selection in registry-based RCTs: the registry selection level and the trial selection level. At both levels, systematic selection can occur and generalizability can be diminished. Nevertheless, using a registry as a basis for recruitment, randomization, and data collection results in an advantage: the trial selection takes place within the registry framework, where baseline characteristics of non-enrolled patients are automatically documented as well. By comparing the baseline variables of the enrolled and non-enrolled patients, the trial selection can always be investigated, which gives a sound basis for discussing the generalizability to the registry population. Hypothesized to be “the next disruptive technology in clinical research”  and “a nOn the contrary, Laurer and D’Agostino ask the fundamental questions, whether representativeness can be assured, given that even within a registry there may be a systematic difference between patients who are randomized and those who are not. This ambiguity indicates that the situation is more complicated and calls for a closer look at the alleged generalizability of rRCTs.In this commentary, we concentrate on different levels of patient selection in rRCTs to achieve a better understanding of the possibilities, limitations, and the generalizability of rRCTs results. Other relevant aspects of the rRCT design such as the choice or endpoints, data quality, or practical implementation are not covered in this article.While internal validity in RCTs is strong, the lack of external validity or a profound assessment of the generalizability is the most frequent critic of RCTs , 6–8. RoProblems in the assessment of the external validity of RCTs are hoped to be solved or at least diminished in rRCTs by recruiting patients from an existing patient registry , 11, 12.i)the registry selection: the registry population is selected from the rRCT target population;ii)the trial selection: the trial population is selected from the registry population.Figure Li et al.  and JameFor analyzing the patient selection of conventional RCTs, it is necessary to evaluate information on the screened patients and the target population , 20. TheIt is important to note that comparing enrolled and non-enrolled patients regarding any outcome variable, which is influenced by treatment decisions (randomized versus non-randomized), is potentially biased. The first differentiating factor between the trial patients and the registry-only patients is the fact that only the former have been included in the trial and likely there is a clinical rationale. The second differentiating factor is the treatment allocation mechanism applied: in the enrolled group, treatment is allocated randomly, while in the non-enrolled group the treating physician decides about the treatment. Thus, enrolled and non-enrolled patients differ structurally regarding at least two factors and a difference between these groups observed after treatment allocation cannot be causally attributed to only one of the factors. Consequently, no variable measured after the treatment decision  should be used for the assessment of patient selection and generalizability. Only a comparison regarding baseline characteristics allows an unbiased assessment of structural differences between the populations to evaluate the patient selection of the rRCT from the utilized registry.The rRCT design was applied in the Thrombus Aspiration during ST-segment Elevation myocardial infarction (TASTE) trial. This trial is based on the SCAAR platform, which holds data on consecutive patients from all 29 Swedish, one Icelandic, and one Danish coronary intervention centers . TASTE wp values, because the p values can indicate structural differences for clinical non-relevant effects due to the large sample sizes of both populations. The population comparison regarding baseline characteristics (see subsection 'As explained in section 'The investigators’ evaluation of the generalizability of the findings in the randomized trial to the overall registry population was inconsistent. In an early publication , the invRRCTs are hoped to be less selective and thus to have enhanced generalizability compared to conventional RCTs not using registries for recruitment and data collection. The investigation of this alleged advantage through the analysis of selection mechanisms in rRCTs shows that there are two levels of patient selection. Both levels, the registry selection and the trial selection, impact the generalizability. Therefore, rRCTs are neither automatically less selective nor have a higher external validity per se. However, by using a registry as a platform and documenting baseline variables of non-enrolled patients, the rRCT design provides a sound basis for researchers to explicitly investigate the trial selection and thus potential limitations of the generalizability of trial findings to the registry population. This addresses the plea by Rothwell to increase considerations of external validity in trial design and reporting . Such inTo conclude, the general advantage of rRCTs in comparison to conventional RCTs is not a less-selected patient population, which is often an illusion, but rather a more thorough ground to investigate the trial selection. The analysis of the trial selection is possible for every rRCT and should be conducted and reported by default to assess the generalizability to the registry population.


The ubiquity of sensor-rich smartphones provides opportunities for a low-cost method to track indoor pedestrians. In this situation, pedestrian dead reckoning (PDR) is a widely used technology; however, its cumulative error seriously affects its accuracy. This paper presents a method of combining infrastructure-free indoor acoustic self-positioning with PDR self-positioning, which verifies the rationality of PDR results through the acoustic constraint between a sound source and its image sources. We further determine the first-order echo delay measurements, thus obtaining the mobile user position. We verify that the proposed method can achieve a continuous self-positioning median error of 0.19 m, and the error probability below 0.12 m is 54.46%, which indicates its ability to eliminate PDR error, as well as its adaptability to environmental disturbances. The increasing number of sensor-rich smartphones has raised interest in using their sensors for indoor localization applications, such as indoor navigation , locatioBased on specific technology, it is possible to categorize methodologies for smartphone-based indoor pedestrian self-positioning systems into two distinct groups: 1) infrastructure-based systems that use auxiliary equipment or a cooperation between nodes to realize target tracking  infrastr,6,7,8,9,To reduce the influences of noise on the source tracking, the motion and observation models of the moving source as well as the probability distribution model of the errors  must be To alleviate the aforementioned problems, this study provides an acoustic constraint algorithm to verify the rationality of the PDR results, which reduces the cumulative errors by using the geometric relationship between the sound source and its image sources.The rest of this paper is organized as follows. We assume that a sounding smartphone is always carried by the indoor pedestrian. The pedestrian moves autonomously inside a room. At every step, the loudspeaker of the smartphone produces a chirp pulse, the microphone of this smartphone registers the echoes, and the inertial sensors record the accelerometer and gyroscope readings. We define the room to be a The first step is to compute the image sound sources, denoted as  the AIM ,29. Then the AIM  was adopIM [Ltra  and the [Ltra [θ ) and the Mensing , and a bIn the AIM, the reflections from the walls are replaced with signals produced by image sound sources across the corresponding walls. For a first-order echo and the u et al. . For exaDenote As the sound propagation speed When the loudspeaker operties , a chirpoperties  with simoperties  was chosoperties , which pThe chirp impulse, emitted from Let the time-domain received signals be beration . In our Given a fixed reflecting surface with a fixed orientation and a sound source point, the expression for the position of the image point can obtained with Equation (1). If also given the boundary values of the room size, this position can be explicitly expressed by Since the smartphone is carried by the moving pedestrian, the PDR information—which is regarded as the distance moved  of the reflected sound is greater than 10 dB of the SPL of the direct sound. Thus:Thus, If some of them  approach, the cost function is:However, the main limitation of the WNLS is that, in order to maintain optimal robustness, its learning rate parameters are usually set to small positives, resulting in a slower convergence rate. Thus, the application of the Levenberg–Marquardt algorithm (LMA) to WNLS could accelerate the convergence while ensuring robustness, and satisfy real-time positioning requirements.We validated the proposed approach with the data collected from the corridor of the fifth floor of the Jinji Campus Library in GUET, GuiLin, Guangxi Zhuang Autonomous Region, China. The cloister size was The data collection tool used in this experiment was a Huawei Rongyao 7 smartphone installed with a chirp application developed by our team and already authorized by China National Intellectual Property Administration, which was used to emit and store the chirp sound signal. The chirp sample frequency was set as We had the loudspeaker of the smartphone facing the nearest wall, opened the chirp application, and then walked normally from the starting point (green dot) at To obtain the adaptive step length Using the acceleration measurements, step detection and step length estimation can be accomplished through the walking frequency When walking along the corridor, the changing trends of distances from the sound source to the four walls were directly reflected in the values of Firstly, from step Next, the user moved from the west to the east; that is, from the first corner , there were some fluctuations in the corner areas in the tracking trajectory, which is consistent with the change parts marked with the black dotted rectangles in (1)When (2)As shown in (3)The box figure c detailsThe errors presented in We proposed a sensor-rich smartphone-based indoor pedestrian self-positioning system for continuous position acquisition based on image acoustic source impulse. Along with the processing, an acoustic principle-based constraint algorithm was proposed to update the first-order echo measurements generated from the PDR and ITM methods, increasing the reliability of the final positioning results compared to the PDR method. Additionally, the LMA-WNLS model was adopted to reduce the computational complexity of the continuous self-positioning process, thereby increasing time efficiency. Despite this, we noticed some limitations of this system. For example, the used smartphone must have an application that can emit and receive chirp sounds because it is impossible for ordinary smartphones to play chirp sound signals. The arbitrariness of pedestrian motion during walking is limited. If the actual trajectory of walking deviates from the dominant direction, heading angle errors are produced, resulting in positioning error.Related future work will mainly focus on the data processing of the heading angle and the separation of the close echo arrivals, to further improve the positioning accuracy and fully port this complete system to a smartphone application.


Multi-sensor data fusion is a feasible technique to achieve accurate and robust results in fault diagnosis of rotating machinery under complex conditions. However, the problem of information losses is always ignored during the fusion process. To solve above problem, an ensemble convolutional neural network model is proposed for bearing fault diagnosis. The framework of the proposed model contains three convolutional neural network branches: one multi-channel fusion convolutional neural network branch and two 1-D convolutional neural network branches. The former branch extracts the coupling features based on multi-sensor data and the latter two branches extract the inherent features based on single-sensor data, which can collect comprehensive fault information and reduce information losses. Furthermore, the support vector machine ensemble strategy is employed to fuse the results of multiple branches, which can improve the generalization and robustness of the proposed model. The experiments show that the proposed can obtain more effective and robust results than other methods. Rotating machinery is widely used in modern industry. Due to long-time running under complicated conditions such as high speed, heavy load and strong impact, rotating machinery will inevitably have some faults, which can result in enormous losses and serious casualties . TherefoTraditional fault diagnosis methods are mainly based on model analysis or signal processing techniques. The model-based diagnosis methods emphasize the deep understanding of the dynamic characteristics of rotating machinery. Immovilli et al.  conduct Different from traditional diagnosis methods, Intelligent fault diagnosis aims to effectively analyze massive data and automatically provide diagnosis results, which has become a new trend in the field of equipment condition monitoring . GeneralIn recent years, deep learning provides a new research hotspot for analyzing and processing the big data, which has made great achievements in the fields of image, finance, meteorology, and natural language . As a tyAccording to the literature , multi-sRecently, the CNN models based on multi-sensor data have been used for fault diagnosis. Generally, multi-sensor signals are connected into long signals  or arranTo solve the above problems, a novel model named ensemble convolutional neural networks (ECNN) using multi-sensor data is proposed for bearing fault diagnosis in this manuscript. The proposed model, ECNN, can automatically and effectively extract features for classification that gets rid of the dependence on signal processing techniques and diagnosis experience. Besides, a multi-channel fusion convolutional neural network (MCF-CNN) and two 1-D CNNs are designed to construct ensemble model. The coupling features between multi-sensor data are extracted by the MCF-CNN model and the inherent features of the single-sensor data are extracted by 1-D CNN model, overcoming the problem of information losses. Furthermore, to improve the generalization and robustness of the proposed model, the SVM combination strategy is employed to integrate the result of multiple CNN models. The proposed model uses frequency spectrum as the input of CNN, verified based on two typical rolling bearing fault databases.In lth convolutional layer is A and B are the dimensions of the input data. Then the output of the convolutional layer can be calculated as follows:lth convolutional layer, * represents the convolutional operation, f is the Relu activation function. The output CNN is one of the most prevalent deep learning models in recent years. As shown in After several operations of convolution and pooling, deep feature maps are expanded into fully connected layer:ith class, ith output neuron of last fully connected layer.For the output In this manuscript, a novel model, ECNN, is developed for bearing fault diagnosis. The input of the model is the frequency spectrum of the collected signals so that we select the one-dimensional convolutional neural network (1-D CNN) with the one-dimensional filter as the kernel of CNN. It is clearly seen that the framework contains three CNN branches based on two-sensor data. As shown in After the fusion layer, two fully connected layers are used to classify the input signals. The first fully connected layer has hundreds of neurons and the number of neurons in the last fully connected layer corresponds to the categories of classification tasks. The softmax function is used to convert the vector of the last fully connected layer into a probability distribution form.The traditional fault diagnosis methods based on multi-sensor data ignore the coupling information between signals. The CNN fault diagnosis models based on multi-sensor data focuses on the coupling information between the signals and ignores the inherent information of the single-channel sensor. These methods all result in information losses. In addition, the performance of CNN fault diagnosis model is often limited due to single network framework. Ensemble learning is a new technique, which uses multiple individual learners and a certain combination strategy to get better results than each individual learner. Recently, a lot of ensemble learning methods have been applied for machinery fault diagnosis. Thus, ensemble learning of ECNN is constructed for fault diagnosis.To collect more comprehensive fault information, the 1-D CNN model and the MCF-CNN model are used to construct ECNN. Since the databases contain the data from two sensors, two 1-D CNN models and one MCF-CNN model can be obtained. The input of the 1-D CNN is single sensor data, while the input of the MCF-CNN is the data from two sensors which are accepted by the two channel of MCF-CNN respectively.The next step is to design a fusion strategy to combine the results of three CNN branches. Among the common fusion strategies, the majority voting is a widely used fusion method for ease of execution. However, the majority voting treats each classifier equally, resulting in poor performance when there are fewer classifiers. Obviously, it is not appropriate to use this method here because there are only three classifiers. The SVM is a popular machine learning method for classification, regression because of the small structural risk. In addition, the SVM model with kernel function can also learn the non-learning relationship of the input data. Thus, The SVM model with RBF kernel is chosen as the ensemble learning algorithm to fuse the results of three CNN branches.nth samples, define the predicted probability that belong to the ith class of the mth CNN as N represents the number of the train samples. On the training set, we use the deep feature representations nth training sample as the input and the real label n sample on the testing set and For the This manuscript develops a new model called ECNN for bearing fault diagnosis. Step 1: The signal acquisition devices collect data from multiple sensors.Step 2: The collected data are divided into training and testing set and the raw signals are divided into a series of segments. The frequency spectrum of the segment is used as the input of the CNN model.Step 3: The MCF-CNN based on multi-sensor data and two 1-D CNNs based on single-sensor data are designed to construct ECNN based on training set.Step 4: The SVM ensemble algorithm is employed to combine the results of three CNN branches.Step 5: Validate the performance of the ECNN based on the testing set.The proposed model does not require complex mathematical models and does not rely on signal processing techniques and expert experience compared with the traditional fault diagnosis methods.Compared with intelligent diagnosis methods, the proposed model does not require the process of complex feature extraction and feature selection. Besides, the deep learning framework can effectively extract more useful fault information for classification, which can further improve the diagnostic accuracy. Similarly, the new network structure, MCF-CNN, has the better ability to extract features.The MCF-CNN model fuses multi-sensor data at the feature level and ECNN fuses the results of three CNN branches at decision level, effectively overcoming the problem of information losses during the fusion process.Based on the typical model of deep learning, convolutional neural network, the proposed model fuses the multi-sensors data as two level. The advantage of the proposed model is can be summarized as follows:To demonstrate the effectiveness, the proposed model is tested based on two typical rolling bearing databases. The CNN models are created based on the framework of TensorFlow 1.4. The Nesterov Adam algorithm  is used The input of current CNN fault diagnosis models is usually a short segment of raw signals and the CNN models based on multi-sensor data fusion often directly combine segments from multiple sensors to long sample or arrange the segments to a 2-D image as the input of CNN models. Obviously, the length of segment has an impact on the performance of the CNN models. If the length of segment is too long, the splicing or arranging of much time domain data can result in over-fitting and the waste of resources and time. If the length is too short, the model appears to be under-fitting, which cannot learn effective fault features.According to the literature , in the The min-max normalization strategy is used to map each sample into  interval. The equation is as follows:In this subsection, the bearing data for experiment comes from the Bearing Data Center of Case Western Reserve University . The tesAs shown in The excellent performance of the proposed model is proved in comparison with traditional fault diagnosis methods, including SVM, RF and AdaBoost algorithm. The input of traditional methods is the connected frequency spectrum of the two-sensor signals. In addition, a multi-sensor data fusion method based deep convolutional neural network (DCNN)  is also Ten trials are carried out based on the datasets mentioned in the previous subsection. Through the above experiments, two conclusions can be made. On the one hand, the proposed model has higher accuracy than traditional methods. The reason is that the CNN models can automatically learn more effective features from the input data, while traditional methods rely heavily on manual feature extraction. For traditional methods, the selection of sensitive features is generally time-consuming, blind and subjective so that diagnosis results are generally poor. On the other hand, compared with the direct decision of DCNN, three CNN branches of ECNN based on different sensors and different networks can make joint decisions, which not only ensures the high diagnostic accuracy but also has good generalization.The experimental data is from the Chair of Design and Drive Technology at Paderborn University . As showThe test rig can be operated under different operating conditions. Bearings are run at a speed of 1500 r/min with a load torque of 0.1 N·m and a radial force on the bearing of 1000 N. In order to get closer to industrial applications, the dataset in The sample is the raw current data of 256,000 points, which is sliced into 60 subsamples. Each subsample has 5120 points with a shift size of 1024. The frequency spectrum of these subsamples is the input of the CNN models. During the testing, the subsamples from the same raw data are voted for the final result. The detailed situation of healthy, inner race fault and outer race fault bearings is shown in The experiments are divided into two parts. The first part is the comparison among the proposed model and the traditional methods mentioned in In the first part, ten trials are carried out. As shown in In the second part, two current signals are denoted as U and V and define the 1-D CNN based on U and V as CNN_U and CNN_V, respectively. The reasons for the above results can be explained as follows. FCNN can be regarded as ECNN without the branch of MCF-CNN, while the performance of ECNN is better than FCNN. Besides, the accuracy of MCF-CNN is sightly lower than ECNN. Thus, it can be inferred that the excellent performance of ECNN is mainly attributed to MCF-CNN and the CNN_U and CNN_V branches slightly improve the accuracy of ECNN.Recently, the CNN models have developed a variety of complex structures, which make the interpretation of the internal mechanism of the CNN model very difficult. However, with the visualization in the invisible layers, some interesting phenomena can help us understand the mechanism .In this manuscript, linear discriminant analysis (LDA) is used to extract two features for visualization. As shown in The experiments are carried out based on two typical rolling bearing fault databases: CWRU database and Paderborn University database. The collected data of the former is vibration signals and the latter is current signals. For the two groups of experiments, the proposed model has much higher accuracy and smaller standard deviation compared with the traditional methods, which proves that the proposed model is effective and has better generalization. What is more, the results of DCNN on the two databases are quite different, while the results of ECNN are extremely consistent, demonstrating the good robustness of the proposed model.The second part of the experiment in case 2 illustrates that the excellent performance of the proposed model is mainly from the contribution of the MCF-CNN branch and the left two branches play slightly auxiliary roles. According to the visualization analysis, MCF-CNN can extract more effective features for fault diagnosis.In the literature , the deeIn this manuscript, a novel model ECNN using multi-sensor data is developed for bearing fault diagnosis that get rid of the dependence on signal processing techniques and diagnosis experience. The MCF-CNN model fuses multi-sensor data at the feature level and ECNN fuses the results of three CNN branches at decision level, effectively overcoming the problem of information losses during the fusion process. The proposed model is applied on two typical databases, and the results show that the proposed model has higher accuracy and better generalization than traditional intelligent methods.The CNN models using multi-sensor data in the field of bearing fault diagnosis are still in the exploratory stage and there are still many problems to be solved, such as the fusion of “heterogeneous” sensors. In the future, we will explore a more efficient fusion method so that these methods can be applied in the modern industry as soon as possible.


The functionality of three dry electrocardiogram electrode constructions was evaluated by measuring canine heart rate during four different behaviors: Standing, sitting, lying and walking. The testing was repeated (n = 9) in each of the 36 scenarios with three dogs. Two of the electrodes were constructed with spring-loaded test pins while the third electrode was a molded polymer electrode with Ag/AgCl coating. During the measurement, a specifically designed harness was used to attach the electrodes to the dogs. The performance of the electrodes was evaluated and compared in terms of heartbeat detection coverage. The effect on the respective heart rate coverage was studied by computing the heart rate coverage from the measured electrocardiogram signal using a pattern-matching algorithm to extract the R-peaks and further the beat-to-beat heart rate. The results show that the overall coverage ratios regarding the electrodes varied between 45–95% in four different activity modes. The lowest coverage was for lying and walking and the highest was for standing and sitting. Animal computer interaction (ACI) is a new, emerging discipline with significant commercial potential . Its rooIt has been suggested that measurement of heart rate (HR) and heart rate variability (HRV) could reflect a dog’s emotional responses and stress, as well as serve as an indicator of a dog’s welfare ,5. Howev® chest strap, Polar Electro oy, Kempele, Finland) with electrically conductive gel [Due to the above reasons, there has been a growing interest to monitor pet HR and respiratory activity. Electrocardiogram (ECG) electrodes have also been reported to be used with animals . Typicaltive gel  or adhestive gel . These ttive gel . This maIn this study, the main focus was to concentrate on the dry electrodes that could be used with thick and dense haired animals without shaving the fur and also without using electrolyte gels or other conductivity promoters. Our goal was to find out whether these electrodes could be used in maintenance-free canine heart rate monitoring. A literature review revealed that different types of pin electrodes, patches and tape electrodes have been used in animal HR measurements. In this study, the pin type electrode was taken into a more detailed investigation as it is the type of electrode that demonstrated promising results when placed on the hairy regions of a dog . Two neThe measurement system consisted of four electrodes in their respective housings. These were attached to a harness, which the dog wore during the measurements. The data was collected as an ECG potential measurement with a portable device, which along with the ECG data also recorded the accelerometer information. This collected data was later downloaded from the measurement device and further processed offline.Electrodes and the housings in which the electrodes were attached during the measurements were designed and constructed. Additionally, a wearable neoprene harness was designed and fabricated for fixing and attaching the electrodes to the desired locations on the dog’s body. At first, fixtures and fixture molds were designed, CAD modeled and 3D-printed from polylactide (PLA) using an FDM printer . Later, silicone rubber fixtures were cast into the 3D-printed molds.Two different spring-loaded electrodes were newly constructed and evaluated. Additionally, polymer electrodes with Ag/AgCl coating, prepared by the University of Ilmenau, were studied . The sprTwo different electrode housings were fabricated. The first housing was constructed of a printed PLA plate, shown in The second type of housing was cast from silicone rubber  into a custom designed 3D-printed two-part casting mold. This housing was used with both polymer and gold electrodes. After the initial casting, the electrodes were placed in to the cast silicone fixtures and a second casting was performed in order to make an integral electrode silicone rubber structure. This is illustrated in Two different harnesses were used during the measurement to ensure a proper location of the electrodes and as equal surface pressure conditions as possible for all electrodes. Both harnesses were constructed of textile-coated elastic neoprene rubber. The first version was made of 2 mm neoprene with a velcro-coated outside layer. The second version was made of 3 mm thick neoprene and in addition to the velcro layer, it had polyester lining. The harnesses consisted of separately adjustable parts , which were attached together with velcro fastener ensuring that the harnesses were ergonomic and comfortable for dogs. The electrodes were integrated to the lower part of the belt around the chest, two electrodes near to both armpits. The harnesses with the integrated electrodes and monitoring device were put on the dog as shown in A custom made physiological monitoring device named SpiritCor9D was used to record ECG data with 250 Hz sampling frequency. In particular, the measurement device recorded three channels of ECG, impedance pneumography signal, and 3D acceleration and gyroscope data. The data was stored on the internal memory of the monitoring device and later extracted for the analysis performed on a PC computer. Unshielded electrode wires were used to connect the measurement electronics with the electrodes.The animal experiments were conducted at the University of Helsinki. The procedures were approved by the Ethical Committee for the Use of Animals in experiments at the University of Helsinki (statement 2/2018). Three dogs were invited to participate in the test measurements. The dogs were two female Beauce Shepherds  and one male Hovawart .The testing was organized in 36 different measurement scenarios . Each of these measurement scenarios were repeated nine times (n = 9) in three testing sessions to obtain more reliable measurement results. One testing session consisted of three cycles of a sequence of behaviors, namely, standing 60 s; sitting 60 s; lying 60 s; and walking 60 s, with a settling time of approximately 10 s. The blocks were performed in succession forming a total duration of 12 min per session. The four different activity or posture modes were: standing, sitting, lying and walking. During the walking mode, the handler moved at walking speed while the dog followed unleashed keeping the same speed. The dogs were allowed to trot and pace to keep their speed. Walking mode included multiple sharp turns. Before each testing session, the dog was freely moving for one minute, to get accustomed to the harness. The measurement harness was taken off and put back on between the measurement sessions.The three electrode types were compared using ECG R-peak coverage ratios. R-peak was defined as the ratio between a successfully recovered R-peak time divided by the total test measurement time. Thus, a fully recovered HR data would yield a coverage ratio of 1.R-peak detection is a standard operation in HR monitoring devices. There is a large variety of methods developed and published for R-peak detection from the measured EGC data, some of them were developed especially for integration on a low-performance microcontroller ,16,17. ACoverage ratios were computed for each activity mode and each dog. The measured raw ECG and filtered ECG data with different respective activity modes, is presented in The processing was carried out by first applying a fourth order 2–30 Hz Butterworth forward-backward band pass filtering. Next, a pattern finder procedure was applied to the filtered data to find individual heartbeats in the ECG and to obtain the HR. In this procedure, first a typical template  of ECG pAfter the template assignment, the data was split into windows to perform adaptive amplitude scaling for the data. This processing phase started by defining an observation window length. The length was selected in such a way that it was longer than the maximum normal R-R-interval to ensure that in this window at least one R-peak would be present. In this particular case, the window length was selected to be 2 s to ensure the presence of the potential R-peak. Next, the data in the observation window was processed such that the sample with a maximum value in the data window was scaled to 1 and all other values in the vector were scaled linearly with the same coefficient. The actual pattern matching was then performed by computing the sum of the squared difference of the template and the observation window when the template was moved along the observation window vector. This windowing procedure was repeated to process the whole data.In order to distinguish the matched pattern, a threshold of the distances which identifies a matched pattern was defined. Below this threshold level, the two vectors were interpreted to match and thus an R-peak was detected. The threshold level was visually observed from the output of the pattern match distance calculation. Care was taken to select the practical threshold level empirically such that false R-peak detections were minimized. In this particular study, an empirically defined threshold value of 0.3 was used. This yielded to less than 1% of false R-peak detections when the result was visually inspected against the filtered ECG data.Thresholding the pattern matcher output data gives the location of the pattern match. This can also be interpreted as the location of the R-peak or the local minimum of the template data distance. This data was then used to compute the R-R interval signal values, which is the time elapsed between two detected R-peaks. A graph of the raw pattern match data and the respective RR-interval tachogram along with the different respective activity modes, is presented in Electrode coverage ratio averages (n = 27/activity) including all dogs and all electrodes varied between 0.45 and 0.95 with respect to the different activity. The standard deviation of the RR-interval coverage varied between 0.08 and 0.36. Furthermore, the median values varied between 0.44 and 0.99. Finally, the coverage ratio ranges varied between and 0.35 and 0.98. These average, standard deviation, median and range values are listed in The heart rate/heartbeat detection coverages for each electrode type and activity obtained with all three dogs and in all test repetitions (n = 27/activity) are presented as boxplots in p-values for the null hypothesis test are listed in the Statistically significant differences in electrodes were investigated by performing a Mann-Whitney test on the coverage data. The data was organized such that the electrode data was accumulated (n = 27 for each activity mode) to represent each electrode in each activity mode. The computed p < 0.05 and these cases are marked with bold typeface in the The analysis shows that there are statistically significant differences in the electrode coverage performance especially with standing and sitting activity modes. In this analysis null hypothesis is rejected when In this study, we evaluated the performance of three different dry electrodes for maintenance-free canine heart rate monitoring. The results showed that the electrodes performed differently in certain activity modes when evaluated in terms of heart rate coverage ratios. The highest coverage ratios (over 90%) were achieved with relatively stationary postures when the dogs were standing or sitting; while the lying and walking modes resulted in coverage ratios of 75% and 49% on average, respectively. The results are in line with those obtained by Brugarolas .There are several factors that might have affected the performance of the electrodes. Likely, the variation in the coverage ratios may be due to the thickness and quality of the animal hair as well as the flexibility and elasticity of the electrodes. Additionally, the actual electrode length, pin density (pin-to-pin distance), and effective contact pressure may have contributed to the performance by affecting the penetration of the electrode through the hair. Contact pressure was shown prior to considerably affect the contact stability and the signal quality of dry-contact electrodes.Beyond the different activity modes, there are other interfering factors due to the physiological properties of the tested dog breed. Further, it appears that there are also anatomical factors like the structure of the thorax (or even torso) area, which can have an effect on how the electrodes are able to retain the necessary skin contact. The dog thorax shape changes in different postures, affecting the position and the tension of the electrodes in the harness. The effect of the thorax shape change on the coverage ratio is probably the best observed in lying position, where the thorax flattens when compared to, for example, standing position. Future long-term studies are needed to investigate and improve the dry electrode design to adapt with canine thorax shape changes.The limitations of this study are related to the relatively small sample size of the dogs and dog breeds. Therefore, even though the results certainly reveal logical behavior of the electrode harness combination in the different activity modes, it is possible that the results of this study are not generalizable as such. Even though the data was carefully visually inspected to ensure the reliability of the R-peak detection, some part of the uncertainty in the measurement results is also possible because of the lack of a validated ECG measurement reference . However, it is estimated that this does not contribute significantly to the measurement results reliability.This is an ongoing work and we presented the initial results of the first trials of canine ECG electrode measurements, showing that the proposed electrodes can be used to record ECG with sufficient quality for further processing into reliable HR data. When thinking of the mass production of the electrodes studied, it can be concluded that all the tested electrode structures are relatively simple constructions and conceptually mass producible. Whereas, the spring-loaded electrodes are a multipart construction, the polymer electrode could be a single shot injection molded part. This makes the polymer electrodes potentially less expensive to produce in large production volumes.In future studies, other measures could also be used, such as simultaneously measured cortisol in saliva and behavior, to improve the interpretation of heart rate data with regard, for example, to a dog’s emotional responses and stress . Also, sIn this work, three types of dry ECG electrodes were studied. The evaluation included studying the effect of different activity modes in the electrode evaluation. In particular, the interest was in the dry electrodes that could be conveniently used in the canine ECG measurement without the need of gel or shaving of the animal hair. To carry out the evaluation, a method for QRS complex and R-R interval from the ECG signal was constructed. The R-R interval coverage ratio as an output measure of the performance of the electrodes was evaluated to be as high as over 0.9 in favorable measurement scenarios. However, there was rather large variation in the computed coverage ratios in different cases, which may suggest that the studied electrodes may not be as suitable for all dog breeds. It was found that the metal spring-loaded type electrodes work rather satisfactorily with short-haired breeds while the longer and thick-haired dogs may be more challenging for this particular electrode type. Heart rate monitoring in more dynamic activity modes such as walking was found to be less reliable in terms of heart rate coverage.