Monitoring drivers’ emotions is the key aspect of designing advanced driver assistance systems (ADAS) in intelligent vehicles. To ensure safety and track the possibility of vehicles’ road accidents, emotional monitoring will play a key role in justifying the mental status of the driver while driving the vehicle. However, the pose variations, illumination conditions, and occlusions are the factors that affect the detection of driver emotions from proper monitoring. To overcome these challenges, two novel approaches using machine learning methods and deep neural networks are proposed to monitor various drivers’ expressions in different pose variations, illuminations, and occlusions. We obtained the remarkable accuracy of 93.41%, 83.68%, 98.47%, and 98.18% for CK+, FER 2013, KDEF, and KMU-FED datasets, respectively, for the first approach and improved accuracy of 96.15%, 84.58%, 99.18%, and 99.09% for CK+, FER 2013, KDEF, and KMU-FED datasets respectively in the second approach, compared to the existing state-of-the-art methods. The current way of human living relies on intelligent vehicles developed with artificial intelligence. These smart vehicles make life easier for people who are busy in their daily lives. The lifestyle controls human actions in most situations in their daily routine. The most influenced situation of any human being that leads to severe damage to his life is the effect of emotions while vehicle driving on roads.A driver’s attention will get distracted when they are in the emotional stage which will affect the alertness level and judging capability from normal conditions which are inadequate in safe driving. In total, 2.2% of fatalities are caused by vehicle crashes, according to annual road crash statistics , and 90%Detection of human emotion from the camera-captured images is a reliable monitoring source for various safety and security applications. This can be achieved through facial expression recognition (F.E.R.). Presently F.E.R.  is the pEmotions like happiness, neutral, sadness, disgust, surprise, fear, and anger are usually expressed by a person from his previous and or current performed actions. Some of them are considered negative emotions that influence the driver’s behavior and lead to accidents. These negative emotions trigger the loss of control over driving the vehicle, making the destination unreachable.Many research developments have been made to monitor the driver’s emotions, thereby assisting the drivers in a smooth and safe driving behavior environment on roads. Different behavioral and physiological signals have been involved and utilized for driver emotion recognition. Using PERCLOS   architecture. The design involves a neural network with different optimizers. SGDM supports speed up gradient vectors in the right direction, causing them to converge faster, and the Adam optimizer combines better adaptive gradient and root mean square propagation algorithms. This approach provides better optimization to tackle sparse gradients on noisy conditions and monitor the driver’s emotions in different face rotation angles, occlusions like eyeglasses, hair, and illumination conditions. This system aims to improve the efficiency and performance of the algorithms used. Two novel algorithmic approaches have been developed to monitor and detect emotion to ensure the safety of drivers and vehicles while driving. The proposed algorithms involve preprocessing, segmentation, feature extraction, and classification. The captured images will pass through all stages, and the corresponding class of expression will be detected at the final stage. The performance of the proposed algorithms is compared with the state-of-the-art works using different benchmark datasets and show good performance among them. Two proposed approaches are successfully applied to the real-time virtual driving environment dataset consisting of occlusions and other illumination conditions and obtain better driver expression recognition accuracy.In this section, we explain our two proposed approaches for driver emotion detection. The system starts with image acquisition from the datasets, including NIR camera captured real-time driving environment images that are pre-processed for noise removal and image enhancement. In the first approach, the face is detected, and the corresponding ROI (region of interest) is extracted using the Viola–Jones algorithm from the pre-processed image using Haar features. As the KLT (Kanade Lucas Thomasi) algorithm is the fastest among the traditional techniques to monitor the motion estimation and object tracking functions, it has been utilized to track and extract the feature points from the detected face and perform efficiently in real-time processing. The detected ROI from the KLT is given as an input to the deep neural network, which can reduce the neural network’s workload and will enhance the accuracy in classification. In the second approach, we have included the multi-task cascaded convolutional neural networks for more efficient face and landmark detection by passing the pre-processed image through three different neural networks that are cascade-connected. The processed image area is fed to the deep neural network to extract more robust features from them that help in classifying the expressions with high accuracy, as shown in The pre-processing stage of the proposed algorithm involves image resizing, noise removal, and image enhancement. Whenever the image from the dataset is given as input to the preprocessing stage, the image will immediately be resized and sent to the next block for noise removal. The 2D Gaussian and median filters are applied to remove noise from the image in the noise removal stage.x, y are the distance coordinates from the origin in the horizontal and vertical axis, respectively, and σ is the standard deviation of the Gaussian distribution shown in A smoothing mechanism is used to blur the image using the Gaussian function called Gaussian blur or Gaussian smoothing . MathemaIt is a nonlinear digital filtering method commonly used to remove the noise from an image by preserving the edges under certain conditions during image pre-processing. It works on replacing each pixel with the median of neighboring pixels. The neighbor pattern is called a window, which slides pixel by pixel over an entire image.This pre-processing block improves the image by increasing the contrast of the low-intensity pixel regions and blurs an image with the smoothening technique.To improve the contrast of an image, histogram equalization  is used.f. It is represented as matrix rm by cm with pixels intensities 0 to L − 1 range, where L represents the number of possible intensity values and is mainly equal to 256. The normalized histogram of the image f is represented by p. ThenLet us assume the image given as g will be defined byThe histogram equalized image Facial image segmentation is crucial in the conventional face expression recognition system. This block is used to detect the face from a pre-processed input image with the help of the Viola–Jones algorithm. The corresponding region of interest is obtained by locating the output’s facial landmarks like the nose, mouth, and eyes. The generated region of interest of an image will be fed to the feature extraction block for further processing in the proposed architecture.It is a prevalent technique for real-time object detection developed by Paul Viola and Michael Jones . There ax is 24 × 24 pixel image, θ is a threshold, and p is a parity.Two types of Haar feature masks are represented in There are three steps involved in the learning algorithm: training all the weak classifiers and selecting the efficient weak classifier to become a strong classifier component. Then the collected weak classifiers are combined with the other weak learners, which are confirmed as the most efficient in the earlier stage. Next, the weighted error will be calculated to modify the latest weights for the next iteration.T weak classifier with the selection criteria ofThe further weak learners will concentrate on the toughest ones in the training set, and the strong classifier is formed from a combination of the The weights become larger to a weak classifier with a small amount of error value, and smaller to a weak classifier with a larger error value in classification.Attention cascade will be created due to the selection of strong classifiers which can obtain a higher detection rate with a smaller classification error. At this cascade configuration shown in max’ and maintaining λ > λmax (5–10%). The computation starts from the first partial derivative of image function I in vertical and horizontal directions.It is one of the best tracking algorithms which is used for feature tracking. The facial detection process requires more computation time. It can cause the detector to fail at the training stage of the classifier when trying to detect the face during continuous motions. Hence, it is necessary to have an optimum algorithm to track the face for features. This algorithm  is highlThe three computed values for horizontal, vertical, and both direction locations in image  are:These values can form a gradient matrix ‘G’ and are represented as:1 and λ2; vectors v1, v2 such thatG is a scalar matrix, which means there exist scalars λi ≥ 0. The corner point in the image will be given by the maximum eigenvalue.The gradient matrix is in sparse matrix form. So, the power method is used to find the largest eigenvalue, and the expected eigenvalue is λWe have opted for the current popular deep neural network, which has achieved good remarks in multi-level class expression classification. This deep neural network is designed to overcome the gradient dissipation by including ReLU as the change in activation function and batch normalization. If the network layers increase, the network can perform effectively in extracting features, though the extraction process is more complicated. Thus, a deeper model yields better results. Another problem is that when the network layers are deep, then the network’s accuracy can be degraded, but the deep neural network  is desigDeep neural networks are introduced with a new concept called multi-layers’ deep connections, shown in Zero padding is a method to allow the user to maintain the original size of the input image. It happens during the padding of zeros as a border of pixels around the input image edges. After padding, it will be fed to the convolution layer in which the convolution kernel with its parameters is to be learned. The convolution kernel should be smaller compared to the input image size. The input image is convolved with each kernel to obtain a feature map during the convolution process. The acquired feature map is superimposed with the depth dimension to find the output image at the convolution layer. After receiving the feature map from the convolution layer, the further step is to integrate and classify the features. These features will be fed as an input to the softmax classifier, consuming extensive computation time. To overcome this problem, a pooling layer is used to reduce the feature dimension. It is a process of providing abstractive information. In addition, it reduces parameters, features, and the matrix size, which is generated by the convolution layer to simplify the complexity in calculation at the convolution layer. Two main operations are performed, namely average pooling and max pooling. The average pooling usually reduces the increased variance due to neighborhood size and restores the background data information of an image. Average pooling also guarantees the integrity of the information transmitted and decreases parameter dimensions. Max pooling chooses features with good classification for easy recognition and nonlinear characteristics. In addition, max pooling reduces the parametric error in the convolution layer, which creates a change in the calculated mean value. For instance, if the feature to identify is a car, then until there is a car in any part of the area of an image, it will consider there is a car in the whole image.The batch normalization layer is an optimization method used in the training phase. A batch is the set of images that are trained on the network. The purpose of normalization is for the input data to make the neural network learning process to learn the data distribution. After the training data distribution has differed from the testing data, the generalization capability of the network will be degraded, which in turn affects the training speed of the network due to another factor of training different batches of data with various distributions. Selecting different parameters such as parameter initialization, learning rate, and weight attenuation coefficient, etc., manually will consume most of the time. To adjust these parameters, automatically updated, batch normalization is used. The main principle of this batch normalization layer is to perform a normalization process for each layer acting as input, and a separate normalization layer is added to each layer. The rectified linear unit, in short ReLU, is the activation function used to append the nonlinear factors and make a nonlinear combination of inputs to make the deep neural network classification capability stronger. A piece-wise function makes all negative values zero and does not change all the positive values. For instance, if the input value is negative, the output will be zero. The neuron will not get activated, which means only a few neurons will get activated with the positive values, making the network efficient in computation.We have used the same pre-processing blocks in the second proposed algorithm and used multi-task cascaded convolutional neural networks for face and facial landmark detection.The critical stages in the conventional face expression recognition system are face detection, feature extraction, and expression classification. The main stage is feature extraction, which influences the network’s accuracy. The traditional convolutional neural network architecture gives a better classification with some challenges like pose variations and occlusions affecting the accuracy of the network. To improve the accuracy of the networks, multi-task cascaded convolutional neural networks  are usedThe first network is a fully convolutional neural network shown in The obtained candidates from the first stage will be given to refining the convolutional neural network, shown in It is the last stage in which the output neural network is used shown in Every neural network in multi-task cascaded convolutional neural networks will generate three parts at their stages, and the corresponding loss will also have three parts. Cross-entropy loss function is used directly for detecting faces from an image:Common Euclidian distance is used to find the loss function for boundary box regression and can be calculated as:The network predicted coordinate is Key point decision loss function:Here in the above equation, the network predicted coordinate is This algorithm improved the classification accuracy by utilizing MTCNN for face detection and facial landmarks extraction. Later, the obtained will be given to DeepNet for classification to predict the driver’s emotion.This paper proposes two novel deep network approaches to detect driver’s emotions. The first approach utilizes the Viola–Jones face detection method for frontal face and different angular faces’ detection. Viola–Jones face detection is limited to frontal face detection and obtaining facial landmarks. The corresponding facial features are being detected and tracked by the Kanade Lucas Tomasi method, which are given to the deep neural network for emotion detection. Connecting these methods to a deep neural network brings higher accuracy in detecting a driver’s emotions in different angular rotations of the face and various illumination condition scenarios with partial occlusions like sunglasses and hair involved. The achieved accuracy improves with the second novel deep network approach. We use multi-task cascaded neural networks for face detection and alignment with facial landmark detection in various angular rotations. The obtained features are trained and classified using a deep neural network with ReLU modified to a combination of batch normalization and leaky ReLU to avoid the occurrence of a dying ReLU problem at ReLU, which causes some neurons to die during training. Due to this problem, stochastic gradient descent (S.G.D.) optimization cannot affect the network with the property of the gradient becoming zero even though the input value is negative. So, to overcome this, we use the Adam optimizer, which is faster, stable, and converges faster than S.G.D.Various databases are introduced to evaluate the facial expression recognition performance from image sequences and are used for developing different applications. From those databases, we have chosen the most used and popular benchmark datasets for facial expression recognition, such as the extended Cohn Kanade database (CK+) , facial The parameter settings for the training of our two proposed approaches on all four databases are shown in (I)CK+It is known as the “Extended Cohn-Kanade Database” , one of (II)FER 2013The facial expression recognition 2013 (FER 2013)  database(III)KDEFIt is known as the Karolinska directed emotional face (KDEF)  database(IV)KMU-FEDTo evaluate and verify the performance of our proposed approaches for driver emotion detection in a real-time driving environment, we have selected the Keimyung University facial expression of drivers (KMU-FED)  databaseIn order to verify the effectiveness of the two proposed approaches for driver emotion detection, firstly we have compared their performance with the state-of-the-art approaches which have used the CK+ database to develop face expression recognition systems earlier: (1) D.N.N. utilizing multi-inception layers ; (2) theTo evaluate the performance of the proposed two approaches, we have compared the recognition accuracy of the proposed approaches with the seven state-of-the-art approaches which use either deep neural networks or conventional machine learning algorithms: (1) a D.N.N. that utilizes various inception layers ; (2) a cFrom For performance evaluation, we have compared our two proposed approaches with the state-of-the-art methods that used the KDEF database earlier in For evaluating our proposed approaches in a real-time driving environment, we have compared with the seven state-of-the-art methods in In the process of evaluating the performance of our classification model, we constructed the confusion matrices of our high accuracy obtained proposed approach for the CK+, FER 2013, KDEF, and KMU-FED databases, respectively, as represented in This paper proposes novel deep network approaches to determine the driver’s emotions in a real-time driving environment through facial expression recognition to assist advanced driver assistance systems in intelligent vehicles. These deep network approaches are evolved by detecting the face from captured images by Viola–Jones, and the corresponding features are tracked using the Kanade Lucas Tomasi method. The faces are then fed to the deep neural network for classification and recognition as described in the first proposed approach. In contrast, the second approach has used multi-task cascaded convolutional neural networks for face detection alignment and tracking the features which are given to the deep neural network. These two approaches are trained with different optimizers on the selected benchmark datasets. The work presented in this paper achieves the state-of-the-art result to solve the problems of emotions reflecting a driver’s behavior such as the changes in illumination, side angle positions of the sunlight, occlusions like hair and sunglasses, and different angular face rotations. To assess our proposed approaches’ detection capability, we have conducted experiments on four benchmark databases CK+, FER 2013, KDEF, and KMU-FED, which address the above-mentioned challenges.Due to the COVID-19 pandemic, another challenge has been raised for face emotion detection because of face masks and face coverage. This problem has been discussed in other works ,80,81,82


Here we present a study on the use of non-additive entropy to improve the performance of convolutional neural networks for texture description. More precisely, we introduce the use of a local transform that associates each pixel with a measure of local entropy and use such alternative representation as the input to a pretrained convolutional network that performs feature extraction. We compare the performance of our approach in texture recognition over well-established benchmark databases and on a practical task of identifying Brazilian plant species based on the scanned image of the leaf surface. In both cases, our method achieved interesting performance, outperforming several methods from the state-of-the-art in texture analysis. Among the interesting results we have an accuracy of 84.4% in the classification of KTH-TIPS-2b database and 77.7% in FMD. In the identification of plant species we also achieve a promising accuracy of 88.5%. Considering the challenges posed by these tasks and results of other approaches in the literature, our method managed to demonstrate the potential of computing deep learning features over an entropy representation. Texture is a fundamental feature of complex digital images and texture recognition plays an important role in areas like medicine , materiaDespite the success of learning-based approaches, especially, convolutional neural networks (CNN), texture recognition still poses some challenge, mainly when the images are collected under uncontrolled conditions. In this context, the addition of some extra information to guide the CNN algorithm has significant potential to improve the overall performance.In parallel, an information that is known for a long time to be quite relevant in texture analysis is that provided by local patterns. Methods like that presented in the seminal study of Haralick  and the There are essentially two motivations for the use of non-additive entropy in texture analysis. The first one is that entropy is a complexity measure, which in textures is known to be related with physical properties  that play important role in characterizing materials. Furthermore, non-additive entropy has connections with another successful complexity representation in texture images, which is the multifractal theory. A practical consequence of this is that such definition of entropy provides a framework for multiscale analysis without losing locality information. This substantially enriches the description of local complex patterns arising in the image and additionally providing multiscale information, which is also an important element in texture analysis models.Based on this context, here we propose a hybrid approach that combines the local description power of non-additive entropy with the feature extraction capabilities of pretrained CNNs. More precisely, we develop two independent parallel pipelines. The first one uses the original image as input. For the second, we employ an alternative representation where each pixel is replaced by the non-additive entropy computed over a local neighborhood centered at that pixel. Both pipelines involve the subsequent application of a CNN over the input data. Such CNN is pretrained over the ImageNet and there is no fine tuning in our algorithm, in this way substantially reducing any computational overhead. Finally we take the features at the penultimate layer of the CNN at both pipelines and combine them by concatenation. These features compose our final texture descriptors.Our approach is validated on texture classification, over classical benchmark databases  , Deep TeThe idea of entropy has been introduced to convolutional neural networks for different purposes. For instance, in  the authFinally, non-additive entropies, like the Tsallis definition employed here has been used in image recognition for a long time, with examples of applications in facial recognition , analysiI with a kernelK resulting in a map whose value at each position Convolutional neural networks (CNN) are artificial neural networks especially designed to process multidimensional data, such as images and videos . Their mpooling, which reduces the size of the map in the previous layer. A fully connected network is also frequently used on top of a set of convolutional layers, working as classifiers, whereas convolutional layers act as feature extractors. More details can be easily found in the literature [A typical CNN also includes other operations, such as the application of a non-linear activation function and an operation called terature .backpropagation. Given a set of m training samples, each one corresponding to a pair input/target loss, which measures the error of the network, and epoch) t, they are updated according withlearning rate.The real values in the convolution kernel constitute the learnable parameters Like several other definitions of the so-called generalized entropy, e.g., Boltzman, Shannon, Renyi, and others, Tsallis non-additive entropy  is also p byq and k are pre-defined parameters. In the limit Formally, it is defined for a probability distribution Despite the success of CNNs in texture representation, we still have some room for improvements, especially in the analysis of textures observed “in the wild”, i.e., under uncontrolled conditions. In this context, tasks involving texture recognition can benefit from alternative viewpoints of the same image. An example of such viewpoint that showed promising performance is the non-additive entropy of the original texture, as employed in . At the I and may provide an interesting viewpoint over the original texture. Nevertheless, to be more effective in extracting useful features from that representation we apply softmax classification layer). Similar procedure can also be applied over the original image I and both descriptors can be combined to provide the final descriptors, which will be actually used for recognition tasks. Based on the success previously reported in texture analysis tasks [Based on this, here we propose a hybrid representation for texture images that combines the local power of non-additive entropies for local description with the flexibility of a CNN acting as a feature extractor. More precisely, our method starts by computing Tsallis entropy within the neighborhood of each pixel. Formally, to each pixel at coordinates stant in  as it wostant in  as(8)S. Each class is further evenly divided into 4 samples. And each sample corresponds to particular settings of acquisition, in terms of scale, pose and illumination. Each image has resolution split in , i.e., tFMD  is a collection of color textures acquired under uncontrolled conditions, with a total of 1000 images equally divided into 10 classes. The resolution of the images is UIUC (University of Illinois Urbana-Champaign) database comprises a set of 1000 gray texture images evenly divided into 25 classes. Each image has size UMD (University of Maryland) is a collection of grayscale textures which shares some similarities with UIUC, like the number of samples and classes and the acquisition conditions. The most remarkable difference is the higher resolution, which in UMD is In addition, we tested our proposal in the 1200Tex, which is a database of images corresponding to scanned photographies of the leaf surface of Brazilian plant images. The objective is to identify the respective species. The set contains 20 classes (species) with 20 samples (plant exemplars) per class. The surface image of each sample is a color texture that is partitioned into 3 non-overlapping windows with resolution As for the classifier, we use Linear Discriminant Analysis (LDA) . This isIn this section, we present results on the application of the proposed method to texture classification over the benchmark databases and on the application to plant species identification.q are presented in q, a validation set separated from the training images can be employed, following the usual protocol in machine learning. One possibility for that is to employ a K-fold split, where the training images are randomly divided into K subsets (e.g., q and took that value yielding the best performance. q (around q essentially controls the regularization of the original descriptors. In this context, the results for KTH-TIPS-2b attests that those textures are highly affected by regularization. This is an expected consequence considering the high inter-sample variability, which requires the classifier to identify subtle patterns. Regularization is not effective in this scenario.The accuracies in the benchmark datasets for different values of The confusion matrices of the four benchmark databases analyzed are shown in q on the plant images problem. The behavior is similar to most benchmark cases in The results for the application to the problem of plant species identification are presented in separated figures and tables, given that most methods compared in the benchmark databases do not have results published for the 1200Tex dataset. q. If several values of q are tested, this will linearly increase the time, but such tests are not carried out over the entire database, but rather over only the validation set. As for the image size, the computational time depends on the number of pixels. For the square images with dimension The computational time does not depend on parameter In summary, the results presented in this section suggest that the combination of the original image with an alternative representation in the space of local non-additive entropies has potential to significantly improve the accuracy in texture analysis. The evaluation on datasets with quite different characteristics also suggests the flexibility and robustness of the proposed descriptors with respect to the most diverse variations in attributes like illumination, viewpoint, scale, and others, present in the analyzed databases.This study proposed and investigated the performance of a texture descriptor combining features extracted by a pre-trained convolutional neural network over the original image with features extracted in similar way but over an alternative representation where each pixel is replaced by a measure of local non-additive entropy.The potential of our approach was verified in texture classification, both on benchmark databases and on a practical task of identifying plant species based on the scanned image of plant leaf surfaces. In all situations, our method demonstrated its potential as a robust and precise descriptor, outperforming several approaches of the state-of-the-art in the area.We can summarize saying that our findings confirm that complexity measures, like the non-additive entropy investigated here, have the potential to take advantage of the modern learning-based approaches in texture analysis, especially in the most challenging scenarios, where the extra information captured in the transformed space is highly effective in identifying and characterizing even the most subtle visual patterns.


Goal: To develop a cardiovascular virtual population using statistical modeling and computational biomechanics. Methods: A clinical data augmentation algorithm is implemented to efficiently generate virtual clinical data using a real clinical dataset. An atherosclerotic plaque growth model is employed to 3D reconstructed coronary arterial segments to generate virtual coronary arterial geometries . Last, the combination of the virtual clinical and geometrical data is achieved using a methodology that allows for the generation of a realistic virtual population which can be used in in silico clinical trials. Results: The results show good agreement between real and virtual clinical data presenting a mean gof 0.1 ± 0.08. 400 virtual coronary arteries were generated, while the final virtual population includes 10,000 patients. Conclusions: The virtual arterial geometries are efficiently matched to the generated clinical data, both increasing and complementing the variability of the virtual population. I.in vitro analysis and the in vivo assessment in animal studies. Each subsequent phase requires a different number of patients to be enrolled to secure the efficacy and safety of the new stent. In recent years, computational modelling and simulation enable in silico clinical trials towards reducing, refining, and partially replacing the real clinical trials In silico clinical trials are achieved through the utilization of computational models usually in the form of a medical digital twin and their application to human data. However, the proper evaluation of the in silico stent model is affected by the availability of human arterial samples. Therefore, the lack of large number of patient populations and also the invariability among the enrolled patients enhanced the need of creating virtual patients.The design and development of new stents requires an assessment process to ensure their safety and efficacy. This procedure consists of three phases of clinical trials on humans after the The treatment of a stenosed artery requires the restoration of the blood flow in the lumen area, which is usually achieved through percutaneous coronary intervention (PCI), where the implant called stent, is positioned through a catheter at the stenosed region. In recent years, several studies were performed focusing on the development of computational models that enable the investigation of the stent performance effect on the arterial physiology http://cardiovascularvirtualpopulation.eu.In this work, a workflow for the generation of a virtual population which consists of human patients with cardiovascular disease including clinical data and coronary arterial geometries  is presented. The process for creating the virtual population is implemented using a four-level approach. First, the imaging dataset is used for the 3D reconstruction of the arterial geometries, while clinical data are collected from retrospective patients. In the second level, virtual clinical data are generated using a data augmentation algorithm, which is improved to efficiently replace incomplete data without affecting the statistics of the original population. In the third level, the reconstructed geometries are used as inputs in a computational model describing plaque growth which simulates the atherosclerotic plaque evolution and generates new virtual arterial geometries. At the fourth level, the results of the virtual clinical data with the virtual arterial geometries are combined to generate virtual patients, accompanied by clinical and geometrical arterial data. The novelty of this work regarding the virtual population is that it combines two different approaches for the generation of virtual patients: one for the clinical data generation and one for the arterial geometries. It is the only available population which includes coronary arteries to be used for simulation purposes. The developed virtual population is publicly available for research purposes at II.The developed virtual population and virtual patients consist of realistic virtual geometries and virtual clinical data, respectively. The information flow of the proposed approach is presented in A.Human data are available from the SMARTool B.The InSilc virtual clinical data generation is based on the statistical modeling of real data. Our approach is based on the joint multivariate distribution model. In general, each real patient's clinical data is assumed as a multivariate vector that consists of medical covariates and their corresponding values [35].The objective of generating virtual patients with possible and realistic combinations of data is handled using the method of the total multivariate distribution of the clinical covariates. By using this approach, the distribution and the interrelationships of the real population covariates are evaluated in order to extract the multivariate distribution function, which is a function that describes the probability of a given combination of covariate values. All patient data, which are expressed as vectors of covariates, can be used to create a multivariate distribution, which can be represented as a multi-dimensional surface. Then, each patient vector represents a point of this surface. The function defining the surface of this distribution is statistically addressed as the joint multivariate function. More specifically, it describes the probability of a patient's covariate values to belong in the real population. Vectors with unrealistic values or unrealistic combinations of covariate values have zero probability.Virtual patients are generated by a sampling technique that generates virtual multivariate vectors from surface of the multivariate distribution of the real dataset. By using an appropriate sampling technique, virtual patients are generated in such a way to present the same covariate interrelationships with the corresponding real patients. Using this approach, the possibility of generating unrealistic combinations of covariate values is implicitly eliminated.1)f(x) is known and can be described by n is the number of individuals of the real population, while 1, μ2,..., μnμ)T, defined by Σ is the covariance matrix .This method assumes that the real population presents a multivariate normal distribution. In this case, the joint multivariate function ijΣ is the covariance of ix and jx.The joint multivariate function requires information about the correlation of any medical covariate pair. Specifically, this information is provided by the covariance of every covariate pair in the form of a matrix, the covariance matrix. In case of lack of the covariance values, the covariance matrix can be evaluated using Λ and the eigenvector matrix U of the covariance matrix, the vector of the covariate mean values μ, and k vectors of normal random values k is number of the desired virtual patients (15). The algorithmic approach for this method is the following:1)Λ and U from the eigen-decomposition of Σ.Evaluate Given that the real population presents a multivariate normal distribution and the eigen-decomposition of the covariance matrix is possible, the generation of virtual patients is possible using the eigenvalue matrix Σ is performed using the Cholesky algorithm, which is an iterative algorithm that decomposes any symmetric positive definite matrix, such as the covariance matrix TΣ=UΛU.1)Create the normal random vectors 2)For The decomposition of the covariance matrix 2)CrV) (16). According to this, any continuous value in the range of  is matched to the integer value belonging in this range of values. In this instance, any simulated continuous value in the range  can be matched to the integer of value 1. Therefore, in the case of normal distributions, CrV can be evaluated as the average value of two sequential integers. However, in the case of log-normal distributions, CrV is given by X with k discrete values, μ =mean(ln(x)), σ =SD(ln(x)), iP is the proportion of subjects in the empirical distribution with categorical value i (i ≤ k)x, andThe categorical values are matched to integer values allowing the extraction of the covariance matrix. However, simulation of virtual clinical data can result in non-integer, continuous values of the categorical covariates. The objective is to match the continuous values into integer ones, based on a continuous critical value Z should now be a log-normal random vector belonging to the log-normally distribution (01) All biological covariates are positive by definition. Therefore, a criterion that can be used is to remove all the virtual patients with any “faulty” negative values of the biological covariates. However, this could lead to severe distortion of the virtual multivariate distribution, which would not resemble the original one. Therefore, to efficiently constrain the simulated covariates from getting negative values, a common statistical technique was implemented, using the log-normal distributions for the simulation of medical covariates . According to this, the extraction of the joint multivariate distribution is performed using the logarithms of the real dataset values, while the final virtual clinical data result from the exponentiation of the simulated data . However, the transformation of the normal distributed covariates to log normal distributed covariates requires the alteration of the second step of the algorithmic approach of the method of the joint multivariate function. More specifically, Λ and the eigenvector matrix U of the covariance matrix, the vector of the covariate mean values μ, and a vector of log-normally distributed random values Z ∊ LN. To simplify our analysis, Λ and the eigenvector matrix U of the covariance matrix.A novelty of this work is our approach of replacing any missing values of the real dataset with plausible values based on the inverse procedure of the algorithm used for the virtual clinical data generation. According to the previously described method of joint multivariate distribution, virtual clinical data result from In general, the original problem considered the Z ∊ LN. In this case, to result in a vector LN(01), an optimization algorithm is used. In the case of equations with infinite solutions, as However, given that C.Our approach is aiming to the utilization of a plaque growth model  which is used for the simulation of coronary arterial disease progress in human data with the potential to predict atheromatic areas of risk. The development of this model requires the proper definition of the equations that define the biologic processes responsible for atherosclerosis, using the state-of-the-art models, but also the latest available in literature experimental data. The necessary input for this model is the lumen domain and the arterial wall domain. Moreover, patient specific variables are used in the model, such as the serum LDL and HDL concentration as well as the pressure and the clinical risk factors of hypertension and diabetes. The simulation of each patient is used to extract arterial geometries at yearly time points. In this concept, for each reconstructed artery, four additional arterial geometries  are generated.The employed model has been previously validated and it can be used to simulate the plaque growth and lumen narrowing of 3D reconstructed coronary arteries (20). For this purpose, a multi-level approach is implemented. Initially, blood flow is simulated to calculate the endothelial shear stress (ESS), which is applied as input in the next level of the model. In particular, the main mechanisms of atherosclerosis are simulated using diffusion-convection-reaction equations. The mechanisms of LDL and HDL transport, LDL oxidation, inflammation triggering through the monocytes and macrophages accumulation, foam cells formation as well as the collagen formulation and the smooth muscle cells proliferation, are simulated. The plaque volume is estimated assuming that the plaque consists of smooth muscle cells, collagen and foam cells. The final level of the plaque growth model is to utilize the calculated plaque volume and its integration in a structural finite element analysis simulation to calculate the arterial wall deformation.D.•Define a matching probability threshold for A-C and B-C dataset matching, •Apply the probability recording linkage method on datasets A and C, and B and C. For each sample C, •For each sample •The virtual population consists of dataset C, and all combinations of Through the utilization of the aforementioned methodologies, two virtual datasets are created: (i) one with clinical information, based on the statistical modeling approach and, (ii) one with imaging/morphological information, using the plaque growth modeling approach. The final virtual population combines the two datasets to generate virtual patients with clinical and geometry data. For the purpose of combining the two virtual datasets, we utilize the original SMARTool population of 186 patients which includes clinical and geometrical data. Thus, three different datasets have been utilized  (Table II) to create the final virtual population.For the creation of a large database of a virtual population, yet realistic patients, the concept of probabilistic record linkage (22), originally developed for matching two datasets, will be extended. This is an extension of the original concept of deterministic matching which relies on the exact match of specific fields. In previously developed algorithms its use was mainly for clinical or research purposes where the accuracy of matching is crucial. In virtual populations, on the other hand, a degree of “error” or “variance” is required to create a large variety of cases, keeping, however, all generated cases biologically valid. In our case we have three datasets, A, B and C; dataset C (186 patients from SMARTool population) includes all types of features from both A  and B . Dataset C will be included (after de-identification) in the virtual population per se. To include the other two datasets (A and B) we need to merge them with C or together using C as a “linkage” table. The method is presented graphically in III.http://cardiovascularvirtualpopulation.eu. The Supplementary material presents some screenshots  from the developed database. The database integrates several functionalities such as creating a sub-population based on the applied filters or viewing the arterial geometries of the selected patient. To our knowledge, this is the largest population with clinical data and 3D coronary arterial geometries.A patient population of 186 patients (SMARTool project) was used by the previously described statistical model, to enable the extraction of a virtual population of 10000 patients. The generated virtual population is uploaded and available for research purposes at The virtual clinical data generation required less than a minute to be generated. However, to ensure the validity of the statistical model, several analyses were performed to compare the virtual with the real data. Initially, the mean averages of the virtual covariates were evaluated and compared to their corresponding realistic ones. We also used the evaluation metrics of Goodness of fit (gof) using the Kolmogorov–Smirnov test, the Pearson's product moment correlation coefficient, the mean, standard deviation, skewness and kurtosis.Fig. S5  present the gof and the distribution of some indicative variables, while in silico clinical trials in the stent industry.Plaque growth modelling was applied to 100 patients from the SMARTool population where clinical, molecular and CTCA imaging data were available. This enabled the generation of 400 virtual arterial geometries since we extract the geometries at 1, 2, 5 years, and the last time point of the simulation. Additionally, the dataset of the virtual arterial geometries included 50 arteries which have been reconstructed using the OCT/IVUS and X-ray angiography. Some examples of the baseline arterial geometry and the new one at the final time point presented in IV.in silico clinical trials for coronary stents.The generation of the InSilc virtual population includes several conceptual novelties, as well as innovative implementations and methodologies. To the best of our knowledge, this is the first attempt to integrate 3D arterial information and clinical information towards the creation of a virtual population of human coronary arteries. More specifically, the generated virtual population combines information from two methodological levels: (i) a statistical model of clinical data augmentation, and (ii) a computational model of plaque growth. In brief, regarding the clinical data augmentation, we employ a statistical modeling approach using a real dataset of 186 patients, implementing a new approach for the imputation of the missing values. Using this methodology, we achieved the creation of 10000 virtual patients with clinical information, which is in very good agreement with the real cases. Concerning the plaque growth, our work has been based on an already available computational plaque growth model. It is the first time, in which a plaque growth model generates and delivers as output realistic 3D arterial geometries. This virtual population is the only one available in the literature which can be used for The generation of virtual populations based on the data augmentation of existing datasets is gaining popularity over the last years. An example of a virtual population generation mechanism is the Synthea in silico clinical trials are performed by complex system models, which are built using real patient data Nowadays, In our case of statistical modeling, the selected method of the joint multivariate function is a common and efficient method to introduce virtual patients in the limited original population of our study . However, this method can result exclusively in normal distributed covariates, which is in line with the current literature in silico cloud platform used to perform in silico clinical trials for the design, assessment and optimization of stents. The InSilc platform includes five multi-disciplinary and multi-scale models that simulate the performance of the scaffold in terms of deployment, drug-release, fluid dynamics and degradation while providing indications on the Myocardial Perfusion. Virtual patients are included in several scenarios of use, specially designed towards fulfilling the requirements of the different stakeholders of the InSilc platform, which are the stent industry, the interventional cardiologists, the Contract research organizations (CROs) and the research community. The following scenarios of use have been implemented: (i) Comparison of the performance of existing stents in the same virtual population, (ii) Comparison of stent performance in different anatomy configurations and patient conditions, (iii) Comparison of stent performance in different clinical procedures, and (iv) Design entirely new stents.The creation of this virtual database is part of the InSilc platform Inadequacy to demonstrate efficacy or safety. Coronary stents clinical trial failures occur frequently in the last 10% of the pipeline where 90% of the activity needed to get the device out to market takes place. (ii) Difficulties in patient enrolment. Patient inclusion/exclusion criteria should result in a population that matches statistically the intended general patient population; however, study designers must account for additional concerns, including whether or not particular segments of a target population may have several comorbidities, leading to an additional higher risk of withdrawal and adverse events. (iii) One size fits all issue. The clinical trial designs essentially do not take into account the patient variability and complexities, and the heterogeneity of the enrolled in clinical trial patients are being translated into a similar heterogeneity of responses to stent implantation. The availability of a virtual population applied in the context of new stent design will increase patients’ safety, reduce clinical trials costs and as the final outcome will improve the clinical practice offering highly efficient stents with reduced side effects. FDA has recognised the benefits of modernizing real clinical trials and has already begun exploring the utilisation of in silico technology and virtual population to improve clinical trials. This strategy has been communicated through the 21st Century Cures Act that focuses on accelerating medical product development and bring innovations and advances to patients as well as through the creation of the Technology Modernization Action Plan (TMAP) with the mission to bridge the gap between scientific and technology advances The importance of generating a virtual population in the field of virtual stent deployment is significant. Cardiovascular disease has a significant economic impact estimated to cost the EU economy 210 billion euro per year V.in silico clinical trials in the stent industry.In this work, a novel concept for the generation of a virtual population of cardiovascular disease patients with virtual clinical and arterial geometrical data has been introduced. For this purpose, an approach for the statistical generation of virtual clinical data has been implemented, while a novel method was developed to treat efficiently incomplete data. Also, for the first time, a computational model has been used to reproduce virtual arterial geometries. Finally, the results of the two models have been combined to generate a unique virtual population of human patients with coronary arterial information adequate for


A multiple-input multiple-output (MIMO) method that shares the same frequency band can efficiently increase radar performance. An essential element of a MIMO radar is the orthogonality of the waveform. Typically, orthogonality is obtained by spreading different signals into divided domains such as in time-domain multiplexing, frequency-domain multiplexing, and code domain multiplexing. This paper proposes a method of spreading the interference signals outside the range bins of interest for pulse doppler radars. This is achieved by changing the pulse repetition frequency under certain constraints, and an additional gain can be obtained by doppler processing. This method is very effective for improving the angular accuracy of the MIMO radar for a small number of air targets, although it may have limitations in use for many targets or in high clutter environments. As the application field of radar systems has expanded, the number of radar sensors operating at the same frequency has been continuously increasing. Moreover, the use of the multiple-input multiple-output (MIMO) method can efficiently increase radar performance by sharing the same frequency band. There have been many studies on MIMO performance in recent years ,2,3,4. MMultiple transmission methods in MIMO radar include the array-space multiple transmission method that transmits different waveforms from different arrays and synthesizes them in the receiver, and the beam-space multiple transmission method that transmits different waveforms in different directions. In all cases, because the different transmit waveforms should be received independently in the receiver, it is necessary to use signals with orthogonal characteristics. Unfortunately, because the delay of the reflective signal is specified by the target position and cannot be synchronized from the receiver in radar systems, transmit waveforms should be orthogonal for all time delays, which is almost impossible. Therefore, the transmit waveforms in MIMO radar are usually designed to minimize the cross-correlation for all time delays—called a quasi-orthogonal waveform—while minimizing the sidelobe level of the autocorrelation to improve detection performance.Orthogonality, which removes other waveforms, is achieved by dividing the power of each waveform into the different domains rather than cancelling them. The typical methods for designing orthogonal waveforms are time division multiplexing (TDM), frequency division multiplexing (FDM), and code division multiplexing (CDM) ,8,9. AltThe signal processing of the pulse doppler radar consists of a matched filter for intra-pulse modulation and coherent integration between pulses. If the received pulse is modulated by different codes, the output of the matched filter is the sum of correlations between the matched and mismatched codes. In this first step, the signal-to-interference ratio (SIR) is improved in terms of both noise and cross-correlations, constituting the interference signal. However, because the interpulse coherency is maintained even for the mismatched codes, interference by cross-correlation is not suppressed by coherent integration processing, and only the portion by the noise is further suppressed. Therefore, if the noise level is relatively high that normally occurs within the maximum detection range of a radar, the SIR is improved by both intrapulse and interpulse processing; however, as the target comes closer, the signal and the cross-correlations increase, and the interference is dominated by cross-correlations. In this close range, SIR does not change by distance and cannot be improved beyond a certain level determined by the matched filter, and it eventually affects the angle estimation accuracy.In this paper, we propose a novel method to further improve the SIR ratio in interpulse coherent processing and enhance the angular estimation performance of virtual arrays of a MIMO radar. The proposed method uses different PRFs and intrapulse modulation codes while retaining the MIMO beamforming condition; thus, the pulse-to-pulse coherency from different transmitters is not maintained, and cross-correlations cannot obtain inter-pulse integration gain. This method is useful for improving the accuracy when there are a small number of aerial targets, but also has limitations that are difficult to use in cluttered environments because it spreads the power of other transmit signals to the irrelevant range bins.The intrapulse code used in this study is based on polyphase codes that are designed to optimize autocorrelations and cross-correlations. There are two well-known design methods for a polyphase code. One is the family of cyclic algorithm-new (CAN) algorithms, including stopband CAN (SCAN) and periodic CAN (PeCAN), where the objective is to minimize the sum of the cross-correlation and the sidelobe of auto correlations through a cyclical process ,11. The The remainder of this paper is organized as follows. d and the transmit array is a sparse ULA with d intervals. If v is identical to that of a ULA composed of 12 elements  with elements .T is the pulse width, Three arrays transmit different waveforms which are represented byThe signal processing on each receiver starts with the matched filter to extract each transmit signal. The output of The desired output is the first term of the auto correlation and the remaining cross-correlation and noise terms are all interference.i-th transmitter is not correctly aligned in the j-th processing chain supporting another PRI.Next step is the doppler processing of coherent pulse integration, which is performed for the range bins of the same location among pulse trains. Because transmit waveforms are typically designed to use the same PRF, all reflected signals are in the same range bin. And both the matched code and the mismatched code obtain the same coherent integration gain; thus, no further improvement of the ratio between matched code and mismatched code is made at this stage. In this paper, we suggest a novel method of using different PRFs to suppress the level of cross-correlations further. Using different PRFs is to change the reflection range bins from other transmitters and prevent them from obtaining the coherent integration gain. In other words, the signals from other transmitters cannot be aligned in the same range bin among pulse trains if the PRF is different. To perform MIMO beamforming in Equation (3), it is important to ensure that the phase information is not changed by different PRFs. Therefore, we first synchronize the start time to align the initial phase and use the same Coherent Integration Time(CPI) to achieve the same doppler resolution. If This should be constant for all the transmit signals. The maximum value of k increases, where If This means that the maximum amount of shift is determined by (n is an integer, thenTherefore, the reflective powers are spread evenly around the entire range bin. If only two transmit signals are used k is an integer greater than 1.Second, the measurement scale should be the same for all the processing chains. Because the number of pulses is In this section, we present the simulation results obtained by applying the proposed method. The used waveform of the pulse train is described in Three transmit arrays and four receive arrays are used for MIMO beamforming, as shown in The remaining interference reduces SIR and eventually affects the angular accuracy of MIMO beamforming. We propose a method to use different PRFs in a MIMO beamforming system and demonstrate its performance through simulations. To perform MIMO virtual array beamforming, the transmit signals should be separated in each receiver and must be designed as orthogonal waveforms. In radar systems that cannot synchronize the reflected signal, it is very difficult to eliminate the undesirable mismatched codes for all delay times. Moreover, these signals increase with the signal power and limit the performance when the SNR is high. The proposed method spreads the interference signal to other range bins and achieves additional suppression through doppler processing.Because this method basically spreads the interference signal over the entire range bins, applications in an environment where there are many targets or a lot of clutter will be difficult. However, it is very effective for improving the angular accuracy of the MIMO radar for a small number of aerial targets.


A female patient who underwent successful reconstruction of an acute high-grade acromioclavicular (AC) joint separation with hook plate presented with failure of the reduction eight weeks after removal of the hardware. Surgeons and patients should be aware of the risk of late failure of acromioclavicular reconstruction after removal of the hook plate. Blunt injuries to the acromioclavicular (AC) joint are among the more common injury patterns to the shoulder in athletes and often lead to pain, disability, and associated decreased function and performance . These iOver the years, a wide variety of operative CC ligament reconstructive techniques has been developed to identify an optimal approach to the surgical management of this challenging shoulder injury. The current techniques described are either arthroscopic, open, or combined. Though many of these techniques have demonstrated good results, they each carry with them very specific advantages and disadvantages with regard to risk, patient satisfaction, and functional outcomes ,6-9. OneThe patient was informed that data concerning the case would be submitted for publication, and she provided consent.A 52-year-old female non-smoker with an uncomplicated medical history presented to the orthopedic clinic with right shoulder pain after an all-terrain vehicle (ATV) accident involving a direct blow to the lateral aspect of the shoulder. The injury had occurred the previous day and radiographs from the emergency department revealed a high grade (Rockwood type V) right AC joint separation Figure . In the Fifteen days after her initial injury, the patient was taken to the operating room. She underwent a right shoulder AC joint reduction using a right acromioclavicular four-hole hook plate . A superior approach to the AC joint was used, and the hook of the hook plate and a reduction clamp were used to achieve reduction. The surgery was successfully completed without complications, with the position of the hardware and reduction of the AC joint confirmed by fluoroscopy as well as postoperative radiography Figure . She wasAt six weeks postoperatively, she was removed from the brace and progressed to a course of therapy for the range of motion of the shoulder and self-stretching with continued limitations of heavy use, lifting, pushing, or pulling.At three months postoperatively, the patient demonstrated maintenance of reduction with a pain-free range of motion of the shoulder, which was symmetric to the contralateral shoulder. Her examination revealed a well-healed incision with no residual tenderness over the AC joint. Active ROM was measured with forward flexion to 160 degrees, external rotation to 70 degrees, and internal rotation to the level of T9. Radiographs at that time demonstrated the appropriate position of the clavicle with near anatomic reduction of the AC joint and some minimal bony reaction on the undersurface of the acromion. The decision was then made to remove the hardware according to the manufacturer’s recommendations to avoid potential complications with the prolonged indwelling of the implant.The patient ultimately returned to the operating room four months postoperatively. She underwent removal of the hook plate and associated hardware. The surgery was uneventful, and the AC joint was found to be stable after the removal of the hardware. No loss of reduction was observed intraoperatively, and intraoperative fluoroscopy and postoperative radiographs demonstrated a reduction of the joint Figure . FollowiSix weeks after the removal of the hardware, the patient returned to the office with a pain score of 0 out of 10. Her exam revealed a well-healed surgical scar, a reduced AC joint with no deformity, and the full range of motion of the shoulder. Her physical therapy was continued with further emphasis on gradual strengthening and stretching.Eight weeks after hook plate removal, the patient presented with pain and deformity over the AC joint. She denied new trauma and reported a feeling of movement at the clavicle during stretching with the physical therapist. Her examination revealed the clavicle had re-displaced to a posterosuperior position relative to the acromion with point tenderness over the AC joint that was aggravated in cross-chest adduction. Bilateral AC joint radiographs were obtained, which demonstrated a two-fold increase in CC distance on the right side when compared to the left side, confirming the diagnosis of re-separation of the AC joint Figure .Four weeks later, she underwent revision reconstruction of the AC joint using a semitendinosus tendon allograft. To date, she is six months post-revision and reports a pain score of 0 out of 10, has a subjective shoulder score of 95%, and has resumed her work as a hairdresser.There are over 150 surgical techniques described for the correction of AC joint separation, but there is no consensus on the ideal fixation method . HoweverThe hook plate technique was first introduced in 1976 by Balser . The tecUnfortunately, the hook component of the implant can cause bursa and rotator cuff tendon irritation and tears in addition to bony attrition of the acromion, which can lead to acromial fracture ,18. As aOur case is unique due to the delayed failure of the reduction. At the time of hardware removal, there were no radiographic or clinical findings to suggest a failure of ligament healing. However, at eight weeks post hardware removal, the patient experienced a spontaneous recurrence of AC type V separation, suggesting that she had not yet fully healed. It is likely that this patient could have benefitted from a longer fixation period prior to hardware removal, although there were no components of her history or injury to suggest that this was necessary. To date, there are no other reports of spontaneous failure of the AC joint multiple weeks after hardware removal.In summary, this patient presented with a late failure of the reduction despite early operative intervention, a successful postoperative course, and relatively delayed removal of the hardware at four months. Based on this case, we suggest that consideration should be given to delayed removal of the hook plate in patients who do not have significant clinical or radiographic evidence of subacromial or rotator cuff irritation or bony attrition, to decrease the possibility of failure and need for further surgical reconstruction.


It is of utmost importance to develop a computational method for accurate prediction of antioxidants, as they play a vital role in the prevention of several diseases caused by oxidative stress. In this correspondence, we present an effective computational methodology based on the notion of deep latent space encoding. A deep neural network classifier fused with an auto-encoder learns class labels in a pruned latent space. This strategy has eliminated the need to separately develop classifier and the feature selection model, allowing the standalone model to effectively harness discriminating feature space and perform improved predictions. A thorough analytical study has been presented alongwith the PCA/tSNE visualization and PCA-GCNR scores to show the discriminating power of the proposed method. The proposed method showed a high MCC value of 0.43 and a balanced accuracy of 76.2%, which is superior to the existing models. The model has been evaluated on an independent dataset during which it outperformed the contemporary methods by correctly identifying the novel proteins with an accuracy of 95%. The oxidative damage due to free radicals is prevented by the antioxidants which are naturally produced in the human body . Small aRecently, deep learning (DL) based classification methods have been proposed to perform prediction in almost every field. In particular, many bioinformatics applications have taken advantage of the outstanding capabilities of deep neural networks (DNN) . The preThe major advantages of the proposed approach is its ability to learn the non-linear embedding of the features whilst maintaining the properties of the original features through reconstruction. This consistency helps filter out fluctuating data points by evaluating their distance from the class centers. As a result, the model can be an important tool for designing a classifier with noisy annotations. Another advantage of using such configuration is the elimination of the need of a crucial step of feature engineering, which involves the selection of dominant features for the training of classifier. The built model has been evaluated by the widely known statistical parameters, and the performance is compared with the existing methods. The results show that the proposed method offers superior performance on the benchmark dataset.The rest of the paper is organized as follows: In To evaluate the performance of the proposed method, we used metrics for the imbalanced class samples including sensitivity . It is a comprehensive encoding scheme, and has shown significant performance in variety of protein prediction tasks [k denotes the gap or space between the pairs of the amino acid fragments. For The protein features are made compatible with the machine learning algorithms by encoding them numerically. Several encoding schemes have been utilized by the researchers in accordance with the adopted machine learning method ,15,16,17on tasks ,19,20. IAs observed in The proposed Deep Latent Space Encoding (DeepLSE) model comprises of three modules: (1) encoder, (2) decoder and (3) classifier. The input layer of the encoder of the baseline model consists of k). The output of encoder/decoder and classification network uses sigmoid and soft-max activation functions respectively, while the hidden layers of all modules use rectified linear unit (ReLU) activation function to avoid the occurrence of vanishing gradient. Two loss functions; mean squared error and binary cross-entropy for auto-encoder and classifier respectively are being minimized using the default learning rate of RMSprop optimizer for 1000 epochs with an early stopping tolerance of 100 epochs. The convex combination of two losses is achieved byThe model is trained using Python on Tensorflow-Keras  platformk during the encoding, and the number of units in the latent space LS. An ablation study has been designed to acquire latent-space encoding (LSE) with varying number of aforementioned variable parameters according to the workflow depicted in k, the model is trained with several values of latent space variables k) between the amino acid pairs. The model with the best average results is finally selected as the base model to perform prediction and is named as AoP-LSE.The workflow of the proposed study is aimed to obtain the best classification model based on two variable parameters, i.e., the gap between the two amino acid pairs, which is done by setting different values of The summary of the ablation study is provided in a visual and tabular form. The results of the BACC, MCC and PSNR are provided as a surface plot in k and For the aforementioned combination of It was observed from the analysis above that a shallow model  + multi-layer perceptron (MLP) neural network. However, the way it has been configured/trained i.e., combined training of AE+MLP makes it different from the conventional approach where AE is trained separately for dimension reduction and obtain the most representative (low-ranked) features in the latent space (LS) for signal reconstruction. The proposed DeepLSE on the other hand, not only learns the most representative features but also impose a classification constraint on the latent variables. Thus, creating a tug of war between the decoder and the classifier forcing the encoder to generate useful features. To emphasize this point and for a fair comparison with existing techniques, an evaluation on three different models has been presented.scikit-learn package [We trained the auto-encoder network with similar configuration  ,36, and In this study, we proposed a deep latent space encoding method for the classification of anti-oxidant proteins using sequence derived features. In particular, the composition of k-spaced amino acid pair (CKSAAP) and a densely connected multi-layered perception neural network are used which are trained in tandem. The proposed method can be used to extract a non-redundant compact feature space, which is shown to outperform the conventional antioxidation protein classification approaches. Furthermore, the effect of varying parameters and number of gaps in CKSAAP and latent variables in Deep-LSE is analyzed, which suggests that a sufficient separable encoding can be learned by keeping moderate number of neurons in the architecture with


Quantitative tracking of equine movement during stall confinement has the potential to detect subtle changes in mobility due to injury. These changes may warn of potential complications, providing vital information to direct rehabilitation protocols. Inertial measurement units (IMUs) are readily available and easily attached to a limb or surcingle to objectively record step count in horses. The objectives of this study were: (1) to compare IMU-based step counts to a visually-based criterion measure (video) for three different types of movements in a stall environment, and (2) to compare three different sensor positions to determine the ideal location on the horse to assess movement. An IMU was attached at the withers, right forelimb and hindlimb of six horses to assess free-movement, circles, and figure-eights recorded in 5 min intervals and to determine the best location, through analysis of all three axes of the triaxial accelerometer, for step count during stall confinement. Mean step count difference, absolute error (%) and intraclass correlation coefficients (ICCs) were determined to assess the sensor's ability to track steps compared to the criterion measure. When comparing sensor location for all movement conditions, the right-forelimb vertical-axis produced the best results  followed closely by the right-hindlimb . Limitations included the small number of horse participants and the lack of random selection due to limited availability and accessibility. Overall, the findings demonstrate excellent levels of agreement between the IMU's vertical axis and the video-based criterion at the forelimb and hindlimb locations for all movement conditions. During recovery from an injury, horses are frequently confined to a stall for the initial phase of their rehabilitation. This confinement can result in restricted movement due to limited space and avoidance of painful weight-bearing on the injured limb. Movement is essential and aids the healing process by promoting circulation and the subsequent delivery of nutrients and oxygen to damaged tissues. Identification of reduced movements  during stall confinement could significantly influence treatment outcomes by providing early recognition of complications and the implementation of preventative measures .Although experienced veterinarians can subjectively assess most gait events , they are limited by the temporal resolution of the human eye and inter-observer agreement –5 and haTherefore, the purpose of this study was to determine the concurrent validity of an IMU for equine step count determination during stall confinement compared to a visually-based (video) criterion. The specific objectives were: (1) to compare IMU and video-based step counts for three different movement types, and (2) to compare three different body positions and axes to determine the ideal location for IMU placement. It was hypothesized that the IMU would be capable of producing excellent levels of agreement  at one or more of the test locations when compared to the video-based criterion measure.The study was approved by the University of Saskatchewan's Animal Research Ethics Board and adhered to the Canadian Council on Animal Care guidelines for humane animal use. For client-owned horses, informed client consent was obtained.Six horses of varying breed , sex and age (4–23 years old) participated in the study . The numThe protocol for determining IMU step count validity involved three different movement conditions: free movement, circles, and figure-eights. These movements were selected to mimic the different kinds of movements horses might display in a stall environment. The free movement condition simulated a horse in its natural environment, randomly moving around the stall for a specific period of time (5 min). Circles were selected to mimic a horse pacing around the stall, which occurred in some earlier preliminary trials. All six horses performed the free movement and circle conditions. The two horses tested in the hospital stall also completed a figure-eight movement pattern, as this would occur during stall confinement when attached to a computerized rehabilitation support device previously reported by our group . This moEach horse completed four test sessions on separate days see . Each seThree IMUs  with a range of ±32 g were placed on the right forelimb, right hindlimb and at the withers  to deterThree IMUs were used to permit data collection at all three locations simultaneously so that direct comparisons of different locations could be made for the same trial. The accelerometer-based step count was compared to the visually-determined step count to assess the sensor's validity for stall confinement movement analysis. A single observer analyzed the video recordings to determine the criterion step counts. All trials were recorded using a GoPro Hero 7 White  video camera placed in the corner of the stall on the floor for clear visualization of the participants' limbs. The camera recorded at 60 frames per second  while the accelerometer sampled at 100 Hz. After the video recording was started, the IMUs were activated, 5 min of walking was recorded, and the video was stopped once all IMUs were deactivated. The 5 min trial period was determined based on the time elapsed on the GoPro video recording application, which was observed externally through a mobile phone.The sensor locations were selected based on previous studies that used IMUs or accelerometers , 11, 15,Accelerometer vs. video-based step counts were compared to determine sensor validity and the most accurate location for placement. A step was defined as the visual determination of lifting the heel off the ground and a flexed carpus or tarsus. When the accelerometer was placed on a limb, only the steps from that limb were counted, given that the accelerometer would only pick up movement from the limb to which it was attached. When the accelerometer was placed at the withers, the steps of both front legs were counted. The raw accelerometer data were processed using a zero-lag, fourth-order Butterworth low-pass filter set to a cut-off frequency of 10 Hz. A peak-detection algorithm see , 18. SomAn intraclass correlation (ICC) was used to determine the absolute levels of agreement between the two methods (sensor step count vs. video-based step count). A model 3 , 3 ICC wBland-Altman plots were also determined for each sensor location to visualize the limits of agreement between the two methods . The ICCThe ICC, mean % error and step count differences  are shown in . These cThis study investigated the concurrent validity of an IMU to determine equine step count during stall confinement. The hypothesis that the IMU-based method would achieve excellent levels of agreement  compared to the video-based criterion was supported. The majority of axes and locations for all movement conditions had excellent levels of agreement (ICC > 0.75); however, several results did not meet this criterion, particularly for the anteroposterior and vertical axes at the withers location. Overall, IMU step counts agreed well with the criterion measure with more than half of the ICCs above 0.9, with low percent errors and step count differences. Consequently, the results demonstrate that IMUs are capable of accurately determining step count in horses during stall confinement.Specifically, the vertical axis at the forelimb and hindlimb locations was the most accurate, and while both locations were best for different movements , they behaved similarly in terms of outcome. Most importantly, the right forelimb had the highest accuracy for the combined movement analysis and in the free movement condition, which would be the most representative of a standard rehabilitation setting. When interpreting these results, it is important to appreciate that the ICC quantifies the level of agreement between two independently determined variables, in this case, the sensor-based step count and the video-based step count. Consequently, the ICC assesses the degree of similarity between these two measures. A high level of agreement will be achieved when a greater number of steps are counted in the video, and a corresponding greater number of steps are recorded by the sensor (and similarly for a lower number of steps). However, the absolute percent error (based on the absolute step count difference) measures the proportional discrepancy between the two measures. In addition to the level of agreement, this discrepancy is important because, in a rehabilitation context, a comparison of values across hours or days would be used to determine how much a movement had increased or decreased. In the presence of large step count errors, between-day comparisons may not have sufficient accuracy to identify small changes in movement. Accuracy is key in being able to detect changes early and possibly prevent life-threatening complications. Unfortunately, acceptable levels of agreement and absolute errors for equine rehabilitation monitoring have yet to be determined. Future studies on injured horses should attempt to determine clinically significant levels of movement reduction associated with conditions such as muscle wasting, osteopenia, or supporting limb laminitis (SLL). It is also not known how long movement reductions can persist before veterinarians need to intervene. If an IMU is used to monitor recovery from injury, daily step counts that indicate slight declines over time, could suggest that more significant interventions are needed to increase activity to promote cyclic loading and blood flow, preventing life-threatening complications from developing.The results of this study are comparable to those reported by Fries et al. , who useThe most important finding of this study pertains to the free movement condition, given that rehabilitation monitoring would likely consist of monitoring a horse in a stall with random intermittent movements. Injured horses that require monitoring are less likely to pace around the stall if they are in pain. As such, the reason for monitoring a horse is usually insufficient movement, which may be indicative of inadequate pain control or the development of immobility-associated complications. Measuring movement in injured horses could prove to be challenging with added alterations in movement, such as reduced and altered loading patterns due to pain, changes in gait patterns due to asymmetries , 21–23, Finally, it should be noted that this study has several limitations, including potential errors in the criterion step count determination  and misidentification errors in the raw accelerometer data by the peak-detection thresholds. It was also not possible to randomly select the participants, as the horses that did participate did so based on accessibility and availability. This resulted in a higher proportion of older and middle-aged horses in the sample, which, while not affecting the comparison between the methods, could have affected the frequency and speed of movements as well as the total number of steps. In addition, the horses may have moved less if they were uncomfortable with the sensation or the placement of the IMU's, but this is unlikely to have affected the results given the total number of trials, movements, locations and steps that were analyzed. Future studies should compare IMU and video-based methods using a larger sample of horses and additional gait and/or stall-related movement parameters, such as duration of time spent walking vs. standing, limb kinematics and postural sway when standing for prolonged periods of time. The results of this study suggest that the development of an iOS or Android app to track movement and provide updates over days or weeks has the potential to provide a substantial benefit to the practice of equine rehabilitation.This study found that the vertical axis of a sensor placed at the forelimb or hindlimb was the most accurate to determine step count in a stall environment. Excellent levels of agreement suggest that an IMU could continuously monitor the quantity of movement in a stall, which would indicate any abnormalities associated with reduced movement before those changes could become severe. The results show that accelerometry has the potential to provide an easy and valuable tool for veterinary medicine in the early detection of movement reduction that could signal increasing levels of pain or immobility-associated complications. This early detection could provide an opportunity for timely preventative interventions during the injury recovery and rehabilitation process, possibly leading to more successful outcomes.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.The animal study was reviewed and approved by the University of Saskatchewan's Animal Research Ethics Board and adhered to the Canadian Council on Animal Care guidelines for humane animal use. Written informed consent was obtained from the owners for the participation of their animals in this study.SS wrote the article. SS was a graduate student in biomedical engineering co-supervised by JM and JB. The field of research is equine biomechanics and rehabilitation. All authors revised the manuscript and approved the final version for submission.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.