Technologies such as robot-assisted gait trainers or functional electrical stimulation can improve the rehabilitation process of people affected with gait disorders due to stroke or other neurological defects. By combining both technologies, the potential disadvantages of each technology could be compensated and simultaneously, therapy effects could be improved. Thus, an algorithm was designed that aims to detect the gait cycle of a robot-assisted gait trainer. Based on movement data recorded with inertial measurement units, gait events can be detected. These events can further be used to trigger functional electrical stimulation. This novel setup offers the possibility of equipping a broad range of potential robot-assisted gait trainers with functional electrical stimulation. The aim of this paper in particular was to test the feasibility of a system using inertial measurement units for gait event detection during robot-assisted gait training. Thus, a 39-year-old healthy male adult executed a total of six training sessions with two robot-assisted gait trainers (Lokomat and Lyra). The measured data from the sensors were analyzed by a custom-made gait event detection algorithm. An overall detection rate of 98.1% ± 5.2% for the Lokomat and 94.1% ± 6.8% for the Lyra was achieved. The mean type-1 error was 0.3% ± 1.2% for the Lokomat and 1.9% ± 4.3% for the Lyra. As a result, the setup provides promising results for further research and a technique that can enhance robot-assisted gait trainers by adding functional electrical stimulation to the rehabilitation process. The human gait is a complex interaction between the human sensory and motor systems and counts as one of the most important activities of daily living. Numerous pathologies such as stroke, spinal cord injury, and multiple sclerosis can disrupt the finely adjusted system and can cause gait disorders or the incapability of walking. In order to counteract gait disorders, various rehabilitation techniques are used in clinical routine. Besides conventional physiotherapy, electromechanical gait training/robot-assisted gait training is used. The used gait trainers provide a harness and a body weight support system that supports the paretic lower limbs. Additionally, it compensates for the lack of balance reactions and it supports the patient in case of obesity. The gait pattern is externally induced by a robot-specific strategy. Robot-assisted gait trainers such as the Gait Trainer GT II , G-EO , Lyra , and the Lokomat  are used in clinical routine. These robots can be classified depending on their movement principle into two main groups. Exoskeletons move joints such as the knee and ankle joint during the gait cycle, whereas controllers of endpoint trajectories move only the feet. Typical examples of these robots are the Lokomat as an exoskeleton and the Gait Trainer (or Gang Trainer) as end-effector based system . A curreReal-time gait detection from an easy-to-use system setupArbitrary sensor alignmentFunctional electrical stimulation during robot-assisted gait trainingDespite introducing new technologies, keeping the additional time expenditure for the therapist on a low levelThe overall concept of the system aims to be used as an add-on for a robot-assisted gait trainer such as Lokomat or Lyra, thus creating a hybrid robotic rehabilitation system. The system deals with specific scientific and technological challenges, including:The overall concepts are shown in Lokomat: The Lokomat is a commercially available robot-assisted gait trainer that is composed of robotic gait orthosis, a body weight support system, and a treadmill. Programmable actuators in the orthosis enable a cyclic gait pattern. The system is a state-of-the-art gait rehabilitation system [n system  and provn system .Lyra: The Lyra is an end-effector based robot-assisted gait trainer. It provides a physiological gait pattern and a short setup time due to the ground-level entrance of the system, which allows easy accessibility for wheelchairs. For stroke patients, gait trainers with the end-effector principle yielded a superior effect on gait function, balance, and activities of daily living compared to conventional gait training [training .The RehaStim  is a portable electrical stimulation device that generates impulses on up to eight channels simultaneously. Surface electrodes are used to activate paralyzed muscles. This stimulator is certified according to the EU guideline EN60601-2-10 for medical technical devices and systems. The device can be used alone or in combination with external processing units and has been used as stimulator in recent research .RehaStim is capable of providing currents up to 130 mA with a step size of 5 mA; the frequency can be set from 10 Hz to 50 Hz with a step size of 5 Hz. The according pulse width can be adjusted from 20 μs up to 500 μs with a step size of 10 μs. The stimulator provides biphasic rectangular impulses in order to provide charge neutrality within the stimulated area. Thus, a large variety of different settings is provided, which gives clinicians the capability of adjusting the parameters according to the needs of the patient.Data acquisition was realized with two custom-built inertial measurement units . Each IMU consisted of a three-axis accelerometer (±16 g) and a three-axis gyroscope (± 2000°/s) . The samThe human gait cycle consists of two phases: the stance phase and the swing phase. These phases are determined by certain gait events . Gait trAs not all gait events are reliably detectable with IMUs, the presented algorithm focuses on the most important gait events for defining the gait cycle. These events are the Initial Contact, Full Contact during Midstance, Heel Off, and the Toe Off. With two IMUs, these gait events are detectable for each lower limb, resulting in an amount of eight detectable events during one full gait cycle. As gait trainers provide a cyclic movement resulting in a cyclic gait, unwanted switching between the gait phases  can be excluded. Additionally, gait trainers limit the movement to the sagittal plane. Taking these boundary conditions into account, a finite-state diagram as shown in In order to detect the gait events, one IMU per lower limb is used. Inertial measurement units measure their data in their local coordinate system .Tightly attaching the sensor to the foot and overlapping the two coordinate systems manually.Calculating the current orientation of the sensor for each data point using an arbitrary sensor alignment algorithm.For further processing of the gathered data, the local coordinate system of the IMU has to be aligned with the local coordinate system of the foot. This can be done by two approaches:The latter option is more robust against changes of IMU position, which can happen during robot-assisted gait training. Additionally, it allows an arbitrary sensor positioning, which makes it easier for the therapist to use, as they do not have focus on the correct positioning of the IMU. Furthermore, the orthosis of the robot-assisted gait trainer is rigidly attached to the patient’s leg; as a result, a pre-defined sensor position—as mentioned in the first option—could interfere with the attachments of the orthosis, making it impossible to realise a feasible setup.For realizing an arbitrary sensor positioning, the local coordinate system of the sensor  must be (1) Full Contact detection and n denotes the current sample, k denotes a reasonable amount of passed samples, and During the cyclic movement of the gait trainer, Full Contact can be detected when the angular velocity When the Full Contact condition is satisfied, the (2) Defining the main rotation axis and After Midstance, the heel starts to rise , causingthe foot .n after Full Contact, and ending after a certain amount of data points k.After Midstance has passed, the main rotation axis of the IMU can be determined by summing up the angular velocity of each axis starting with the first sample The absolute angular velocity components are kept in a vector defining the ecessary .(3) Cross-Product and The (4) Normalizing and creating of the rotation matrixAfter calculating all three axes, each axis has to be normalized. Afterwards, it can be stored in a final rotation matrix:After the rotation matrix is determined, the angular velocity and linear acceleration data can be rotated for further processing: This process is done for each incoming data point. Furthermore, the rotation matrix is recalculated for every new Full Contact, and thus taking variations of individual movements into account.Initial Contact: A gait cycle is initiated by an Initial Contact. When the heel hits the ground, the body needs to be damped and stabilized, either by the muscles or by the robot. This impact results in a peak of the jerk, One has to mention that the jerk during walking and the jerk during robot-assisted gait training varies from step to step. Therefore, a floating threshold is introduced, resulting in a continuous adaption of it cycle .Full Contact: The detection of Full Contact is already realized in the arbitrary sensor alignment algorithm and can be used for gait event detection without any further computational effort.Heel Off / Heel Rise: After the stance phase, Heel Off can be detected. In the terminal stance phase, the ankle extends up to 10 degrees in the dorsal direction [irection . A rotatThe parameters Toe Off: The pre-swing phase, which follows after the Heel Off, causes a plantar flexion of around 20° in the ankle, followed by reduction of the flexion to around 5° in the initial swing phase and a dorsal extension of the ankle in the middle swing [le swing . This mothe foot , indicatn denotes the current sample, and k denotes a reasonable amount of passed samples. For the Toe Off detection, the conditions (11), (12), and (13) are used and have to be satisfied sequentiallyThis motion sequence leads to three conditions:Detection of the gait cycle: For adequate timing of the stimulation, it is crucial that the gait events are detected sequentially. Therefore, they must be passed according to the finite-state diagram depicted in Throughout the measuring chain, various errors can occur, most of them resulting in missing data points , false detection of gait events caused by outliers , and badly calibrated or defect IMU sensors resulting in erroneous data. These possible errors must be intercepted by the algorithm in order to make the stimulation safe and avoid unwanted stimulation patterns. For error handling, different approaches are implemented. One approach is the previously mentioned finite-state model . This apDifferent speeds of the orthosis require adaptive thresholds. Therefore, an angular velocity-dependent adaptation is introduced. To realize that, the maximum angular velocity during Toe Off is detected. Depending on the maximum angular velocity, the temporal dependencies from 3) during an Initial Contact, resulting in lower thresholds and different threshold adaption. Another adaption was made for the temporal conditions where the minimum swing time and the minimum roll time were adjusted to the specific needs of the robot. Due to the characteristics of the linear acceleration, the resulting jerk, and the angular velocity data of the Lyra, there was the need to implement an additional error handling condition for the Initial Contact detection. In Both systems (Lokomat and Lyra) claim to provide a physiological gait pattern. Nevertheless, when having a look at the linear acceleration and angular velocity data of the sensors located on a distinct position , the data differ between the systems. This results in robot-assisted gait trainer specific adaptions of the algorithm. In these particular cases, the differences can be identified in various events. The first adaption was done at the Initial Contact detection where the Lyra provides a much lower jerk will be used.This section presents the results of the developed algorithm. Well-defined shape of the angular velocity of the 2, which is caused by the gravitational fieldAcceleration in the Acceleration in the Arbitrary sensor alignment permits arbitrary mounting of the IMUs, which is crucial for the simplification of the usability, allowing the therapist to focus more on the individual therapy. To make sure that the algorithm works properly, the following properties of the (rotated) sensor data must be satisfied:As the rotation around the Raw data acquisition: To avoid synchronization issues, two sensors at the ipsilateral side were used. One sensor was tightly attached to the foot between the calcaneus and the cuboid bone, representing the axis of the foot (reference position); for that purpose, a fixation strap from the company HASOMED GmbH was used : As the distinct shape of the signal should be compared, regardless of the amplitude, the signals need to be normalized, taking the following aspects into consideration:Removing of the offset by subtracting the mean value of the signalNormalization of the amplitude with the following calculation (written as MATLAB code):This normalization leads to the fact that the result of the autocorrelation of the signals equals 1, indicating a similarity of 100% and providing a base for comparison.Normalized signals (angular velocity:): For better comparability, a single gait cycle of the reference position and the rotated signal was used. Results (angular velocity:): Based on the normalized angular velocity signals, a cross-correlation was calculated for 20 gait cycles. This allowed to compare the shape of the signal for each step individually, and thus verifies whether the rotation was successful. This comparison was made in order to verify the functionality of the arbitrary sensor alignment algorithm before the recording sessions (sessions  were exeResults (acceleration): For the acceleration, all the resting phases within the above-mentioned 20 gait cycles were considered. The resting phase was chosen as it is the only phase during the gait cycle where an assumption of the acceleration of all three axes can be made. During an ideal resting phase, the rotated 2 . The other two axes should ideally be 0, as under perfect conditions, the IMU does not move during a resting position. As the measurements were executed under normal conditions and can thus not fulfill ideal conditions, small deviations  are considered as normal. To check whether the rotation was successful, the mean value and the corresponding standard deviation of the resting phases were calculated  represent typical velocities used for rehabilitation. During the Lokomat training, the movement was recorded using IMUs . The IMUThis approach was used in order to generate a repeatable and comparable analyzing process. It was executed once for each recording session. The start of the windows was chosen to be a resting phase of a gait cycle; the according end of the window was determined by the size of the chosen window. The detection rate represents the correctly detected steps and is calculated as follows:The detection rate in Equation (15) was calculated for each window by determining the amount of correctly detected steps of the algorithm (Type-1 errors  represenThe type-1 error in Equation (16) was calculated for each window by determining the amount of incorrect detected steps of the algorithm (The Lyra measurements were executed at the NRZ rehabilitation center in Magdeburg. The foot support of the end-effector based system and the body weight support system were adjusted and checked by two specially trained physiotherapists; thus, an optimal training setup was provided. Three measurements were carried out, and executed by the same 39-year-old male adult as during the Lokomat measurements. Each training had a duration of 60 min, including setup time. The same velocities as during the Lokomat training were tested, and the same approach as stated in Similar to the analysis of the Lokomat measurements, the type-1 errors of the Lyra system  represenThe discussion section aims to evaluate the results of 2 (The arbitrary sensor alignment is an essential part of the gait detection algorithm. It is one of the main components to guarantee an easy setup, as the clinician who will be using the technique in future does not have to stick to a pre-defined position of the sensor. Additionally, the arbitrary sensor alignment is one of the first computations that the algorithm executes. This leads to the fact that all the other calculations are dependent on a correctly executed rotation of the sensor data. As the angular velocity in the .81 m/s2 . Based oIn order to evaluate the results of the Lokomat measurements, the detection rate  and the The detection rate and the type-1 errors of the Lyra measurements can be seen in Based on the results discussed in the previous sections, a system using inertial measurement units for gait event detection of robot-assisted gait training is feasible; thus, a system as conceptually shown in Robot-assisted gait trainers detect the patient output . PatientThe described concept offers the possibility of stimulating the muscles according to the most important gait events . As a reThe presented algorithm works as intended; nevertheless, it is important to mention that no validation with gold standards has been done. Despite some limitations, the presented concept offers a novel and feasible approach that could enhance the rehabilitation process of robot-assisted gait training with functional electrical stimulation.


Classification of electromyographic signals has a wide range of applications, from clinical diagnosis of different muscular diseases to biomedical engineering, where their use as input for the control of prosthetic devices has become a hot topic of research. The challenge of classifying these signals relies on the accuracy of the proposed algorithm and the possibility of its implementation in hardware. This paper considers the problem of electromyography signal classification, solved with the proposed signal processing and feature extraction stages, with the focus lying on the signal model and time domain characteristics for better classification accuracy. The proposal considers a simple preprocessing technique that produces signals suitable for feature extraction and the Burg reflection coefficients to form learning and classification patterns. These coefficients yield a competitive classification rate compared to the time domain features used. Sometimes, the feature extraction from electromyographic signals has shown that the procedure can omit less useful traits for machine learning models. Using feature selection algorithms provides a higher classification performance with as few traits as possible. The algorithms achieved a high classification rate up to 100% with low pattern dimensionality, with other kinds of uncorrelated attributes for hand movement identification. Electromyography (EMG) is an electrodiagnostic medical procedure to assess the health of muscles and the nerve cells that control them, with the detection, recording, and analysis of surface electromyography signals (sEMG) . EMG proMost EMG applications involve real-time systems, which need to run with low-cost computational features . As a maSeveral authors used autoregressive models and the characteristics of random processes, such as first and second moments, etc., in tasks related to the classification of myopathy or neuropathy . For exaIn a different research work dedicated to hand movement, Phinyomark et al. reported a high classification rate of 97.76%, achieved by applying a quadratic discriminant analysis and four AR coefficients per channel, including a preprocessing stage whose output was the first differentiation of sEMG . They exLiu et al. described the use of a support vector machine (SVM) ensemble  to classThe method of Khezri et al. used an adaptive neuro-fuzzy inference system to test its classification rate in a six-hand movement dataset containing four channels . The conRuangpaisarn et al. presented a feature extraction technique for hand movement classification, considering two pairs of EMG electrodes and the merging and transformation of both channels into a squared matrix to perform factorization via singular-value decomposition . They reZhai et al. suggests a self-recalibrating classifier for hand movement, based on a convolutional neural network , where tThe aim of this work is to develop a classification algorithm for sEMG with low computational cost and with a competitive classification rate. The remainder of this paper is presented as follows: This section provides a complete description of the dataset of hand movements used for the training and validation of the proposed algorithm, as well as the mathematical background for the preprocessing of the signal and its classification by the Burg reflection coefficients.We used an EMG dataset from the University of California at Irvine (UCI) machine learning repository, the same as in ,30. The As described in , EMG datN is the total number of samples. The application of the mean (1) implies a new sample value, which is described as Before feature extraction, a simple preprocessing treatment was applied to each signal. In the first preprocessing stage, the method eliminated the initial samples, where muscle activation was absent and only noise was present, so that the feature extraction did not lack the information about the phenomena. The next step comprised the extraction of the signal mean value at all data points; this operation is important to comply with the restrictions imposed by the optimal linear filtering theory . The linThese two conditioning steps have a linear complexity and supply the feature extraction stage with an appropriate sEMG.The integrated EMG feature is defined as the cumulative addition of each signal sample absolute value:The simple squared integration feature describes the energy of sEMG and is mathematically defined as cumulative addition of the absolute squared value of each sample:A stochastic process, such as sEMG, can be defined by its first and second order moments, i.e., the mean and variance values. Therefore, these features might be part of the pattern. The mathematical definition for the variance considers that sEMG is a near to zero mean process, so its definition becomes:The root mean squared value (RMS) reveals the information about the amount of strength yielded by a muscle and is defined as the square root of the mean squared values. In many research works, this attribute is considered important for different tasks:The wavelength is the distance between a pair of adjacent samples along all sEMG:The zero-crossing feature describes the number of times that the sEMG amplitude becomes positive or negative. Its definition considers a threshold, whose aim is to count only the events produced by muscular activity:The slope sign attribute considers three adjacent samples to determine the number of times that a slope sign between these sEMG values changes:The quantity of motor unit action potential is estimated through the Willison amplitude by counting the number of times that two adjacent samples overcome a threshold, reducing artifacts produced by noise:The amount of muscular pulses is described by the log detector, which uses a threshold to avoid noisy samples. p coefficients Burg autoregressive coefficients (Equation ): Arb = Reflection coefficients: K = [KClassification models involved in this research work were the Bayesian, ool WEKA  and wereThe classification algorithms were trained once, and the performance was obtained by K-fold cross-validation with a K value of 10, because it has been widely used in related state-of-the-art works and the datasets lacked class unbalance. Moreover, each instance took part in the training and testing set for a single run of the learning algorithm. Burg autoregressive coefficients  were choA high dimensionality feature vector sometimes implies having redundant or irrelevant traits that can affect the model training step because it is harder to find classification boundaries in a large dimensionality space; therefore, the output model would have a low performance. This phenomenon is known as “the curse of dimensionality”, and it is commonly faced in some feature selection methods.X, it reaches a dimensionality of 60. There is a possibility that some of these traits are redundant or irrelevant and that the dimensionality space can be reduced by applying feature selection techniques.Feature Selection guarantees a reduction in dimensionality with or without degradation of the classifier model’s performance. As observed in Principal component analysis  producesSubset Evaluation generates an original trait subset whose features are considered the most relevant because they are highly correlated with the class and have low intercorrelation . The resl traits and removing the remaining r features in such way that the classification performance remains high. If the exclusion of an attribute causes a lower performance than achieved prior, the removed sEMG characteristic is returned to the feature vector because it is deemed helpful in class instance assignment. This process is repeated until the dimensionality cannot be reduced without affecting the classification rate. “Plus l-Take Away r” approach  is basedFeature Reduction process start by finding redundant time domain energy and counting event-related attributes, followed by Burg autoregressive and reflection coefficients. The reference classification performance is the highest when obtained by the instance-based classifier (IBk) in this case study.r increases by one and l decreases by the same rate, so that IEMG was removed; through Iteration 2–3, MAV and SSI were deleted. After this algorithm was repeated 20 times, removing any remaining attribute in the last row implied that the desired performance decreased; therefore, the removal process was stopped.From The resulting dataset was built from 10 features per channel, and 20 attributes were removed because they were not useful to build and maintain the performance of the IBk model.Forward Selection method starts with an empty feature set, and attributes are added in such a way that the classification performance increases . If addiP denotes the total population.Different classification models were trained with TD, Arb, K, and a combination of these datasets. In the experiments, the implementation of the classifiers in WEKA software was done. The classification results were evaluated in terms of the classification accuracy rate:k in the IBk classifier were k = 1 and k = number for classes k = 6 would cause a tie among the six classes; the extra value will establish the majority class. This section provides the results of the classification of the movement of the hand with separate datasets. The modified parameter in the SVM model was the kernel function. For the rest of the models, the default WEKA parameters were not modified. Two values chosen for One can observe from K and Arb. In particular, with this dataset, the radial kernel SVM yielded the lowest performance in conjunction with the Bayesian models; on the other hand, the remaining models (IBk and MLP) reached competitive classification rates. The evaluation metrics in the remaining graph among models trained with TD features show that random forest had the highest weighted area under the ROC curve (WAUC); IB1 had the biggest sensitivity; and regarding specificity, IB1, random forest, and polynomial SVM obtained the same value of 0.986. From the accuracy rate graph in K of the reflection coefficients, classifying 93.55% of instances using IBk IBk was more appropriate to classify the dataset built only with maximal entropy autoregressive coefficients (Arb column) considering seven neighbors more than just one. Other learning algorithms such as MLP and decision trees offered competitive classification rates (78–90.77%); Bayesian models outperformed linear kernel support vector machines with a performance of 63.33%. The best performance among the different datasets was obtained using the reflection coefficient dataset K, and the Burg autoregressive coefficients, Arb, yielded patterns that were best classified by the IBk model with a k value of one, slightly above the IBk using In This subsection describes the results of dimensionality reduction trying to reach a higher classification performance with as few traits as possible. After running the feature selection algorithms, all instances continued to be classified correctly with 20 and 26 features with the nearest neighbors and support vector machine models, respectively see .k = 1 was still the highest and SVM with any kernel the lowest in performance. Datasets SE traits exclude amplitude-related values such as RMS, MAV, and so on. They provided high classification rates with the exception of the linear kernel SVM. The PC dataset was built with the combination of the less correlated features, resulting in a more uniform performance through all tested classifiers, reaching a 100% performance rate using the SVM with a radial kernel. The The classification accuracy rate of different learning algorithms depends on the data distribution. This behavior is expected according to the “no-free-lunch theorems” , which sK (K traits are needed to compute the Burg autoregressive traits, Arb (While testing classifiers with separate datasets, Burg reflection coefficients K  are moreits, Arb , their cDespite the Burg maximal entropy autoregressive and reflection coefficients being closely related, the classification rate increases when these two traits take a part of the dataset. This means that they are not redundant or irrelevant in feature construction tasks; however, as mentioned before, Arb characteristics take more time to compute because they rely on first computing the reflection coefficients. In addition, as a result of combining three different theoretical frameworks’ attributes, the highest classification rate was obtained. Hence, a synergy of different attributes is needed for a higher accuracy in the sEMG classification tasks.X had at least 34 redundant or irrelevant traits, which needed to be removed using feature selection tools, can explain this. As a result, different data distributions were obtained; for instance, principal component analysis performed by WEKA produced a data distribution suitable for support vector machine with a radial kernel since all patterns were correctly classified.As it is shown in k. For instance, the TD and K dataset classification rates decreased as a result of increasing the value of k, contrasting what happened with the Arb patterns (see Excluding the PC dataset, the best classifier for the remaining datasets was the IBk model with a different number of neighbors used for class assignment, because the performance of the nearest neighbors-based model depended on the value of erns see . This beThe advantage of IBk is its simple training phase, which is based directly on the dataset, compared to the Bayesian models, which require the computation of probability distributions and the cost function. The design of the hidden layer of MLP can be complex, and the time for reducing back propagation error might be long. The decision trees, as well as multilayer perceptron are hard to design, and they require a pruning process to reduce irrelevant leaves and branches, while SVM has a complexity of K features, an improvement of 0.11% was achieved. This is a sign that most of the support vectors were found in the TD traits using such a kernel. Another example is comprised of the Arb and K traits: they were better classified with a linear kernel, and by joining these two datasets, a considerable increase in the classification rate was obtained (see An example of kernel selection can be seen from ined see  as a conThe results that are considered for the discussion are those that were obtained using the same signal dataset; otherwise, any comparison concerning the feature extraction and preprocessing stage based on the classification performance would be unfair.The feature extraction method presented in  yielded As shown in , decompoThe results of hand movement classification of myoelectric signals using different traits have been presented. The Burg linear autoregressive model yielded the reflection coefficients that were shown to be useful for sEMG classification; these traits showed a higher performance that was increased with the standard time domain features and was less complex to compute than the autoregressive coefficients.A high classification rate was reached using a simple preprocessing stage, which fit the signals to the theoretical framework in the feature extraction tools. Low pattern dimensionality resulted from removing redundant traits, building patterns with uncorrelated features such as Burg maximal entropy reflection and autoregressive coefficients, and considering different time domain features related to the signal energy and to event counting. Despite a lower feature dimensionality, a classification rate of up to 100% has been achieved separately and with other kinds of uncorrelated attributes for the hand movement identification.The applications of the classification technique for signals presented in this work can lead to the possible development of state-of-the-art active prosthetic devices, where the myoelectric classification does not represent a challenge and the implementation of the classification algorithm can be performed in an integrated low-cost device.


Pooling layer in Convolutional Neural Networks (CNNs) is designed to reduce dimensions and computational complexity. Unfortunately, CNN is easily disturbed by noise in images when extracting features from input images. The traditional pooling layer directly samples the input feature maps without considering whether they are affected by noise, which brings about accumulated noise in the subsequent feature maps as well as undesirable network outputs. To address this issue, a robust Local Binary Pattern (LBP) Guiding Pooling (G-RLBP) mechanism is proposed in this paper to down sample the input feature maps and lower the noise impact simultaneously. The proposed G-RLBP method calculates the weighted average of all pixels in the sliding window of this pooling layer as the final results based on their corresponding probabilities of being affected by noise, thus lowers the noise impact from input images at the first several layers of the CNNs. The experimental results show that the carefully designed G-RLBP layer can successfully lower the noise impact and improve the recognition rates of the CNN models over the traditional pooling layer. The performance gain of the G-RLBP is quite remarkable when the images are severely affected by noise. Nowadays, using deep learning architectures to dig out information and extract features from images have drawn a lot of attention in computer vision and machine learning tasks. Among them, CNN has gradually become the most effective method since it can extract essential features quickly from images and has been widely applied in face recognition, target tracking, expression analysis, and other fields. For instance, in 2014, the Deepface  and DeepCNN is effective for visual recognition, but sometimes it is also very susceptible to the noise injected into the input images in the real-world applications. Taking Alexnet  and ZF-5A carefully designed CNN is mainly comprised of three types of layers: convolutional layer, pooling layer, and fully-connected layer. In general terms, the objective of pooling is to transform the common feature representations into a new, more usable one which preserves essential information while discarding irrelevant details . HoweverThere are many effective hand-crafted methods to extract features from images. For instance, HOG  construcThe remainder of this paper is organized as follows. Convolutional Feature Maps are the outputs of the first convolutional layer of the network.RLBP Feature Maps are the robust LBP coding results of the convolutional feature maps.RLBP Weight Maps are the weights of each pixel in the sliding window according to the RLBP feature maps.In the traditional CNNs, the convolutional layer, pooling layer and fully-connected layer care little about the noise injected into the input images. However, the noise impact introduced by the input images would accumulate layer by layer. When the intensity of the noise reaches a certain degree, the recognition rate of the network will drop sharply, as The values in the RLBP weight maps reflect the probability of each pixel affected by noise in the convolutional feature maps. Utilizing this weight maps to down sample the convolutional feature maps, the pixels which are more likely to be affected by noise would be assigned smaller weights to lower the noise interference to the networks. We give a detailed description of this RLBP weight maps in the next section.LBP is a very effective method to extract local texture features from images. In recent years, LBP and its variants have been successfully applied to various pattern recognition tasks, such as texture analysis, face detection, facial expression recognition and so on.Ahonen et al.  firstly In this section, we utilize a noise-robust LBP coding method called RLBP in which we can judge whether the pixel value of the image is affected by noise before feeding into the network based on its probability.P-bits binary string . Finally, this P-bits binary string is converted to a decimal number between 0 and The basic LBP algorithm encodes the signs of the pixel differences between the central pixel and its neighboring pixels in a sliding window. The coding criterion is as follows:b(zp)=1,iThere are patterns . Thus, upatterns .u) according to Equation  of salt and pepper noise into the samples of the ORL database [d is 0.05, 0.1 and 0.15, the proportion of the non-uniform mode is 27.86%, 31.46% and 34.49%, respectively. Based on this observation, we can preset the value of u in different coding patterns.The RLBP is different from LBP in which Equation .(2)bURLBIf R indicates the radius of the current sliding window and P is the neighbor number of the central pixel.Finally, the pattern with the max probability is regarded as the RLBP value of the central pixel.In conclusion, the RLBP value of the central pixel in any sliding windows can be summarized as follows:Case 1: If the center pixel of the sliding window is encoded as only one uniform pattern according to Equation , the corCase 2: If the center pixel of the sliding window is encoded as more than one uniform patterns, then the probabilities of each uncertain RLBP in the Case 3: If all the uncertain RLBP patterns belong to the non-uniform, the RLBP weight of this central pixel is set to be 0.We have given a detailed discussion of the RLBP coding process. There are three cases to be considered to generate the RLBP weight maps in our G-RLBP method.N is the pixel number in a sliding window, and N is set to 9. y is the output of the current sliding window after the G-RLBP pooling.The Alexnet, ZF-5net, and GoogleNet are three baseline network architectures studied in the experiments. The specific configurations of the Alexnet and ZF-5net are shown in In the recognition stage, the nearest neighbor classifier is introduced to calculate the distance between two feature vectors with three different distance measures: the Chi-Square distance, the Euclidean distance, and the Cosine distance. During the experiment, the AR and ORL databases were injected with different intensity of Gaussian noise and salt and pepper noise to test the performance of the G-RLBP pooling method.We can see clearly in The first set of experiments were carried out on the ORL database which contains 40 subjects with 10 images for each subject. One image of each subject was selected as the training set and the rest as the testing set per turn. The experiment was repeated 10 times so that each image could be used as the training set for evaluation. Before the testing images were fed into the network, we injected different intensity of Gaussian noise and salt and pepper noise to evaluate the performance of the G-RLBP pooling layer.Comparing In d = 0 in The results in Finally, we also evaluated the time cost of these network models both at the stage of training and classification, as In The second set of experiments were conducted on the AR database which contains 126 subjects with 13 images for each subject. The experimental settings were the same as Experiment 1. We chose one image from each subject as the training set per turn. Finally, all recognition rates were averaged as the final results. The AR database is more challenging than the ORL database with more subjects and some uncontrolled conditions.A face image in the AR database is selected randomly, and the image was fed into the networks . The visIt is evident that the edges and other texture information are severely affected in We also quantitatively analyzed the recognition rates of these network models  when theGoogleNet is another effective visual recognition model which has been used in many recognition fields such face recognition, image classification, target tracking and so on. Here, we conduct some experiments on this model to further evaluate our pooling layer. We use the entire CASIA-WebFace database to train our models. For data augmentation, the training data were injected with different intensity of Gaussian noise. The others experimental settings were the same as Experiments 1 and 2. The testing data were chosen from the ORL database and the AR database, respectively. Before the images were fed into the networks, they were injected with different intensity of Gaussian noise. We can see that the average filter and the BM3D algorithm are both effective when there is some slight noise injected in the testing images  With the robust LBP guiding, each pixel in the input feature maps is assigned with a different weight based on the probability affected by noise. In this way, the proposed G-RLBP can successfully remove the pixels which are likely to be affected by noise and then calculate the weighted average of the rest pixels as the final results to get more noise-robust features. (2) The proposed pooling method can extract more discriminative information from the feature maps and preserve more crucial edges of the face images during the down-sampling. The experimental results in References


Recent electrophysiological observations related to saccadic eye movements in rhesus monkeys, suggest a prediction of the sensory consequences of movement in the Purkinje cell layer of the cerebellar oculomotor vermis (OMV). A definite encoding of real-time motion of the eye has been observed in simple-spike responses of the combined burst-pause Purkinje cell populations, organized based upon their complex-spike directional tuning. However, the underlying control mechanisms that could lead to such action encoding are still unclear. We propose a saccade control model, with emphasis on the structure of the OMV and its interaction with the extra-cerebellar components. In the simulated bilateral organization of the OMV, each caudal fastigial nucleus is arranged to receive incoming projections from combined burst-pause Purkinje cell populations. The OMV, through the caudal fastigial nuclei, interacts with the brainstem to provide adaptive saccade gain corrections that minimize the visual error in reaching a given target location. The simulation results corroborate the experimental Purkinje cell population activity patterns and their relation with saccade kinematic metrics. The Purkinje layer activity that emerges from the proposed organization, precisely predicted the speed of the eye at different target eccentricities. Simulated granular layer activity suggests no separate dynamics with respect to shaping the bilateral Purkine layer activity. We further examine the validity of the simulated OMV in maintaining the accuracy of saccadic eye movements in the presence of signal dependent variabilities, that can occur in extra-cerebellar pathways. Saccades are rapid eye movements, observed in primates, carried out to bring a selected spatial target into the center of the fovea. The saccadic eye movements are executed in the absence of visual feedback information, suggesting a contribution of internal estimates (that predict sensorimotor consequences) in generating accurate eye movements  information . The third pathway involves inhibitory connections from the omnipause neurons (OPNs), that acts as static hold for the oculomotor plant until it receives a drive trigger from the SC region to initiate eye movement. The MLBNs are driven by dynamic motor error, which is the difference between a desired target displacement and current eye displacement. An estimate of this eye displacement could be considered to be derived from integrating the velocity commands generated in the MLBNs . Several closely related interpretations have explained the nature of this inaccurate eye position estimate , with random inhibitory interconnections , cerebellar output (yc), and the displacement integrator that performs imperfect integration of its own activity (u) over time, as presented in Equation (1). The distributed connections of excitatory and inhibitory burst neurons  in the brainstem are simplified in a grouped control block, for emulating the average burst behaviors observed during the inactivation of cerebellar compensation for the saccade control. The net output from the brain stem is devised to represent the burst properties of the involved neurons as presented in Equation (1), similar to the approach followed in Van Gisbergen et al.  represents the magnitude of the burst at time t, A is the amplitude of the burst and fixed to a value of 1, 100Hz. The parameter k varies the end-point error in the absence of cerebellum. k = 0 results in continuous increase in the eye position, and k = 1 causes the eye to reach the target displacement yd with significantly lower speed. The intermediate values k ∈  result in non-zero reaching error with no control in the eye speed. For the monkey oculomotor system, the value of k is considered to be a fixed parameter equal to 0.72 and σ is fixed at a value 16 , in the MF projections onto the OMV. The MFburst activity is simulated as feedback from the brainstem burst generator during the movement.On the other hand, the progress of eye movement information to the OMV does not need to come directly from the visual or proprioceptive sensors. Indeed, the experimental results on monkeys in Lewis et al. , show anm(t) = (utarget(t), uburst(t)) at a given time t. This information should be used by the OMV to compute the necessary compensations, yc as depicted in Figure 6. These MF afferents excite the downstream granule cells whose activity is computed as follows:The overall mossy fiber afferents carry the activity z(t) is the vector of N granule cell activities at time t, S(t) is the net synaptic projection on to each granule cell at time t given bywhere, τ is the time constant, wmf−GrC represent the vector strengths of N mossy fiber projections onto the granular layer, w represents the N × N matrix of connection strengths in the granule layer. The weights w are strictly positive so that the term −ρwz(t) is always negative, and hence results in inhibitory connection as represented by Equation (3). Separate populations of Golgi inter-neurons are not included in this model. Moreover, both wmf−GrC and w do not contain any manually set values of connection weights, but are a set as sparse and random distribution for having a stable granule layer activity. f is considered to be a sigmoid function. An additional parameter ρ referred as the spectral radius, is used to limit the maximum weights in the granule layer to ensure the stability of the granular layer dynamics. As the exact connectivity patterns in the granule layer are not yet clear from experimental recordings, this sparse-random connectivity should serve as a reasonable and minimalistic assumption for gain and timing control studies  is transmitted by means of PF connections (p) to the PCs, with wpf−pc connection strength vector of length N. The important concept underlying the objectives of this work is the sorting of burst-pause PC populations onto bilateral cFN, so that they can be enabled to be separately tuned for the directions of task space error. In experimental terminology, the PC units that show maximum firing for a given error direction can be labeled as CS-on in that particular direction, and the opposite direction (at 180°) as CS-off . wmf−cFN and wpc−cFN are constant, the PC populations are responsible for the driving and braking commands in the opposite manner as that of their respective cFN. Overall, the net contribution by the bilateral cFNs is considered as follows,The negative sign on the PC contribution to cFN activity in the above equation represents the inhibitory projections of PCs onto the cFN. Bilateral cFN activities described in the above equation affect the total motor command delivered to the oculomotor system through their projections onto the MLBNs. The contralateral cFN adds to the saccade command during the beginning of the movement by its excitation of the ipsilateral MLBNs. The ipsilateral cFN provides a braking signal at the end by means of a late burst and subsequent excitation of the contralateral MLBNs  is the gaze error between desired eye displacement (yd) and the original eye displacement (ye) at time t, wpf−pc represent the adaptive synaptic connection strengths of the bilateral OMV, and γ represents the relative significance of weight regularization compared to the gaze accuracy represented by the first term.where r1 and r2 can result in different types of cFN responses—for example: only burst, only pause, burst before pause, or pause before burst responses. The exact kind of responses elicited are dependent upon the physiological constraints on the PCs and cFNs, such as the relative distributions of each type of cells, and their discharge properties. These additional physiological details can be added as constraints on the cost function depicted in Equation (7).The PC activity for a given MF input can be modulated by changing the PF-PC connection strengths. This results in downstream cFN activity modulation. In light of different contributions of the bilateral cFNs to the ongoing motor command as given in Equation (6), it is worthy to note that considering different proportions of We present the simulation results in four parts. In the first part, we present the saccadic trajectories and how they are related with physiological evidences on monkeys. In the second part, we illustrate the emergence of PC layer activity from our model, and its predictive encoding of saccade kinematic metrics such as amplitude and speed. As described in previous sections, the simulated PC layer activity represents the total amount of inhibitory activity projecting onto each cFN, by the combined burst-pause PC populations organized by their common complex-spike/error property. We present a comparison with the experimental evidences described in Herzfeld et al. . This iswpf−pccontra and wpf−pcipsi. The eye plant with the orbital tissue is approximated by a second order plant of the form,The simulations are carried out to have adaptive weight updates at the PF-PC connections t) is the eye position and m(t) is the net command or motor neuron firing rate imparted to the eye plant at time t. k1, k2, and k3 are the plant constants and set to the values 0.003 and 0.6 and 4, respectively. All the parameters considered for simulations are presented in Table direct and indirect pathways, resulting in specific eye movement toward the given target locations. These target locations are randomly chosen horizontal eye displacements between 4 and 20°. The endpoint error information obtained at the end of each trial is used to update the modifiable weights in the control loop, using fmincon optimization toolbox in MATLAB/SIMULINK. This toolbox uses finite difference based search for estimating the value of gradients at each iteration of optimization, hence alleviating the requirement to supply analytic incremental gradient update rule . Each adaptation trial is run for T = 0.4s, and the weight update is carried out until there is saturation in the cumulative endpoint error. As the properties of the reservoir neurons affect the shape of the produced granular layer activities and hence the overall PC layer output, instead of considering hand-set values of granule cell time constant τ and spectral radius ρ, we input these as variable parameters in the optimization loop. The values of τ over simulation trails is obtained to be close to 20ms, and the value of ρ is obtained to be equal to 0.5. The regularization coefficient, γ, presented in Equation (7) is determined empirically and fixed at the value 0.0001, such that these values emulate the PC population activity shape derived from electrophysiological observations  are presented in Figure Figure t = 0s), followed by relatively milder dip in the activity below baseline, that continues until the MF activity persists. The size of the PC layer activity is observed to increase with the size of the saccade amplitude . Although, the experimental observations in Herzfeld et al. , in the simulations we observe a non-linear relationship .Furthermore, the CS-off PC layer activity can be observed to be proportional to the saccade speeds , whose intensity increases with the increase in saccade amplitude.Bilateral cFN activity has been simulated to be the summation of activities from the PC populations, and the excitatory MF projections from the burst generating units. The resultant cFN activity is depicted in Figures The experimental recordings on individual cFNs , yielded similar PC and cFN layer activity patterns as that of the bilateral granular layer configuration. The bilateral OMV outputs could, in principle, emerge from similar dynamics of the granular unit activities, by means of different PF-PC and MFburst-cFN synaptic strengths. This indicates that direction specific dynamics is not necessary in the granular layer. As the granular layer is composed of multiple cell types with specific roles and as few recordings are available, it is difficult to verify this argument.Figure Having simulated the model to generate sensory corrections, we chose to examine if the same model can explain the variabilities in eye trajectories observed during primate saccade adaptation experiments. In Xu-Wilson et al. , the autA, mentioned in internal feedback loop (see Equation 1). We perform simulation for the cases of an intact OMV that actively takes in the MF inputs and computes the compensations for the perceived reduction in motivation level (injected at the level of burst generating units), and a clamped OMV output which does not vary in relation to the variabilities in the burst generating units.We can simulate the reduction in saccade speeds in our model setup, by reducing the burst amplitude of the burst generating units, ms (indicated by the dotted line), in the intact OMV trial. The same pronounced compensation at a later time cannot be seen in the clamped OMV output case, resulting in saccade dysmetria. It is important to note that, for these variability trials no further learning is required in the OMV. The previously learnt synaptic strength configuration is sufficient to provide compensation for variabilities. To do this, the OMV requires only information from the MFburst projections.Figure The principal outcome of this work, resulting from the proposed saccade model, is the definite encoding of saccade kinematic information in the combined burst-pause PC population response converging onto a common cFN. The total inhibitory activity from the PC layer on a given cFN, displayed a definite relationship with the speeds and amplitudes of the saccadic movements. The peak population response in the PCs increased linearly with the peak eye speeds, which corroborates the experimental evidences from Herzfeld et al. . While ewpf−pc connection strength adaptation inorder to associate different motivational states of the primate with different PC population responses. The OMV acquires forward estimation of sensory consequences as a function of target displacement, current state of the burst generator and eye position. Variabilities in the burst responses of the internal feedback loop are reflected in the information carried by mossy fibers, arising from the burst generating units. This mechanism automatically provides an input state separation, which the presented OMV organization is able exploit to generate appropriate trajectory corrections. The same compensatory mechanism was not observed in the clamped OMV simulation.Further, our simulation results indicate the possible mechanism behind the OMV, in compensating for the inter-trial variability of saccade trajectories associated with the motivational states of the primates. In the experimental studies conducted on humans  and golgi cells (GCs). Our current model is agnostic to the distinct GrC and GC populations, and considers a population of random-sparse-recurrent inhibitory connections between uniform granular layer units. This kind of granular layer computation, is inspired from the proposed computational studies in Yamazaki and Nagao , Rössertforward model). In the latter case, the cerebellum could function as an inverse model, that provides feedforward compensation in the motor space. Though the forward model function of cerebellum has been hypothesized to have computational advantages in a number of studies (Dean et al., The plastic PF-PC weights, have been modulated by a visual error teaching signal. Unlike motor tasks such as vestibulo-ocular reflex and simple eye blink conditioning, the desired output of the cerebellum from the deep cerebellar nuclei does not need to be directly available from the target information, but could be a result of an unreferenced motor control problem Harris, . HoweverThe fundamental unit of cerebellar computation can be considered to consist of a repetitive pattern of connectivity, called microzone (Dean et al., Though the implementation of timing dependent plasticity at the plastic synaptic connections (Schweighofer et al., The raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to any qualified researcher.HK, TG, EF, and CL conceived and designed the experiments and wrote the paper. HK performed the experiments and analyzed the data.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


In this study, a preload monitoring method using impedance signatures obtained from a piezoelectric-based smart interface is presented for bolted girder connections. Firstly, the background theory of the piezoelectric-based smart interface and its implementation into the health monitoring of bolted connections are outlined. A simplified electro-mechanical (EM) impedance model of a smart interface-embedded bolted connection system is formulated to interpret a mechanistic understanding of the EM impedance signatures under the effect of bolt preload. Secondly, finite element modeling of a bolted connection is carried out to show the numerical feasibility of the presented method, and to predetermine the sensitive frequency band of the impedance signatures. Finally, impedance measurements are conducted on a lab-scaled bolted girder connection, to verify the predetermined sensitive frequency range and to assess the bolt preload changes in the test structure. Bolting is a widely-accepted method for making connections in steel structures in the field. Bolts are torqued to a high tensile stress, developing clamping pressures at the interfaces of the structural members to hold them in position. Followed by the use of high-strength bolts, this fastening method has enabled the advantages of easy installation, time efficiency, and a high strength for field connections. After a long-term service life, however, the bolted connections could experience a loss of preloads  due to repetitive external forces and vibrations, which threaten their functionality. Therefore, bolt preload monitoring is essential and recently gained growing interest in efforts to ensure the safety of bolted joints, and to prevent the catastrophic failures of the entire structures ,3,4,5,6.To assess the structural integrity of the local critical members in the mechanical and civil systems, there have been many research attempts on the impedance-based method ,11,12,13Bolt-loosening in a bolted joint can be monitored via its EM impedance responses, measured by piezoelectric sensors or piezoelectric washers ,20,21,22The mountable interface technique can be a potential solution to cope with the above-mentioned problem . This teIn this study, a PZT interface-based impedance monitoring method is developed to detect the bolt-loosening events in a bolted connection. To demonstrate the theoretical feasibility of the presented method, a simplified impedance model is newly designed with the consideration of the bolt preload effect. Next, the sensitive frequency band of the EM impedance responses is numerically predetermined for a bolted connection example embedded with a PZT interface. Finally, impedance measurements are conducted on a lab-scaled bolted girder connection to verify the pre-analyzed sensitive frequency band and to evaluate the effectiveness of the proposed method for bolt-loosening detection. An impedance monitoring method using the PZT interface technique is designed in order to acquire the impedance data with predetermined sensitive frequency bands from bolted connections. As shown in To monitor multiple bolts in a connection, the PZT interface should be mounted to the splice plate connection, which is clamped by the bolt preloads, as shown in ck, cc), whose values represent the amount of bolt preloads, see im, ik, and ic) and the splice plate can be also modeled by the respectively structural parameters . When the bolt preloads are changed, the contact parameters of the connection are altered , leading to the variation in the coupled responses of the system at resonance. By monitoring the impedance responses of the system in the resonant band, it is possible to detect the bolt looseness or preload changes that have occurred in the connection.In equilibrium, the bolt preloads can be transformed into contact pressures and bearing stresses at the contact between the main structure and the splice plate, as seen in im, ic, and ik), and the other dof refers to the splice plate  with the contact parameters (cc and ck).The impedance responses of a bolted connection measured via the PZT interface can be theoretically derived from a simplified impedance model. Based on the previous studies ,30, a twV(ω), with a current, I(ω), a harmonic force is introduced into the system at the PZT driving point. The equation of motion under the PZT’s harmonic force im and sm, respectively. As illustrated in iU and sU are complex quantities that are dependent on the excited frequency and structural parameters of the system. By substituting Equation (2) into Equation (1), equations to obtain iU and sU of the system are given as follows:Under the harmonic excitation force, the steady states of the interface and the splice plate can be described by the following: eqZ, is defined as the ratio between the excitation force if and the velocity at the PZT driving point The equivalent mechanical impedance of the system, iU, is obtained. By substituting the obtained iU into Equation (4), the equivalent mechanical impedance of the system eqZ is obtained as follows:s/kcξ = k is defined as the ratio between the splice plate’s stiffness and the contact stiffness. ξ ≈ 0 indicates the infinitive value of contact stiffness , while ξ ≈ ∞ indicates the unnoticeable value of the contact stiffness . If the splice plate’s stiffness, sk, remains unchanged, the increment of the ratio s/kcξ = k will be equivalent to the decrement of the contact stiffness, ck, which can be interpreted as the bolt preload reduction .By solving Equation (3), the quantity, Z(ω), of the bolted connection measured via the PZT interface is a combined function of the equivalent mechanical impedance of the interface-bolted connection system, eqZ, and the mechanical impedance of the PZT sensor, aZ, given by  is the expectation operation; ith frequency before and after a damage event, respectively; According to , the CCDN denotes the number of swept frequencies.As another damage-sensitive feature, the RMSD index can be obtained by  the follμ and σ are the mean and the standard deviation of the impedance feature values, respectively. The UCL determined by three standard deviations of the mean has the confidence level of 99.7%. For distinguishing the bolt-loosening state from the healthy state, an alarming threshold known as the upper control limit (UCL) can be established using the values of the impedance features under the intact state ,37, as fThe RMSD control chart was constructed using the impedance data in the predetermined frequency band of 10–50 kHz. It should be noted that the 31% torque-loss of a single bolt is equivalently corresponding to the 3.8% preload reduction in the test connection, which consists of eight bolts. This means that the impedance signatures of the PZT interface were quite sensitive to the small preload changes occurred in the bolted connection. The loosened bolts near the PZT interface  were detected with higher severity estimations than those far from the interface . These results confirmed that the single PZT interface was able to monitor multiple loosened bolts on the splice plate. For the comparison, the CCD control chart was also constructed by using the same impedance data, as shown in In this study, the piezoelectric-based smart interface technique was developed to acquire sensitive impedance signatures from a bolted connection for bolt-loosening detection. To demonstrate the theoretical feasibility of the proposed method, a simplified EM impedance model was newly designed with the consideration of the bolt preload effect. Secondly, the EM impedance’s sensitive frequency band was numerically pre-analyzed for a bolted connection via the PZT interface technique. Finally, the impedance measurements were conducted on a lab-scaled bolted girder connection to verify the pre-analyzed sensitive frequency range and to assess the preload change in the test connection.(1)The PZT interface’s sensitive frequency band, predetermined by the numerical simulation, was quite consistent with that measured from the experiment.(2)The impedance signatures obtained from the PZT interface were sensitive to the minor preload change in the bolted connection. For the tested eight-bolt connection, a 31% torque-loss of a single bolt can be detected using the PZT interface technique.(3)A single PZT interface was able to monitor multiple loosened bolts in a connection, thus reducing the number of sensing channels for the impedance monitoring of a large bolted connection.From the numerical and experimental observations, the following concluding remarks can be made: Future works will need to optimize the geometric size of the PZT interface so as to enhance the sensitivity of the impedance signatures and to quantitatively estimate the sensing area of the PZT interface technique. Also, there is a need to evaluate the presented method for the simultaneous loosening of multiple bolts.


The main hindrance to realistic models of fiber-reinforced concrete (FRC) is the local materials property variation, which does not yet reliably allow simulations at the structural level. The idea presented in this paper makes use of an existing constitutive model, but resolves the problem of localized material variation through X-ray computed tomography (CT)-based pre-processing. First, a three-point bending test of a notched beam is considered, where pre-test fiber orientations are measured using CT. A numerical model is then built with the zone subjected to progressive damage, modeled using an orthotropic damage model. To each of the finite elements within this zone, a local coordinate system is assigned, with its longitudinal direction defined by local fiber orientations. Second, the parameters of the constitutive damage model are determined through inverse analysis using load-displacement data obtained from the test. These parameters are considered to clearly explain the material behavior for any arbitrary external action and fiber orientation, for the same geometrical properties and volumetric ratio of fibers. Third, the effectiveness of the resulting model is demonstrated using a second, “control” experiment. The results of the “control” experiment analyzed in this research compare well with the model results. The ultimate strength was predicted with an error of about 6%, while the work-of-load was predicted within 4%. It demonstrates the potential of this method for accurately predicting the mechanical performance of FRC components. With recent developments in structural and material mechanics, assessments of safety margin with respect to non-linear system response and failure, instead of admissible stresses, became possible and even required by several codes ,2,3. NumAccurate numerical modeling within the non-linear regime is related to the appropriate selection of a constitutive model, capable of accounting for phenomena that are taking place at the material level . Such a constitutive model would offer a framework for the accurate modeling of a structural response in the general context, beyond the one represented by the experiment performed for its calibration. Therefore, the quantification of the parameters that govern the constitutive equations should not be merely reduced to the fitting of a single experimental response.The importance of appropriate constitutive model selection becomes more evident when a complex material, such as fiber-reinforced concrete (FRC), should be modeled. Owing to the presence of small fibers, the structural response of FRC with respect to conventional reinforced concrete is considerably different. With conventional reinforcement, significant elongation of the steel bar is required, so that it can carry tensile loads, which requires the notable opening of macro cracks within the concrete. In contrast, in FRC the cracks are often barely visible to the naked eye, and are developed in the form of a network, which gives the structural member greater ductility and, at the same time, limits the exposure of fibers to the ambient conditions ,9.In previous years, considerable research efforts have been devoted to studying the mechanical response of FRC. Significant attention has been devoted to analyzing the influence of fiber orientation on the mechanical response of structures ,11. SincThe mechanical response of the structural components made of FRC depends, to a large extent, on the local distribution and orientation of reinforcing fibers. Such information can be collected through the use of X-ray computer tomography (CT), but its effective incorporation into numerical modeling still needs to be solved. The major difficulty for successful modeling is related to the fact that the existing orthotropic constitutive damage models, which are implemented in commercial finite element codes, are suitable for defining anisotropic material behavior only at the structural level. While this can be an appropriate strategy to model conventional reinforced concrete, it is not appropriate for the FRC, where locally strengthened directions, achieved by reinforcing fibers, vary considerably within different regions of individual structural components.For reliable numerical simulations, fiber distribution and orientation should be included within the constitutive description, thus requiring multi-scale approaches with the capability of incorporating the inherent variability of the internal structure. For this purpose, discrete models ,19, can Considering the nature of the phenomena that take place on the fiber scale, a reasonable approach would be to employ a damage model. The major difficulty related to the employment of existing constitutive damage models within commercial finite element modeling (FEM) codes is that even though they can simulate either isotropic or orthotropic behavior, the orthotropic behavior can only be modeled along the directions defined at the structural level. This can be an appropriate strategy to model conventional reinforced concrete, where reinforcing bars have well established directions with respect to the structure, but is not sufficient for FRC, where locally strengthened directions change from one point to another.A detailed description of the FRC material as well as the characteristics of the CT measurements and the analysis of fiber orientation data are presented in The FRC material used during this research program is an ultra-high-performance concrete (UHPC) developed by the US Army Engineer Research and Development Center . As a reactive powder concrete mixture, this material contains no course aggregate particles and is characterized by a low water-to-cement ratio and high cement paste content ,22,23. TThe two beams analyzed in this study were nominally 220 mm long, 48 mm high, and 30 mm wide. After curing, nominally 18 mm deep by 5 mm wide “notches” were saw cut at the bottom of each beam midway along the length. Given the large aspect-ratio of the beams and the relatively small height and width dimensions relative to fiber length, it was expected that significant anisotropies in fiber orientation could occur, in particular with the fibers oriented principally in the direction of the beam length. Following CT scanning, the specimens were loaded to failure in a three-point bending configuration. Further detail about the materials, specimens, and mechanical testing was reported by Williams, Roth, Trainor, et al. ,24,25. Prior to mechanical testing, the central section of each beam was scanned using CT; for details of the specific scanning procedure and parameters . Three-dAnalysis of fiber orientation, including the compilation of fiber data into a finite element mesh, was completed using the Fiber Composite Material Analysis module of VGSTUDIO MAX . A sectiIn order to achieve a compact description of fiber orientation, a tensorial representation was adopted. Specifically, for a unit vector  product  namely:. For a group of N fibers, each is associated with a fiber unit vector pi, i=1, …N describing its orientation. The average orientation tensor is computed by taking the mean value of entries from the individual orientation tensors, calculated for each pi [The eigenvalue analysis of matrix  each pi , namely:D: By performing the eigenvalue analysis of ) theory ,29. WithD and It is easily demonstrated that matrices The developed numerical model uses two-dimensional domains. Volume cells, over which fiber orientation is analyzed, are of the size 1 mm × 1 mm in the modeled plane and have a 30 mm thickness. Fibers crossing this volume are averaged according to the above outlined procedure, and replaced by the first two components of the first eigenvector , see The averaging of fiber properties over the 30 mm cell thickness, corresponding to the width of the entire beam, is the required step for implementing the real, 3D fiber measurements into the 2D numerical model.In this paper, a fully phenomenological orthotropic damage model is adopted to model FRC . The presence of fibers contributes to the increase in load-carrying capacity of the whole specimen. This increase is manifested only along the direction of fibers, while in the perpendicular directions the response is considered to roughly correspond to that of unreinforced concrete (though possibly somewhat weaker). Therefore, to model the response employing the phenomenological constitutive model, it is appropriate to use the orthotropic mode: a model with different behavior along different, well-defined directions. The local variability of the structure, in terms of fiber orientation, is incorporated by previous pre-processing. This is achieved by dividing the portion of the sample most susceptible to progressive damage, due to formation of a crack network, into 1 mm × 1 mm 2D elements. Within each of these regions, local coordinate systems are assigned based on fiber-orientation data collected from CT measurements, averaged over the considered region. Thus, through the use of local coordinate systems, the primary direction strengthened by the reinforcement of the fibers in each individual zone is simulated. The constitutive parameters are the same for the whole sample, while local structural variability is accounted for through specific coordinate systems. This formulation results in a constitutive model capable of predicting unique global specimen behavior, since the variability at the structural level, including significant differences in overall structural response, result from changes in these local orientations. Such an approach provides the advantage that the problem is solved over one scale only, requiring the quantification of only one set of constitutive parameters, here solved on the basis of the designated inverse analysis procedure. The adopted constitutive model is typical for fiber-reinforced composites, existing in the commercial FEM code ABAQUS  . These m1E—Young’s modulus for longitudinal direction 2E—Young’s modulus for transversal direction 12ν, 21ν—Poisson’s ratios fd, md and sd—Damage variables for the longitudinal load capacity, the transversal load capacity and for shear capacity, respectively The material mechanical response is therefore governed by the following parameters:In the general version of the model, the two Young’s moduli are not the same, however within this study, the model is simplified by assuming the same value for both moduli. The damage parameters are bounded by the initial value of zero, corresponding to “virgin” material, and the maximum value of one, corresponding to fully degraded material. The calculation of each particular damage parameter value was related to the formulation of the damage initiation criterion  and the criterion by which the damage parameter evolves up to a value of one, representing complete failure. This second damage evolution criterion completely governed the post-damage behavior of the material in the model.fd) was assumed to occur once the longitudinal tensile stress exceeds the predefined value of σf, corresponding to the beginning of fiber reinforcement failure. Here, fiber reinforcement failure is taken to mean the point at which the load carrying capacity of the fibers begins to decrease. This may be due to one or more phenomena ;Damage initiation relative to the longitudinal tensile capacity  was assumed to occur once the transverse tensile stress exceeded the predefined value of σc, corresponding to the beginning of unreinforced concrete tensile failure. In other words, fibers are assumed not to contribute to strengthening in the direction perpendicular to their long axis;Damage initiation relative to the transverse tensile capacity . This initiation was assumed to occur once the compressive stress in a given direction exceeded the predefined value of cmpσ, corresponding to the beginning of unreinforced concrete compression failure. This simplified damage initiation criterion relies on the assumption that there is no significant contribution of fibers relative to compression failure; Damage initiation relative to compression capacity is defined identically for both the longitudinal and transverse load capacities (corresponding to an increase in sd) was assumed to occur once the shear stress exceeds the predefined value of SLσ or STσ, corresponding to the beginning of concrete shear failure in the longitudinal or transversal directions, respectively.Damage initiation relative to the shear capacity (corresponding to an increase in fd), transversal (md) and shear (sd) loading. In this study, however, in view of the 2D  nature of the model, a simplification was adopted by setting the value of each of these separate fracture energy parameters equal to the value of a single parameter, fG. Within the 2D model, it is not possible to model the complex crack pattern that is developed within the real, 3D sample. To compensate for this limitation of the model, it is expected that the fG parameter will be overestimated. Moreover, since it was expected that the response of the beam was to be dominated by the damage along the longitudinal direction of the fibers, the values of the fracture energy parameters for transversal and shear loading were considered of secondary importance to overall response. The effects of restricting all fracture energy parameters to a single value on the overall beam response was, consequently, considered to be negligible.It was assumed that the damage propagation was to be governed by the fracture energy dissipated up to the full degradation of the element. In the general version of the model, separate fracture energy parameters governed the damage propagation in longitudinal (The model adopted in this study (outlined above) required, besides the elastic parameters, the definition of the following parameters governing the damage: The geometry of the three-point bending specimen considered in this model was adopted , specifiIn order to take the local orientation of fibers into account, the central zone of the beam specimen was divided into 1 mm × 1 mm elements, each of these having its own unique local coordinate system. The local coordinate system of these elements was assigned such that its longitudinal  axis corresponded to the primary axis of the average orientations of the fibers within the element calculated using CT data from the specimen see . For regTo verify the capability of the proposed approach to model diverse structural responses governed by local variability of fiber distribution, the following numerical exercise was performed. Two different numerical models of three-point bending beams were generated. The fiber distributions were arbitrarily selected to produce a significantly different mechanical response. Such variations in fiber orientation and distribution are commonly seen in FRC and result directly from variation in the material flow during the casting process ,31. The Numerical model (b) assumed all the fibers to be perpendicular to the longitudinal axis of the beam, therefore providing negligible reinforcing capacity for the three-point-bending load scenario. Indeed, as shown in In the procedure implemented for this study, a three-point bending test, from which a force-displacement curve is obtained, is treated as the main experimental data. The measured fiber orientation characteristics for the beam used in this calibration are also, naturally, implemented into the numerical model and, thus, directly influence the corresponding constitutive parameter values. A discrepancy function is further constructed to quantify the difference between measured quantities and their computed counterparts . ThroughAfterwards, the assessed parameter values can be treated as material representative data and can be used for arbitrary loading scenarios. Validation of the accuracy and resilience of the model was completed using CT-based fiber orientation measurements from a second three-point bending test in combination with the assessed material parameters to predict the structural response. This is the validation step .1E (fiber direction) and 2E  were defined within the model based on experimental data, while the remaining elastic constants were considered to be a priori known values and were, correspondingly, fixed within the model. The response of the beam is dominantly governed by the damage parameters, regardless of the initial, perfectly elastic conditions. Thus, in order to reduce the number of parameters that had to be assessed, only one Young’s modulus value  was used. This simplification is reasonable, considering that the variation in Young’s modulus due to the presence of fibers has been previously reported as only about 10% of the Young’s modulus typical for unreinforced concrete [The values of Young’s modulus concrete .fσ), the damage initiation stress in tension for the transversal direction (cσ), the damage initiation stress in shearing for the longitudinal (SLσ) and for the transversal (STσ) directions. The stress value for damage initiation in compression was assumed to be the same for both the longitudinal and the transversal directions and did not have to be directly calculated from the experimental data. This damage initiation stress in compression was therefore not subjected to the identification from the experiment and was taken to be eight times the magnitude of the defined damage initiation stresses for tension in the transversal direction, based on standard tensile-to-compression strength ratios described in ACI 318 [number of damage-initiation parameters were also defined in the model based on experimental data: the damage initiation stress in tension for the longitudinal direction  algorithm. Details of the TR algorithm and its numerical implementation are available in the literature ,35,36, wpk+1) and resulted in an improved value of the discrepancy function . Constraints are provided by a “trust region”, here adopted as a circle shape within the two-dimensional sub-space.The minimization started from an initial vector of the parameters. Within each iteration, a quadratic programming problem in two variable spaces was solved, namely a sub-space spanned by the gradient direction and the second derivative direction . The solution of the constrained minimization of this sub-problem provided the modified parameter vector for the next iteration (say pk reached after the kth iteration was generated by a means of the computed gradient in that point and by means of the Hessian matrix approximated through the Jacobean (J), namely:With such a formulation, the minimization problem was solved inside the trust region, in which the quadratic approximation was “trusted” to be a reasonably good approximation of the real discrepancy function. The quadratic approximation of M+1 simulations, where M is the number of parameters.The described features of the TR algorithm clearly imply calculations of first derivatives only, numerically computed through finite differences, namely by separately varying each of the parameters. After individual simulations had been completed for variations in each separate parameter variable, a final simulation was completed with the revised values for all parameter variables applied. Therefore, the iterations contain This iterative sequence was repeated until reaching the convergence criteria, here imposed as the change in the discrepancy function value and the difference between parameter vector norms between two consecutive iterations . This was realized through an appropriate normalization of parameters with diverse orders of magnitude. The resulting parameter vector reached after a certain number of iterations represented the solution to the designed inverse problem and, therefore, also the representative material properties. As a remedy to the possible lack of convexity of the discrepancy function (7) and consequent termination in a local minimum, the TR iterative sequence was repeated starting from another initialization point. The inverse analysis procedure described in The inverse problem was found to be well-posed and the procedure converged to the solution after several iterations. The parameter values, which resulted as the solution to the inverse problem are listed in 1E and 2E. The resulting value of Young’s modulus was somewhat smaller than expected. This discrepancy is likely due to the influence of several factors on the displacement values measured during the experiments . The elastic properties, however, were not of primary interest here, since they can be measured more precisely with some alternative methods, therefore the removal of these effects was not considered. The damage initiation stresses are of the expected order of magnitude, when compared to the values reported in the literature [f is comparable to flexural strengths previously measured for this material [c corresponds well to the tensile strength of unreinforced concrete. The identification procedure converged to these values without any preconditioning. A comment should also be made here regarding the parameter fG. The crack network developed in the two-dimensional model is significantly simpler than in the real three-dimensional case. Therefore, the only way for the numerical model to simulate the experimentally observed ductility is by increasing the value of fG. This could be confirmed by comparing the obtained values of fG with the absolute values of dissipated energy corresponding to different internal mechanisms [Considering the adopted simplifications outlined in terature . The magmaterial , while σchanisms . In order to verify the accuracy of the resulting material parameters for the given sample geometry and test setup, a second sample subjected to the three-point bending experiment is further considered. The two samples were made of nominally the same material, but with fairly different fiber orientation and distribution. Another numerical model was built, in which the fiber orientation was distributed within the model following the procedure described in This research showed that with a fully phenomenological model, without detailed modeling of fibers, the ductility of the fiber-reinforced concrete could be successfully simulated. Further, it is clear that such behavior can be captured with a single material property set, while the structural variability is incorporated through localized fiber orientation. The novelty of the proposed approach lies in the combination of the phenomenologically based model construction with the inverse analysis based calibration procedure. One obvious limitation of the modeling strategy presented here is the use of a simplified two-dimensional numerical model. The extension of this approach to a three-dimensional model requires the implementation of the proposed constitutive model using the USER subroutine of the commercial software ABAQUS. Such an extension would be a desirable development, but requires further research, which is presently ongoing, and additional numerical implementation that extends the relevant functionalities of the commercial software. The results achieved and presented within this paper, however, provide a meaningful contribution by demonstrating the advantages and limitations of this modeling approach using readily available numerical tools.


Onymacris unguicularis, a beetle species living in the coastal regions of the Namibian desert, has recently caught the attention of the engineering community, as suggesting a viable biomimetic approach to address the problem of harvesting water in arid regions of the globe. Previous research has focused on observation and analysis of the beetle’s elytron properties and how these affect fog-collection rates. The head stance taken by the Onymacris unguicularis when fog basking is well documented. However, how this stance affects droplet collection has not been studied up to now. The present paper addresses this problem from a computational fluid dynamics perspective, where three-dimensional numerical simulations are used to characterize the fog flow properties around a simplified geometry mimicking the beetle’s body. The simulations employ two-way coupling between the gas flow and the dispersed fog phase to account for feedback effects of fog droplets on the carrier fluid (air), and assume that droplets are captured after hitting the elytron surface. The study considers several combinations of free-stream velocity and droplet volume fraction. The analysis reveals that there is a range of head-stance angles, corresponding to an inclination of the beetle between 35 deg and 45 deg with respect to the horizon, that maximizes water collection on the beetle’s back, in qualitative agreement with observations in nature and laboratory experiments. A rationale is proposed to explain this phenomenon, finding that the specific head stance corresponds to the maximum residence time of fluid particles above the beetle’s elytron surface. This, in turn, designates the maximum likelihood for water droplets to be captured in the boundary layer developing over the beetle and subsequently hit the surface where they get captured. The results reveal the importance of the fluid flow pattern around the beetle’s body in addition to the microphysical properties of the elytron when reliable predictions of the water droplet collection efficiency are sought.The fog-basking behavior of the  Water in the atmosphere has been an elusive freshwater resource in arid regions of the globe. In their 2001 article, Parker and Lawrence ,4. SubseOnymacris unguicularis is one of them; its dorsal surface (a.k.a. elytron) features uniform wettability [O. ungucularis beetle takes a head stance, so that the ventral side of the animal is at an angle of ∼23 deg to the horizontal. Such posture was consistently observed in experiments conducted with live beetles  using a three-dimensional Navier–Stokes solver with two-phase flow capabilities to track water droplets.Several comparative studies that evaluated fog basking among the more than 200 Namib Desert tenebrionid species, have found only two species—both Onymacris—to be fog baskers . Onymacrtability . Nørgaartability  reportedtles see . The sigreactingParcelFilmFoam, is based on an Eulerian–Lagrangian two-phase flow modeling for the continuous phase  and the dispersed phase (water droplets), respectively (see OpenFOAM documentation) https://openfoam.org/resources. The two phases are coupled through a two-way coupling procedure in which the carrier fluid and the convected droplets interact with each other via drag forces. The continuous phase behavior is governed by the Navier–Stokes equations for an incompressible fluid,reactingParcelFilmFoam implemented in OpenFOAM was used to resolve Lagrangian particle motion and for the impingement of droplets on the surface of the body. To that end, a special external mesh region was extruded from the body’s surface. For modeling of the film layer, a thin-film approximation was used: the velocity normal to the wall was zero while, along the wall, the tangential diffusion was considered to be negligible compared to normal diffusion. The surface film flow was solved using the continuity and momentum equations. The continuity equation comprised of a source term representing the mass added to the film layer due to the incoming fog droplets (All simulations were carried out in OpenFOAM (open source field operation and manipulation), an open source C++ toolbox for developing numerical solvers for various problems in the field of computational fluid dynamics . The Opeting see . Water de fluid,∇·u=02)reactingPBased on the structure of the beetle’s body, a simplified geometry was chosen consisting of a semi-ellipsoid with axes of 8 mm, 3 mm, and 3 mm, in the horizontal (−x), vertical (−y) and transversal (−z) directions, respectively. The legs were not modeled due to their much lower surface area compared to the rest of the body. The semi-ellipsoid (body hereafter) was placed in a surrounding external box (cuboid shaped with side lengths For each body inclination angle quations  and 2)  α, the nIn the second step, Lagrangian fog droplets were injected into the environment with the same free-stream velocity of the carrier air stream . In all simulations we used the patchInjection module of OpenFOAM, where sert fog , the totOne possible explanation for this intriguing behavior for the mass collection efficiency exhibiting a consistent peak around 40 deg is related to the three-dimensional aerodynamics of the flow around the semi-ellipsoid, which deflects the initially straight droplet trajectories. This, in turn, influences the velocity of droplets entering the boundary layer and depositing on the surface in a complex way that cannot be predicted solely on the basis of geometric arguments. In order to confirm this hypothesis, we analyzed droplet trajectories obtained by connecting the positions of selected droplets at various instants lane see . The aveIn the above equations, To further investigate the water collection along the body surface, we determined the film thickness ngth see  for varyimum see . The samn is the size distribution parameter. The values considered are In order to further examine the fog mass collection efficiency of the semi-ellipsoids, we compared the results for predicted mass-collection efficiency for fog with polydisperse droplet size distribution. We cast the diameter of the droplets into a Rosin–Rammler distribution, and repeat the numerical simulations with a polydispersed input flow condition in OpenFOAM see . FollowiO. unguicularis). With the help of numerical simulations, we examined the aerodynamic and geometric conditions that possibly motivate the beetle to bend forward during fog basking. By calculating the fog collection rates at varying geometrical alignments of the beetle’s body with respect to the horizontal ground, we determined that water collection is consistently maximized in the range 35 deg–45 deg inclination for all combinations of droplet velocities and volume fraction considered in this study . These results are relevant to fog basking, as they provide evidence of a beetle stance that is optimized to capture water for given body shape and incoming flow conditions. The results of this study, the first to analyze the flow around a model beetle using CFD, are encouraging and confirm, at least qualitatively, the behavior of real beetles as observed in laboratory experiments and in their natural environment. The difference between the observed head stance angle observed experimentally by others and that predicted in the present work for maximum fog collection could be attributed to the simplified assumptions used in this study. First, the microphysical properties of the surface were not considered in the present model. In addition, the details of the beetle body’s geometry, including legs, core, mouth etc., as well as the skin tessellation, wettability, etc. are properties that may also influence the effectiveness of water capture (in addition to head stance) and, indeed, represent interesting avenues for future follow-up studies. These properties could also inform possible strategies to develop materials and geometries that mimic the beetle’s fog-basking characteristics.We have presented quantitative reasoning for the fog-basking abilities of a Namib desert beetle (