The use of fiber-reinforced polymers (FRP) in civil construction applications with the near-surface mounted (NSM) method has gained considerable popularity worldwide and can produce confident strengthening and repairing systems for existing concrete structures. By using this technique, the FRP reinforcement is installed into slits cut into the concrete cover using cement mortar or epoxy as bonding materials, yielding an attractive method to strengthen concrete structures as an advantageous alternative to the external bonding of FRP sheets. However, in addition to the two conventional failure modes of concrete beams, sudden and brittle debonding failures are still likely to happen. Due to this, a damage identification technology able to identify anomalies at early stages is needed. In this work, some relevant cluster-based methods and their adaptation to electromechanical impedance (EMI)-based damage detection in NSM-FRP strengthened structures are developed and validated with experimental tests. The performance of the proposed clustering approaches and their evaluation in comparison with the experimental observations have shown a strong potential of these techniques as damage identification methodology in an especially complex problem such as NSM-FRP strengthened concrete structures. In the last few decades, strong research efforts have been devoted to the continuous structural condition assessment for the civil engineering infrastructure . StructuBy controlling and analyzing damage-sensitive features, extracted from raw experimental data on a monitored structure, it is possible to detect structural anomalies which may be due to damage. Within the field of strengthening of reinforced concrete (RC) structures with fiber-reinforced polymer (FRP) composite materials, some previous studies have been devoted to the identification of damage at their earliest stages on externally bonded (EB) FRP reinforcement. For this purpose, different non-destructive testing (NDT) methodologies such as infrared thermography ,4 and ulUntil now, to the knowledge of the authors, no previous SHM work has been experimentally focused on the near surface mounted (NSM) technique using FRP. In the NSM-FRP methodology, the FRP reinforcement is installed into slits cut into the concrete cover using cement mortar or epoxy as bonding materials. Its use has several advantages over the EB FRP technique, such as protection, improved bond, better aesthetics, and surface preparation; due to this it has become an attractive method as an effective alternative for strengthening RC structures. De Lorenzis and Teng , and morA conventional global modal analysis, such as performed in  on reinfDifferent machine learning approaches have been successfully applied for damage detection in the past ,19. DensDifferent alternatives exist to perform the clustering. In this work, some relevant cluster-based methods and their adaptation to EMI-based damage detection in NSM-FRP-strengthened structures SHM are developed. The proposed methodology is validated on datasets extracted from two RC beams strengthened with NSM-FRP experimentally tested under increasing loading until failure. The paper is organized as follows: In i and the output current I0 of the PZT transducers, is directly related to the mechanical impedance of the structure Zs(ω) and, therefore, to the growth of structural damage, as follows:a(ω), of the PZT patch although, usually, material properties of the PZT are assumed to be constant.The electromechanical impedance technique is based on the self-sensing property of piezoelectric material which allows exciting a host structure using a PZT transducer attached to it and the acquisition of the response using the same PZT transducer. As defined in the 1-D theoretical model introduced by Liang et al. , the eleAdditionally, PZT transducers excite the host structure within a high frequency range which makes EMI technique very attractive for local damage detection because of its sensitivity to even small damage ,31,32,33Different statistical metrics, such as the root mean square deviation (RMSD), the correlation coefficient (CC), the covariance (Cov), or the mean absolute percentage deviation (MAPD), have been used in the past to detect variations of the impedance signatures . These vAs commented previously, the use of the EMI method in combination with PZT transducers can become very interesting in detecting incipient damage prior to more catastrophic deterioration in NSM-strengthened RC structures due to their high frequency and localization capabilities. The use of a single type of metric  for EMI-based damage detection has some limitations which affect to the identification of damage location and severity. For instance, small damage at close range to the PZT transducer can present the same results as larger damage further away. Furthermore, evaluating quantitatively the damage degree can be very difficult using this approach. To solve some of these deficiencies, in this work, we propose the use of a novel non-model-based clustering method able to correlate the location and type of PZT transducers with their performance as well as with the level of damage of the structure to be analyzed. Basically, clustering is an unsupervised learning algorithm able to organize objects into groups according to their similarity or some recognized pattern.The proposed clustering approach in this work is mainly based on two hypotheses. Firstly, on one hand, sensors located close to each other and under the same conditions are assumed to respond similarly to the same excitation if they are of the same type. According to this, the damage detection capability will be dependent on the type of sensor and on how the sensors are integrated with the host structure since PZT transducers can be directly attached to the surface of the structure or internally embedded. Based on the previous hypothesis, nearby sensors of the same type should have similar behavior and, therefore, should be grouped into the same cluster. Any measurement identified as an outlier might be interpreted as an anomaly or symptom of damage. Secondly, by using the same previous reasoning, measurements captured from the same sensor at different stages of the inspected structure are assumed to show a similar behavior except if some anomaly has occurred. If the environmental variability on the system is controlled, the anomalies should mainly be associated to the existence of mechanical damage.According to the previous assumptions, potential anomalies can be identified by grouping clusters of similar sensors or, alternatively, clusters of similar stages for the same sensor. We think that combining both assumptions within a clustering framework, a procedure might be implemented able to increase the potential of correctly identifying and locating structural damage.Different alternatives exist to perform the clustering. In this work, we will use two, an agglomerative or hierarchical algorithm and a k-means algorithm.By using a hierarchical clustering approach, each one of the different observations is initially treated as a single cluster. Subsequently, using an iterative procedure, nearby observations are merged into a new bigger cluster; then, successively, new nearby points are added to the nearest group and so on. In this sense, the algorithm uses a bottom-up or agglomerative strategy. Alternatively, a top-down or divisive approach might be followed, i.e., all observations start in one cluster and, recursively, new splits are formed as one moves down the hierarchy. The different levels of grouping can be visualized using a dendrogram, which is a sort of connectivity plot giving the big picture of the level of similarity between adjacent clusters. By cutting the dendrogram at different heights we can decide about how many clusters we want. Although this procedure can become computationally expensive, it produces very intuitive results which help to figure out which clusters combination makes more sense.Assign each item or observation to a cluster. i.e., if we have N observations, we will have initially N clusters. In this way, distances between observations (similarity) will agree with distances between clusters.Find the closest pair of clusters and merge them into a single cluster. The number of clusters will be reduced in one unity.Compute the new distances between the new cluster built in step b) and the old clusters to get a new similarity framework.Repeat steps b) and c) until grouping all observations into one single cluster or until reaching the number of desired clusters.In an agglomerative methodology, the algorithm is as follows:Step a) and reconfiguration of clusters in steps b) and c) require the use of a measure of dissimilarity between sets of observations; for it, a suitable distance metric and a linkage criterion must be defined. st, between each pair of observations, s and t, and their corresponding cophenetic distances Different distance metrics can be applied. By choosing one or another the shape of the clusters will be affected. Usually, the cophenetic correlation coefficient, CCC, is used to define the best clustering method according to the distance measures. This coefficient is defined as the linear correlation between the dissimilarities, dIn some way, this coefficient indicates the fidelity with which the clustering represents the distance matrix. In this work, good values have been obtained with the Manhattan or city block distance and, therefore, it has adopted in the study:According to Equation (3), considering n dimensions by observation, the Manhattan distance between two points or observations, s and t, is computed adding the distances corresponding to each dimension. In our study, for each loading stage, each observation is associated to one specific sensor.Once the distances between observations belonging to the different clusters have been computed, the linkage criterion allows defining the dissimilarity between clusters from the pairwise distances of observations belonging to them. Different alternatives are possible. In this way, single-linkage and complete-linkage consider the distance between two clusters from the shortest and greatest distance from any point of one cluster to any point of the other cluster, respectively. On the other hand, if an average-linkage clustering is used, the distance between clusters is computed from the average distance between points of both clusters. This last approach has been used in this study.K-means is a widely used clustering algorithm in which the user decides a priori the number of clusters (k) to create. With this approach, those items or observations which fall outside of these groups might potentially be classified as anomalies. Unlike hierarchical clustering algorithm, only one parameter, k, is used in k-means.Initialization of k centroids. This initialization might be performed randomly.Allocation of all the observations to their closest centroid.Updating of each centroid by computing the mean of all the items assigned to it.Reallocation of the observations according to the updating of step c).The characteristic algorithm of this procedure is as follows:Steps c) and d) are repeated iteratively until reaching convergence, which means that no new reallocations are carried out in all the observations.is, associated to one sensor, the Euclidean distance to the centroid of the cluster to which the sensor belongs is calculated as:Additionally, for this clustering algorithm, once the grouping of the different sensors has been initially defined for a baseline condition, representing usually the healthy condition of the structure, a damage indicator can be assigned to each sensor and observation for subsequent stages using the centroids of the clusters as previously stated. For it, for any test observation vector, kµ, and covariance matrix, kΣ, are computed for each cluster k from the values of the characteristic features of the members belonging to the cluster in a baseline stage, which might agree with the healthy structure or not depending on the purpose of the study.In this expression, the mean vector, Once Equation (4) is applied for each cluster, the damage indicator or index D for each observation is defined as the smallest value computed for all the clusters:Equation (5) is applied to each sensor and stage of the structure for which some impedance measurements have been performed. One important aspect to be addressed is about the choice of the clustering algorithm since significant differences occur among them. Either of them should be chosen depending on the requirements of the problem. In this work, we have used both agglomerative hierarchical and k-means algorithms, in a combined way, with the purpose of extracting the maximum amount of information possible of the impedance measurements given by the PZT transducers.Before the application of any clustering algorithm, some meaningful attribute extracted from the impedance data captured by each PZT transducer should be extracted to perform the grouping process. The selected attribute should be especially sensitive to damage but not to operational and environmental conditions.The collected data for each sensor and load case include five frequency sweeps providing five raw impedance signals. An average of the five signals has been performed and the real part of the average impedance signal was used as feature for the hierarchical clustering algorithms since it gives more information and is more sensitive to damage than the imaginary part.For the application of k-means clustering, a meaningful feature extracted from the real part of the measured impedance spectrum (Re(Z(ω)) has been used for the grouping. In particular, moments of the measured impedance responses have been chosen as the feature in the k-means approach:1, ω2), of the signal and, therefore, capture the information of the entire spectrum. To choose the order n of the moment, we have to take into account that the higher the order the higher the weight given to the higher frequencies. As we are interested in the local damage identification, high-frequency components are more suitable and, therefore, a moment with order 3–4 can provide useful information of the spectrum.These moments depend on the entire frequency range,  for steel reinforcement, the elastic moduli and elastic limit were taken to be E = 210 GPa and fy = 510 MPa; (c) for the reinforcement, one CFRP bar with E = 170 GPa and tensile strength of 2500 MPa was inserted into the concrete cover.An experimental study was carried out on two NSM-FRP specimens. One of the specimens (specimen A)  was expeIn the test program performed, the two strengthened beams were subjected to a series of four-point increasing static load tests with the purpose of gradually introducing deterioration into the specimens .In order to verify the applicability of the proposed methodology, impedance measurements were captured from an array of eight PZT transducers once the beam has been unloaded after each static test and before loading it again. Two different transducer types and technologies of integration into the test specimens were applied. Two P-876.A12 DuraAct patch transducers of dimensions 61 mm × 35 mm × 0.5 mm  five loading steps (D1 to D5) were applied on both specimens until reaching the failure. The electromechanical impedance at each sensor was measured initially and after each loading step by exciting a wide frequency band of 10–100 kHz using each one of the PZT transducers and acquiring the response signal with the same transducer. An HP 4192A analyzer from Agilent was used. To measure several transducers, the device was coupled with a 3499B multiplexor from Agilent as well. Five frequency sweeps were conducted for each sensor at each damage state, resulting in five impedance signals. Therefore, 35 measurements were taken by each PZT transducer, although the signals after step 6 were not very useful considering that the beam was already severely damaged. The selection of a suitable frequency range is very important and, in fact, it is a problem to be solved in the application of EMI technique. In this study, the frequency range used has been up to 100 kHz. Considering that lower frequency ranges result in a larger sensing area, we think that working with ranges below 100 kHz, a relatively large area will be covered which can be enough for our damage detection purposes. Initially, the real part of the impedance measurements obtained with the eight sensors for specimen A after the first loading stage are shown in Additionally, Imaginary part of the impedance should mainly capture the changes due to temperature. Although not shown here by simplicity, negligible variations were observed between imaginary impedance signatures corresponding to two consecutive loading stages. During the tests the environmental conditions of the lab were kept constant since studying the effect of environmental variability on the system is outside the scope of this paper, and will be presented in future works. In this sense, variations in the real part of the impedance should be mainly due to mechanical damage of the specimens and will be used in the further clustering analysis. Once an initial clustering has been performed, the main study aimed at identifying sensors and stages with abnormal responses with the purpose of predicting the possible failure of the specimen. In this first test beam, the failure was due to a tensile fracture of the FRP bar on the right side of the middle section. It occurred suddenly, which makes difficult the identification by PZT sensors. However, all sensors have been inspected to check if they provide relevant information of severe damage in the last load steps. Initially, the dendrogram is constructed for all sensors and the baseline stage . It showSensors PZT3 and PZT4 can be gathered in the same group, since both of them differentiate between two clusters, one with the damage states 1–3 and another one with the damage states 4 and 5. However, they do not succeed in differentiating these two damage states, which can be due to the fact that the area around these sensors was subjected to a lower growth of cracks during the last loading steps. In addition, once the experiment concluded, no cracks were observed crossing the sensors. During the last two loading stages, growth of cracks occurred mainly internally as a previous step to the final failure.With respect to the most external sensors on the concrete face, which are of the same type, PZT1 only detected the first minor cracks appearing during the first load stage. Two cracks surrounded this sensor during this step but no new crack appeared during the subsequent steps.Sensor PZT2 shows a clear differentiation between damage states 4 and 5 and the rest of damage states. It should be noted that these two damage states are those in which the beam was the most severely damaged. Therefore, this sensor gives more useful information, since it is clearly able to distinguish the more severely damaged states.This methodology was applied for all sensors and loading stages using as feature vector the average third order moment (n = 3 in Equation (6)) from the five conducted frequency sweeps. Although not shown here, results for the second order SM were similar. The number of clusters was set to three in agreement with As commented previously, according to the analytical estimations, the predicted failure mode for this specimen should be the debonding of the FRP bar due to a loss of adherence. Furthermore, to favor this, some areas of the FRP bar were not adhered with epoxy to the concrete in order to simulate a defect at the concrete-FRP interface, such as shown in blue color in The same test procedure than for beam A was performed with this beam. Impedance values associated to the baseline stage showed an anomalous behavior and, for this reason, were not used for further analysis.From In this specimen, unlike the previous specimen, the high and sudden severity of damage reached in the last loading stage for the internal sensors PZT6, PZT7, and PZT8 might be a symptom, even although failure by rupture of FRP occurred, of imminent loss of adherence at the FRP-concrete interface close to the artificially induced initial debonded areas. In fact, from the visual observation of the specimen, the initial debonded areas extended to regions close to these sensors because of the appearance and widening of internal cracks. Due to this, it was difficult to predict the failure mode which unexpectedly occurred. This methodology was applied in a similar way to specimen A but without considering the baseline results. Three clusters and 20 replications were set. The early prediction of the failure of NSM-FRP strengthening systems is critical and remains an open issue. Typical failures initiate locally and, most of time, without previous warning. In this work, a high frequency range methodology, based on EMI, able to track in a fast, intuitive, and reliable way the evolution of the damages in concrete structures strengthened with NSM-FRP systems has been proposed.EMI measurements captured with PZT sensors yield massive information covering a wide frequency range. A suitable processing of this information is key to implement a reliable detection system. In this work, a method combining unsupervised hierarchical clustering and k-means clustering has been developed. By applying these techniques, groups of complex measurements captured from several PZT sensors along different loading stages can be separated into clusters according to dissimilarity measures using two complementary methodologies. In one approach, clustering is performed for each sensor independently considering the different loading stages. In the second approach, grouping is made independently for each loading stage considering all the involved PZT sensors. Both approaches contain information from the entire frequency range by directly using raw impedance data or by extracting moments from the measured impedances. In this way, by combining conclusions of both alternatives, subtle differences between the normal signals and distorted ones and, thus, possible locally damaged areas could be identified. The performance evaluation of the proposed approach has been made successfully with two experimental tests on concrete beams strengthened with NSM-FRP. For future works, the damage detection procedure should be improved in order to consider varying and unknown conditions  which are independent of the mechanical state of the structure but which might mask the effects due to damage. Variations due to operational and environmental changes should be discriminated from variations due to damage. This is, without doubt, an important challenge for the practical application of this proposed methodology in the SHM context.


Human activity recognition (HAR), which is important in context awareness services, needs to occur continuously in daily life, owing to which an energy-efficient method is needed. However, because human activities have a longer cycle than HAR methods, which have analysis cycles of a few seconds, continuous classification of human activities using these methods is computationally and energy inefficient. Therefore, we propose segment-level change detection to identify activity change with very low computational complexity. Additionally, a fully convolutional network (FCN) with a high recognition rate is used to classify the activity only when activity change occurs. We compared the accuracy and energy consumption of the proposed method with that of a method based on a convolutional neural network (CNN) by using a public dataset on different embedded platforms. The experimental results showed that, although the recognition rate of the proposed FCN model is similar to that of the CNN model, the former requires only 10% of the network parameters of the CNN model. In addition, our experiments to measure the energy consumption on the embedded platforms showed that the proposed method uses as much as 6.5 times less energy than the CNN-based method when only HAR energy consumption is compared. The recognition of human activity plays a vital role in context awareness services such as daily lifelogging, surveillance, healthcare, and human-computer interaction. The worldwide availability of smartphones equipped with inertial measurement unit (IMU) sensors has increased the interest in the use of these devices for Human Activity Recognition (HAR) to monitor and track daily activities.Previous research on HAR using smartphones mainly involved two approaches: methods based on handcrafted features and deep-learning-based methods. The former of these methods generally extracts features from the raw signals of IMU sensors and then classifies the activity by using either machine learning or statistical analysis methods. However, handcrafted features poorly generalize to other tasks because they are designed for specific tasks . MoreoveIn recent years, deep neural networks (DNNs) have yielded satisfactory results in audio and visual recognition tasks because of their ability to systematically and automatically learn discriminative features from training samples ,3. Thus,An HAR system using deep learning comprises two major parts: segmentation and activity classification. The segmentation procedure preprocesses raw sensor signals to reduce the noise components before dividing the filtered sensor signals into smaller groups of signals using fixed-size windows. The subsequent activity classification process classifies human activity based on the information contained in these segments. The classification results for each segment can be fused at a later stage to increase the activity recognition rate.Many methods have been proposed to reduce the energy consumption of the activity classification process as part of the overall system. HAR methods based on handcrafted features mainly reduce the energy consumption by lowering or varying the sampling rate of the inertial sensors ,13. In aIn this paper, we proposed an energy-efficient method for human activity recognition with segment-level change detection and deep learning. CNNs are currently widely used in inertial-sensor-based HAR and are known for their high computational complexity and energy consumption. The energy consumption of the CNN is closely related to the network architecture, especially the number of network parameters. Because the CNN parameters occupy most of the fully connected layers, the proposed method effectively reduces the number of network parameters by using a fully convolutional network (FCN) in which the fully connected layers are replaced by convolutional layers. Furthermore, considering that human activities such as walking, jogging, and running have a longer cycle than HAR methods, which have cycles of a few seconds, it is possible to reduce the amount of computation by detecting the time at which the activity changes and by recognizing the activity only at that time. Therefore, we propose segment-level change detection, which enables the identification of activity change with very low computational complexity compared to the activity classification engine.In summary, this research makes the following contributions. First, we propose a method that can detect activity change by monitoring the compact features of IMU signals. Second, we propose an efficient activity recognition method based on FCN and devise an energy-efficient activity recognition method that detects activity change and recognizes the activity only when necessary. Finally, we conducted experiments using a public dataset to demonstrate that the proposed method can effectively reduce energy consumption without sacrificing accuracy.The remainder of this paper is organized as follows. Deep learning has lately been used successfully in the fields of computer vision and speech recognition. This has encouraged many researchers to use deep learning with inertial sensor data for classifying human activities. Several methods using neural networks such as CNNs ,5,6,7,8,Zeng et al.  introducThe previous work mainly focused on increasing the recognition rate by designing the network architecture or by devising a precise encoding scheme for network input. This is because increasing the recognition rate is the primary goal in the field of HAR. However, recent research  pointed Several energy-efficient approaches for human activity recognition have been proposed. Energy-efficient approaches based on handcrafted features usually adjust the sampling rate ,13 or usFew studies aimed at the implementation of energy-efficient activity recognition based on DNNs have been reported ,15,16. RIn this section, we elaborate on our energy-efficient human activity recognition method. A flowchart of the proposed method is shown in The first step of a typical activity recognition process entails collecting raw signals from the inertial sensors. This study only considers accelerometers but the results can be extended to time-series data from other sensors such as gyroscopes, magnetometers, and electrocardiography sensors. The total acceleration signals collected from the accelerometer are separated into body acceleration and gravity acceleration components. The proposed method uses the total acceleration data for segment-level change detection and the set of total acceleration and body acceleration data for activity classification using FCN.Because the raw signal from accelerometers is collected continuously, segmentation is performed on these raw signals, which are divided into minimum units for activity recognition. The sliding window as a segmentation approach is widely used for activity classification and it is divided into a Fixed-size Non-overlapping Sliding Window (FNSW) and a Fixed-size Overlapping Sliding Window (FOSW) depending on whether the windows overlap. The proposed method uses the FOSW approach because the HAR method based on this approach is known to produce superior results .Segment-level activity change detection is achieved by using Signal Magnitude Area (SMA)  as a feaw is the window size, and i th samples of the sensor signal of the x-, y-, and z-axis in a window, respectively.SMA can take both the effects of the magnitude and duration of the accelerometer signal into consideration, which calculates the summation under the magnitude of acceleration over three axes of each window normalized by the window length. The discrete form of the SMA can be calculated as follows (1)SMA=1T is the threshold for determining the activity change, and t and The following equation is used to determine whether the activity of a current window changes by calculating the difference between the SMA value of the current window and that of the previous window:N-th frame when no change in behavior occurs. Our expectation is that performing activity classification using FCN at specific intervals would be able to decrease the error caused by the similarity of SMA values of different activities.Generally, the activity recognition method classifies the activity by using data that are continuously acquired by the IMU sensor worn by a specific user and the proposed method identifies activity change by comparing the difference between the SMA values of the current activity and those of the previous activity. Thus, the proposed system is independent from the subjects. However, similar activities such as walking and ascending stairs can produce highly similar SMA values. Therefore, we perform an activity classification using the FCN model every The SMA calculated for each segment of accelerometer signals is shown to illustrate activity transitions in Lately, several methods using various neural networks such as CNNs ,5,6,7,8,The energy consumption of CNNs is closely related to their network architecture, especially the depth of the network and the number of network parameters. The “Network-in-Network” (NiN) architecture was recently proposed together with global average pooling (GAP), which replaces the fully connected layers in CNN . GAP canThe proposed method transforms the sensor signals into 2D images to use as input for the FCN for activity recognition. The total acceleration and body acceleration of a 3-axis accelerometer are used to generate a 2D image. The body acceleration signal was obtained by applying a third-order high-pass Butterworth filter with a cutoff frequency of 0.3 Hz. The magnitude of the total acceleration and body acceleration were also used because the magnitude of the accelerometer signal is a significant metric for distinguishing activities . Given eIn this study, four FCN models with various shapes were considered to find the best network structure for identifying activities. Each network model has a different number of convolutional layers and every convolutional layer uses a rectified linear unit (ReLU) as the activation function. GAP is used to replace the fully connected layers. Details of the FCN models are summarized in Existing methods focus on increasing the activity recognition rate; hence, datasets that are pre-processed and segmented by activities are popularly used for experimentation. However, to evaluate the proposed method, we need raw sensor signals collected sequentially with a timestamp that reflect various activities in sequence. Therefore, we evaluated our method using the WISDM v2.0 dataset ,28, whicWe first evaluate the activity recognition rate of the proposed FCN models and compare the result with the recognition rate of a conventional CNN model. Then, we explore the effectiveness of the proposed method which adopts segment-level change detection. Finally, we compare the energy consumption of the proposed method with that of the conventional CNN model on embedded platforms.s (90 samples) with 50% overlap. As a result, the dataset produced a total of 65,067 segmented data values. For each window, the body acceleration and its magnitude were calculated and then stacked row-by-row along with the total acceleration and its magnitude. The stacked signals were transformed into a 30 × 24 image and then used as input into the FCN model.We investigated the activity recognition rate of the proposed FCN models using the WISDM dataset. We applied 10-fold cross-validation to evaluate the generalization performance of the proposed method. The dataset was randomly partitioned into 10 equally sized subgroups. Of these 10 groups, a single group was used as the test set, and the remaining nine groups were used as the training set. The cross-validation procedure was repeated 10 times, with each of the 10 subgroups used exactly once as the test data. The dataset was collected with a sampling rate of 20 Hz and the proposed method segmented the raw signals by using a fixed-size window of 4.5 All FCN models used the adaptive moment estimation (ADAM) optimizer  with a cThe performance of the FCN models is compared in In addition, we compared the performance of the proposed FCN model with that of the CNN model. The CNN model has a network structure similar to that of the FCN-III model but the last convolutional layer and the GAP layer of the FCN model are replaced by two fully connected layers comprising 256 neurons. The ReLU activation function and dropout with a probability of 0.5 were applied to the two fully connected layers. We also compared the accuracy of the proposed activity recognition method with that of recent methods ,30. Tabls (90 samples) and 50% overlap and the SMA was calculated using the data contained in half of the window. If the change in the SMA values was smaller than the threshold, the activity of the current window was considered the same as the previous activity. On the other hand, if the SMA difference between the current window and the previous windows exceeded the threshold, the proposed method called the FCN model to classify the activities. The threshold for determining the activity change was manually defined. In addition, in the absence of any change in behavior, the proposed method used the classifier to perform activity classification every 30 frames.In this section, we explore the effectiveness of adopting segment-level change detection for the proposed method. To test the performance of the overall activity recognition system, we used all the data of the WISDM dataset as test data because continuously collected data is necessary to evaluate the performance of the proposed method to best mimic uncontrolled real-life environments. The proposed method segmented the raw signals by using a fixed-size window of 4.5 To find the optimal threshold for determining the activity change, we conducted the experiment with various threshold values. Among the models generated by the result of 10-fold cross-validation, we used the FCN model closest to the average recognition rate of cross-validation. We measured the accuracy of activity classification with respect to the threshold values. The experimental results, which are plotted in Exploration of the effectiveness of segment-level change detection requires continuously collected data that contain the activity change. However, because the dataset used for measuring the performance of activity classification is partitioned without considering the order of the data, we used all the data of the WISDM dataset containing the recordings of 563 individuals. We compared the recognition rates of activity classification methods according to whether classifiers and segment level change detection are applied. We used the FCN and CNN model as classifiers and selected the models closest to the average recognition rate of cross-validation, respectively. The activity classification results of the proposed method are provided in A user activity must be monitored over a long period of time, which is inefficient in terms of energy consumption. However, the need to frequently recharge monitoring devices can result in user dissatisfaction. Thus, energy consumption needs to be taken into account when methods using deep learning are deployed in embedded devices. This led us to explore the energy consumption of the activity recognition method based on deep learning on embedded platforms. We used two embedded platforms, namely Raspberry Pi 3 with a 1.4 GHz 64-bit quad-core processor and 1 GB RAM, and the Jetson Nano with a 1.4 GHz quad-core 64-bit CPU, a 128-core integrated NVIDIA GPU, and 4 GB RAM. The Jetson Nano has two power modes, i.e., a 5-Watt mode for supporting limited power and a 10-Watt mode for maximizing the performance. The energy consumption of the activity recognition methods running on embedded platforms was measured using the Monsoon power monitor tool. The experimental setup for measuring energy consumption is shown in The energy consumption of the activity recognition methods is calculated by measuring the amount of energy used to process all of the segmented data in the WISDM dataset. Therefore, we consider all subjects in the WISDM dataset. The proposed system emulates the sensor by using the public dataset because the proposed system does not gather data from the IMU sensor directly. To mimic the sensor, the proposed system reads the sensor data from file and then, feeds the sensor data to the activity classification method. Thus, we first measured the energy consumption by the operating system and the energy consumption by the process emulating the IMU sensor. We refer to this energy consumption as the baseline energy consumption (baseline). Subsequently, we compared the energy consumption of the FCN-based method with that of the CNN-based method by excluding the baseline energy consumption. Finally, we measured the energy consumption by the CNN-based and FCN-based methods with segment-level change detection. Throughout this section, we refer to the CNN-based method with segment-level change detection as CNN-S and to the FCN-based method with segment-level change detection as FCN-S. We also measured the processing time of the activity recognition methods on the embedded systems.The results of the energy consumption measurements of the activity recognition methods on the embedded platforms are shown in The processing time of the activity recognition methods on the embedded platforms is shown in In this paper, we proposed an energy-efficient method for the overall system of human activity recognition. Unlike many existing methods that focus on reducing the energy consumption of the activity classification engine, our approach involved optimizing the segmentation process. Because human activities have a longer cycle than HAR methods, which perform their analysis in cycles of a few seconds, classifying human activities continuously by using a short cycle is highly inefficient. Therefore, we proposed the segment-level change detection method to identify activity change with extremely low computational complexity. Subsequently, an FCN model, which has a high recognition rate, was applied to classify the activity only when activity change occurs. We compared the recognition rate of the proposed method with that of the CNN-based method using a public dataset and explored the energy consumption of the activity recognition methods that use deep learning on embedded platforms. The experimental results showed that the proposed FCN model achieves a recognition rate similar to that of the CNN model despite using only 10% of the network parameters of the CNN models. We also verified that the adoption of segment-level change detection in the proposed method retains the accuracy of activity recognition even though the frequency of use of the FCN model is reduced by 90%. The experiments that were conducted to measure the energy consumption on the embedded platforms showed that the proposed method uses as much as 6.5 times less energy than the CNN-based method when comparing only the energy consumed for activity recognition, excluding the power essential for system operation. The proposed method consumes 1.39 This paper presented the experimental results for embedded devices by using a dataset constructed in advance, but it would be necessary to verify the performance by using a smartphone to capture signals in actual everyday life. Thus, in the future, we plan to deploy the proposed method on a smartphone to evaluate the performance of our method in real environments.


The objective definition of footprint margins poses a central problem in ichnology. The transition from the footprint to the surrounding sediment is often continuous, and the footprint wall complex, requiring interpolation, approximation, and a priori assumptions about trackmaker anatomy to arrive at feasible interpretations of footprint shapes. The degree of subjectivity of such interpretations is substantial, and outlines produced by separate researchers can differ greatly. As a consequence, statistical shape analysis, regardless if based on linear and angular measurements or on the shape as a whole, are neither fully repeatable nor objective. Here I present an algorithm implemented in the programming environment R that is able to generate continuous footprint outlines based on three-dimensional models—fully automatically, objectively, and repeatable. The approach, which is based on contour lines extracted from the model, traces the outline at the point where the slope of the track wall is steepest. An option for automatic landmark placement is implemented for tridactyl footprints. A case study was carried out on 13 footprints of a single trackway of a theropod trackmaker from the Lower Cretaceous of Münchehagen, Germany. Analysis of the landmark coordinates returned by the script did reproduce statistical results published in an earlier study that was based on human-made interpretative drawings, demonstrating the applicability of the present method for the objective and quantitative shape analysis of tracks. Although faint anatomical details are not always recorded and features not related to the foot anatomy may be included, the generated outlines tend to correspond with human-made interpretative drawings regarding the overall shape. While not suited as a full replacement of interpretative drawings, these generated outlines may be used as an objective basis for such interpretations. Fossil footprints are an important supplement to the body fossil record, given their abundance and nature as life traces that directly record behavior and locomotion. Yet, the potential of analyses combining footprint and body fossil data is not yet exhausted, partly due to the slow advancement of objective and quantitative methodology in ichnology. A central problem in applying such methods to footprint data is the inability to objectively define the margins of a footprint, especially when the footprint indistinctly grades into the surrounding sediment. The problem persists when not only linear and angular measurements but a single, two-dimensional outline abstracting the whole shape of the footprint is to be extracted. The outlines of one and the same footprint, when drawn by separate researchers, can differ considerably from each other , which rRecent efforts to increase objectivity in footprint research rely on 3D-digitization techniques, most importantly photogrammetry, which allows for the fast and cost-effective capturing of footprint morphologies in high resolution e.g., . A relatAlthough a number of methods for the analysis of footprint shapes exist, none can effectively solve or circumvent the problem of the definition of footprint margins. Comparative approaches using 3D geometric morphometrics e.g.,  will incThe algorithm presented herein allows for the fully objective and automatic generation of continuous outlines based on 3D surface models of footprints. The method relies on the steepness of the footprint slope , the probably most commonly used criterium for the definition of footprint margins .The algorithm presented herein  neighbor are compared; if the difference in area between two contours exceeds a pre-defined threshold-value, the upper (or lower) of this contour and all following contours are removed . DiffereEven if the starting point would correspond between all contours and if each contour would contain an equal number of equidistant points (requirements not fulfilled a priori), the individual points of the separate contours would tend to deviate from each other when far from the starting point, as the shapes of the contours are not identical. For this reason, when producing a simple mean shape, points would be averaged obliquely rather than perpendicularly to the footprint wall, leading to erroneous results. The implemented solution detects a number of “bottlenecks”—pairs of points with minimum distance between the innermost and outermost contour. Points forming the bottleneck will be considered homologous , and the points in-between the bottlenecks will be interpolated by resampling.First, all contours are resampled to the same number of equidistant points, using 500 points per default. The resulting contours can be variously oriented clockwise or counter-clockwise; contours are reversed accordingly to achieve uniform orientations. Second, Euclidean distances between all possible pairs of the inner and outer contour of the stack are calculated and stored in a matrix with the dimensions n x n. The pair with the minimum distance, the first bottleneck, is then extracted, and those points of the intermediate contours are detected that are closest to a line defined by the bottleneck points. The resulting set of homologous points is then defined as the starting point of the contours. Third, additional bottlenecks are detected to establish homology. The implemented algorithm first detects a second bottleneck on the side of the footprint opposite to the first bottleneck; two additional bottlenecks are then found on each side between the first and second bottleneck. More bottlenecks are detected within the intermediate sections if the latter are long enough. In all cases, bottlenecks in proximity to existing bottlenecks are prevented, assuring a roughly equal distribution of bottlenecks along the outline . FinallyThe objective outline will be traced along the steepest slope of the track wall. For each point within each set of homologous points, the minimum distance between the neighboring contours is measured. A set of homologous points is not always fully perpendicular to the footprint wall, especially when the section between the bottleneck points is long and contours differ much in orientation. For this reason, the algorithm does not simply compute the distances within the set of homologous points, but the distances between each of the homologous points and all nearby points within and outside of the set. The steepest slope computed this way  is seldoThe continuous and smooth final outline is produced by applying an approximating function to the resulting set of points. Of various tested options, including B-splines and Bézier curves, elliptic Fourier transforms were found to produce results most similar to those expected from a human interpreter , 2. ElliAs the resulting objective outline and contour stack will be rotated arbitrarily, an algorithm attempts to rotate both outline and stack into an upright orientation. This algorithm is based on the assumption that digit impressions are facing upwards and require a longer contour segment  to be described. In a first step, the centroid, or center of mass, of the outline is computed. Then, the length of the part of the outline above the centroid is calculated for all rotation angles (1–360°), and the outline rotated according to the angle that maximizes this length.The script, as described above, can be applied to any kind of depression, as no a priori assumptions on trackmaker anatomy are introduced. Such assumptions are required when linear and angular measurements are to be extracted or when different outlines are to be aligned for quantitative shape analysis. Therefore, a function is included that returns landmark coordinates for tridactyl footprints, which are the most common dinosaur footprints and one of the most common footprint types in general. The algorithm first detects and separates the three digit impressions, assuming that the outline was correctly rotated in a more-or-less upright orientation in the previous step. Then, the rotation angle of the outline is further refined by considering the central digit impression only, which is commonly considered to approximate the mid-axis of the footprint . A totalThe applicability of the presented approach for the quantitative analysis of footprints was tested on 13 footprints pertaining of a single theropod trackway (T3) from the Lower Cretaceous of Münchehagen, Germany (3D-models are provided as data S2). Geometric morphometric analysis of the same footprints was conducted in earlier studies based on interpretive drawings , suggestHuman-made interpretational footprint drawings aim to capture as much information about the trackmaker anatomy as preserved. Although slope steepness is the most important criterion, the steepest slope will frequently fade out along the footprint wall to continue at a different height level, making interpolation unavoidable. Furthermore, humans tend to make a priori assumptions about trackmaker anatomy when producing the outlines, which allows them to take into account extramorphological (unrelated to the foot anatomy) features and include anatomical features of interest such as digital pad impressions and claw marks.The present algorithm is successful in detecting and interpolating outlines even when the steepest parts of the slope are indistinct . It doesCaririchnium kyoungsookimi from the Jindong Formation in Goseong County, South Korea  was saved at successively lower resolutions using MeshLab, and the resulting outlines were compared –3F. ObseThe lack of widely applicable, objective means for defining the footprint margin is among the most vexing problems in the research of fossil footprints. The present algorithm automatically generates continuous objective footprint outlines by employing the criterium of the steepest slope. In contrast to human-made interpretive outlines, these generated outlines allow for analyses that are fully reproducible and free of interpretational bias, as complete samples can be processed using the same mathematically defined criteria. Although the generated outlines tend to correspond well with interpretive outlines, extramorphological features unrelated to the foot anatomy may be incorporated, and anatomical detail not captured by the steepest slope may be excluded. While not a fully appropriate replacement for human-made drawings in most cases, computed outlines may be used as an objective basis for the production of interpretational drawings, and allow for objective qualitative comparisons. Most importantly, the approach paves the way for fully objective quantitative analyses of footprint shapes.10.7717/peerj.7203/supp-1File S1Click here for additional data file.10.7717/peerj.7203/supp-2File S2Click here for additional data file.10.7717/peerj.7203/supp-3File S3Click here for additional data file.


This study exposes the assessment of a piezo-actuated sensor for monitoring elastic variations (change in Young’s modulus) of a host structure in which it is attached. The host structure is monitored through a coupling interface connected to the piezo-actuated device. Two coupling interfaces were considered  for the experimental tests. Three different materials  were prepared to emulate the elastic changes in the support, keeping the geometry as a fixed parameter. The piezo device was characterized from velocity frequency response functions in pursuance to understand how vibration modes stimulate the electrical resistance through electrical resonance peaks of the sensor. An impedance-based analysis (1–20 kHz) was performed to correlate elastic variations with indexes based on root mean square deviation (RMSD) for two observation windows (9.3 to 9.7 kHz and 11.1 to 11.5 kHz). Results show that imposed elastic variations were detected and quantified with the electrical resistance measurements. Moreover, it was demonstrated that the sensitivity of the device was influenced by the type of coupling interface since the cone was more sensitive than the tooth in both observation windows. As a final consideration, results suggest that bio-structures  could be studied since these can modify naturally its elastic properties. In the structural health monitoring (SHM) field, predictive and preventive strategies are playing an important role in the prognosis of catastrophic scenarios in order to guarantee safety and reliability during the service of the structures. Currently, different monitoring applications have been implemented with piezoelectric technologies due to its implementation easiness and relatively low cost in comparison with other nondestructive inspection techniques. The structural integrity analysis based on changes of dynamic characteristics requires a permanent monitoring to detect, locate and quantify damages by means of in-situ diagnosis techniques, such as have been implemented in aircraft wings, space shuttles, concrete, turbine rotor blades, and pressure vessels, among others ,2,3,4,5.SHM technologies are being extrapolated to biological fields with the purpose of detecting specific changes in bio-system properties. Bio-structures modify its intrinsic conditions by natural effects or by modifications in the biological equilibrium. Some examples of bio-systems that can modify its properties are fruits , bone tiElectromechanical impedance technique (EMI) permits to detect structural changes in high frequency  by means of a self-diagnosis of piezo-transducers ,9,10,11.A study reported by  applied Other approaches of bio-monitoring have been explored in the dentistry area. For instance, dental implant stability requires a permanent evaluation of the prosthesis since the load-bearing depends on the integrations of bone–implant system. For this purpose,  assessedThis paper presents an electro-dynamic assessment of a piezo-actuated sensor for monitoring the elastic variations (change in Young’s modulus) of a substrate (host structure) with fixed geometric conditions and variable Young’s modulus. The host structure acts as a main support of the coupling interface  that joints the piezo device with it. The substrate was made of three different materials  to simulate the elastic changes in the monitoring process with the EMI technique.Piezoelectric materials present electro-mechanical properties that have been exploited in structures with different purposes, for instance to monitor the structural integrity in the frequency spectrum ,25,26. IAccording to , a PZT iin a PZT .i” and “s” refer to coupling interface and monitoring structure. As electromechanical impedance theory suggests, each modification in the stiffness  done by ,26. In aterature ,35,36,37This section presents and develops the description and design of an electromechanical piezo-actuated sensor, as illustrated in The piezo-actuated sensor is activated by applying a harmonic voltage (linear sweep) in such way that oscillatory deformations are produced, a scheme is described in For the assessment and comprehension of the piezo-device dynamic behavior, velocity measurements are taken when it is clamped in a mechanical press. It means that the coupling interface and the monitoring structure are not considered in the initial analysis. The main objective is to identify some mechanical resonances of the piezo-device itself to compare it with the resonance peaks of the electrical impedance.The experimental setup is defined by a computer, a data acquisition card (NI DAQ 6211) which has 16 analog inputs , a power amplifier TREK high voltage amplifier 2205  and a POLYTEC CLV-2534  laser vibrometer. A scheme of the experimental setup is shown in An experimental setup was settled in order to correlate the mechanical resonances of the piezo-actuated sensor from electrical impedance applying the EMI technique. The identification will permit to choose which resonances belong to mechanical responses that can be monitored from the electrical measurements by EMI, as explained in The activation principle of the piezo device consists of exciting both piezoelectric patches simultaneously applying a harmonic voltage and reading the current response signals that pass through it using the impedance analyzer . The sweFor the searched application, the piezo-device was coupled to three different metal specimens  to emulate a variation of the elastic modulus in the monitoring structure as shown in Six tests were performed in the chosen materials  using both coupling interfaces. Elastic properties of each material were provided according to the material supplier as 70 GPa (Aluminum), 120 GPa (Bronze), and 200 GPa (Steel). For the first one, three tests were done with the aluminum conic interface (Cone) coupled to the three specimens; and in the second one, another three tests were carried out with a premolar tooth coupled to each specimen. Coupling interfaces were bonded with epoxy adhesive Loctite 4981. It is also important to highlight that all three specimens have the same dimensions and geometrical configuration in order to focus our study on stiffness changes due exclusively to elastic changes done in the monitoring structure.Velocity signals measured in one piezoelectric patch are illustrated in To determine the frequency values that corresponds to these maximum velocities, Fourier’s transform was computed and plotted in This section deals with comparisons of measurements obtained for the velocity and electrical resistance in the frequency domain . Electrical resistance measurements were also carried out with the piezo-device mounted in a mechanical press (isolated) as the velocity experiments. Electrical resistance was measured using the impedance analyzer  shown in Two peaks are visible in the range of 1 to 2 kHz, with the first peak at 1046 Hz (A) and a second peak at 1220 Hz (B). It is observed that the electrical resistance presents two smoothed peaks in the same frequency values, approximately. This correlation indicates that reflected peaks in the electrical resistance correspond with mechanical resonances induced by the dynamics of the device. Nevertheless, that influence is low since the relative amplitude (height from the baseline) of the peak is low, indicating that the piezoelectric patches are being deformed slightly by the vibrations. This can be verified reconstructing the vibration shape with the amplitudes of these peaks. In other words, for monitoring purposes the vibration modes should be favorable to deform the PZTs since it will reflect a higher amplitude in the electrical resonances. Methods based on the idea of electromechanical impedance and vibration modes require a dynamic analysis to obtain a major sensitivity in the electrical signatures when the monitoring structure is modified. For this reason, this interval (1–2 kHz) is not considered as a monitoring spectrum.In the range of 6–15 kHz, three peaks were identified in the electrical resistance, with the first peak at 9.1 kHz (C), second peak at 11.5 kHz (D) and a third peak at 13.7 kHz (E) as shown in The correlation of mechanical parameters through the electrical impedance shows a big advantage in SHM applications. It is not necessary to measure kinematic parameters in localized degrees of freedom to verify the resonances of the whole structure. It implies that a PZT is able to capture the mechanical resonances if these influence the deformation of it, as observed in The main idea of the proposed piezo device is to track changes in the electrical resistance when the elastic properties undergo variations in monitoring structure. As As seen in Other changes occurred in the relative amplitudes (height from its baseline) of the peaks that were incremented, especially the first one. This is caused by the clamped conditions (piezo device) since the vibration mode is less constrained (lower stiffness in the boundary). Also, the peaks were slightly shifted with respect to the peaks of the green curve (clamped device). The presence of new elements in the piezo device caused changes in its mechanical resonances as naturally expected, because these are directly affected by the mass/stiffness relation of the whole system. However, as results showed, it can be inferred that the mechanical resonances did not vary dramatically keeping the mechanical impedance of the sensor as the principal structure of the signal.Analyzing the curves, there were found two observability windows where the signals were sensitive to the change of elastic variations. The searching parameter was based on the progressive changes of the signal when Young’s modulus was increased. The first window was found in the range of 9.3–9.7 kHz, which is located at the right side of the first peak and labeled as A-A, and the second monitoring window is established in the interval of 11.1–11.5 kHz at the left side of the second peak, labeled as B-B, as illustrated in In In order to quantify and monitor elastic variations using the observation windows, RMSD index was calculated (see Equation (4)) and the results are summarized in Besides, the electrical resistances were correlated with the purpose of verifying the relation between both coupling interfaces as illustrated in The mechanical design and assessment of a piezo-actuated sensor for structural monitoring has been described, developed and experimentally tested. The electromechanical impedance technique was applied and its effects were investigated in order to detect elastic variations imposed in the support of the piezo-device. Experiments showed that when the elastic modulus was modified, the electrical resistance consistently varied in intervals 9.3–9.7 kHz and 11.1–11.5 kHz. This permitted the elastic variations to be quantified with an index based on RMSD. Two types of coupling interfaces (cone and human tooth) were tested to evidence the repeatability of the analysis presented in this study. Considering both coupling interfaces (joining part between support and piezo-device), the correlation among its electrical resistances were shown to be effective and sensitive to the elastic variations. Therefore, these changes were identified with the correlations. To find a potential applicability, it is necessary to perform tests in bio-structures in which can be monitored the reduction or the increment of its global elasticity. We see an opportunity to apply this methodology on teeth since these are connected directly with bone. Bone is a sensitive structure in which metabolic changes are reflected in its structures. However, it is important to consider that this paper provided a proof-of-concept that should be validated before a real bio-application.


Here, we present a detailed description of the application of HMM to inelastic mechanics of materials, with emphasis on the efficiency and accuracy of the scale-bridging methodology. The method is well suited to the estimation of macroscopic properties of polymers (and derived nanocomposites) starting from knowledge of their atomistic chemical structure. Through application of the resulting workflow to polymer fracture mechanics, we demonstrate deviation in the predicted fracture toughness relative to a single-scale molecular dynamics approach, thus illustrating the need for such concurrent multiscale methods in the predictive estimation of macroscopic properties.Mechanisms emerging across multiple scales are ubiquitous in physics and methods designed to investigate them are becoming essential. The heterogeneous multiscale method (HMM) is one of these, concurrently simulating the different scales while keeping them separate. Owing to the significant computational expense, developments of HMM remain mostly theoretical and applications to physical problems are scarce. However, HMM is highly scalable and is well suited for high performance computing. With the wide availability of multi-petaflop infrastructures, HMM applications are becoming practical. Rare applications to mechanics of materials at low loading amplitudes exist, but are generally confined to the elastic regime. Beyond that, where history-dependent, irreversible or nonlinear mechanisms occur, not only computational cost but also data management issues arise. The micro-scale description loses generality, developing a specific microstructure based on the deformation history, which implies This article is part of the theme issue ‘Multiscale modelling, simulation and computing: from the desktop to the exascale’. The current approach, largely unchanged in decades, consists of an extensive trial-and-error process followed by testing and certification. This process typically takes around 20 years from concept to commercial implementation ,2. CompuThe macroscopic mechanics of solids are highly dependent on structural detail found at each scale, from the electronic organization at the quantum mechanical level through to the geometry of the sample at the macroscopic scale. Attempts to estimate macroscopic mechanical properties using a single model at nanoscopic resolution are quickly rendered intractable by the astronomical computational cost or limited by unrealistic test set-ups involving, for example, microscopic material samples or simplistic uniaxial tests. Scale-bridging approaches become essential ,8.To render such simulations tractable, so-called hierarchical multiscale models make use of a loosely coupled scheme in which continuum-level constitutive equations are parametrized beforehand using lower-level molecular or coarse-grained simulations –12. HoweAs a  remedy to these problems, computational schemes have been developed that seamlessly couple multiple scales in a semi-concurrent fashion, implemented through the heterogeneous multiscale method (HMM) –16. A siOwing to the increases in the computational power and parallelism available on the biggest supercomputers, as well as sophisticated computational workflows ,23, suchThe developments presented here are applied to the two-scale modelling of thermoset polymers, relying on a MD  descript2.ntmd with variable execution time.The two single-scale models are orchestrated in a cyclic dynamic fashion . Each itrai the displacements of the atom i, U the potential energy of the thermodynamic ensemble, mai the mass of the atom i of the atomic model.The atomistic models are simulated using all-atom MD , a methoγ a damping parameter, T the temperature of the thermodynamic ensemble, kB the Boltzmann constant and R(t) a stationary Gaussian process with a zero time average. The degrees of freedom of the atomistic system rai are the displacements of the particles representing the atoms, subject to constraints imposed by the molecular structure.In the present case, for temperature control, stochastic equations of Langevin dynamics  are solvN), and fixes the volume (V ) and temperature (T) of the system. The atomistic model is initialized in its most recent configuration of atoms. As we deal with inelastic mechanics, the system cannot be started from an equilibrium configuration, as the history must be preserved. Updating the atomistic mechanical state is performed by deforming the system box according to the strain tensor ϵ provided by the continuum model. The volume is varied by applying a constant strain rate τa of the material (this will be the object of further discussion in the following sections). As a result, the number of time-steps to perform in each atomistic model simulation will, in general, be directly proportional to the amplitude of the applied deformation (the length of time-step being constant). Coarse-grained MD is not considered in the present study, as the process of coarse-graining the atomistic system requires recent parametrization techniques to capture mechanisms of interest such as fracture  or  or σ andb illustrates how complex the actual constitutive behaviour can be, and how simple isotropic models would be unsatisfactory. Indeed, constitutive behaviour informed by processes at the atomistic scale would comprise anisotropic yield, hardening and softening. Furthermore, chemical specificity is poorly captured in the constitutive equation due to the difficulty of estimating parameters that often lack direct physical meaning .Figure 95.q. Conversely, at the atomistic level, the requirements imposed by inelastic mechanics can be more cumbersome. History dependency implies that the atomistic system must be brought to the desired mechanical state, with the exact Keeping track of inelastic mechanisms requires storage of the (history-dependent) current state of the material at each spatial location. As constitutive behaviour is handled via atomistic simulations, the continuum level need only know the current strain and stress tensors at each ntmd can be assumed to be constant, but with individual execution times determined by the norm of the applied strain on the atomistic system (which may be zero). As such, efficient scheduling of the many atomistic simulations is non-trivial, and can involve allocating different numbers of cores to different simulations in an attempt to achieve (approximately) equal runtime. This problem could be tackled with the help of resource optimization libraries, which efficiently schedule each atomistic model simulation by taking into consideration queueing time, execution time and energy consumption across distributed resources [The resulting HMM workflow is distributed across many cores and can display significant load-imbalance. The workflow is divided into a single continuum model and multiple atomistic models. At any given time-step, evolving the continuum model is generally far less expensive than evolving one of the many atomistic models . The continuum and atomistic models are evolved alternatively, but the atomistic models themselves are run in parallel, drastically improving the scalability of the method. Nonetheless, there is a highly variable difference in resource requirements between continuum and atomistic simulation phases. This might justify performing the simulation of the continuum model on a separate smaller resource. In turn, we would avoid idleness of a large part of the total allocation when the continuum model is evolved. esources .6.We have described a HMM implementation suited for the study of nonlinear irreversible macroscopic mechanisms and its rigorous testing and application to polymer thermosets. We have discussed in detail the resulting computational workflow and associated numerical tools to solve multiscale problems encompassing atomistic and continuum descriptions of the material. We described an implementation to determine the sampling requirements for homogenization of the material behaviour at the atomistic level and the transfer of the associated information to the continuum level.Via an application to the simulation of the measurement of the fracture properties of an epoxy resin, we illustrate the benefits of the HMM relative to a single-scale atomistic approach. Unlike single-scale or even hierarchical multiscale models, our multiscale approach efficiently handles the complexity induced by the local mechanical behaviour of the material, while integrating the influence of the boundary conditions imposed by the testing set-up at the engineering scale. We have shown a significant quantitative difference in stress–strain response and fracture toughness predicted by a purely single-scale (MD) approach and that predicted by our HMM implementation. Simulated degradation mechanisms, such as void growth, also differ substantially. This demonstrates the necessity of using a concurrent multiscale approach for accurate and chemically specific prediction of mechanical properties at the macroscale.


Acoustic emission (AE) source localization is one of the important purposes of nondestructive testing. The localization accuracy reflects the degree of coincidence between the identified location and the actual damage location. However, the anisotropy of carbon fiber three-dimensional braided composites will have a great impact on the accuracy of AE source location. In order to solve this problem, the time-frequency domain characteristics of AE signals in a carbon fiber braided composite tensile test were analyzed by Hilbert–Huang transform (HHT), and the corresponding relationship between damage modes and AE signals was established. Then, according to the time-frequency characteristics of HHT of tensile acoustic emission signals, the two-step method was used to locate the damage source. In the first step, the sound velocity was compensated by combining the time-frequency analysis results with the anisotropy of the experimental specimens, and the four-point circular arc method was used to locate the initial position. In the second step, there is an improvement of the Drosophila optimization algorithm, using the ergodicity of the chaotic algorithm and congestion adjustment mechanism in the fish swarm algorithm. The smoothing parameters and function construction in the probabilistic neural network were optimized, the number of iterations was reduced, the location accuracy was improved, and the damage mode of composite materials was obtained. Then, the damage location was obtained to achieve the purpose of locating the damage source. Carbon fiber composites have the characteristics of high strength and modulus, small thermal expansion coefficient and fatigue resistance ,2,3. SucAcoustic emission (AE) testing is a non-destructive testing technology, which is different from the conventional non-destructive testing technology. It belongs to the AE method without external excitation. Compared with the traditional nondestructive testing technology, AE technology has the following two characteristics. First, the dynamic information of material and structure damage can be detected in real time. It can monitor the generation and propagation of cracks. Secondly, the AE signal detected is emitted by the material itself, rather than the excitation signal introduced by the outside world. Therefore, AE technology has the following advantages. First, it has initiative. For large components, active signal output can reduce the number of detection equipment and save detection time. Through a large number of wiring and a single loading in the early stage, AE signals will be generated at the damaged parts of the components. From this, the location and types of damage can be determined, and the detection efficiency can be improved. Secondly, the AE signal is used to identify the type of damage and the location of damage. The location of the AE signal is based on the location of the signal receiver, but the geometric shape of the workpiece itself is not required. Thirdly, the AE signal is received by the receiver, which can be regarded as the real-time production of the workpiece damage. It can be applied to real-time and rapid monitoring of damage and its development. Fourthly, the signals of different failure modes in the AE signal source can be separated by spectral analysis technology. According to the dominant AE signal, the type and degree of internal damage and failure of the specimens are judged. AE nondestructive testing technology has become a research hotspot for defect and damage detection in composite materials ,12.Three-dimensional braided composite materials, first of all, it uses three-dimensional braiding technology to braid reinforced fibers into three-dimensional monolithic fabric (preform). Then, the preform is compounded with the matrix to make composite. The reinforcing fibers in three-dimensional braided composites have a spatial interweaving structure. Therefore, in addition to the advantages of high specific strength and high specific modulus, it also has better impact resistance, higher damage tolerance and energy absorption . During In the research of AE detection technology, Barile proposed a method to detect and identify different types of damage in carbon fiber composites (CFRP) using AE technology . Nair A Since there are dozens of characteristic parameters of AE events, there is a certain correlation between them in clustering the analysis of signals, so it is necessary to optimize the characteristics of data. In recent years, with the rapid development of artificial intelligence, neural network has been used to model data. The trained model is used to classify and judge unknown data, which improves the modeling speed and recognition efficiency. Kalafat et al. present an alternative localization method based on the use of neural networks, using experimental training data as a modeling basis. It was shown that the neural-network-based method is not only superior by a factor of six in accuracy, but also results in a lower scattering of the localized source positions by a factor of 11 . Han et The premise of traditional AE source location is to determine the propagation speed of acoustic wave in solid medium, and to determine the location of acoustic emission source by the time difference between the acoustic wave arriving at the two sensors. However, carbon fiber braided composites have anisotropic properties . Sound w3 and a linear density of 0.8 g/m. Its technological parameters are as follows: surface braiding angle is 22.60°; internal braiding angle is 32.14°; pitch length is 6.0 mm; fiber volume content is 45.00%; size is 250 mm × 25 mm × 4 mm. The internal structure sketch and physical diagram of three-dimensional four-directional carbon fiber braided composites are shown in In this paper, carbon fibre braided composites by four-step 1 × 1 three-dimensional four-directional braiding process are taken as the research object for tensile test. The braided yarn of the test sample is T700−12K carbon fiber with a density of 1.76 g/cmBefore the tensile test, the sound velocity was determined by the standard lead-breaking test at the position near the sensor. When the measured sound wave amplitude is greater than 85 db, it is considered to meet the requirements. In the tensile test, the AE signals during the tensile process are obtained by AE testing technology , and the corresponding mechanical properties parameters are obtained by Shimadzu Universal Material Testing Machine for analysis. To avoid testing error, specimens with each parameter were tested five times. The tensile process of a 3D braided composite is shown in In the process of tensile test, mechanical properties such as tensile stress-strain curve and acoustic signal parameters such as energy, ringing count, and RMS voltage can be collected. The load-displacement curves of the specimens correspond to the energy parameters of acoustic emission signals as shown in From the load/KN-displacement/mm (red) curve in It can be seen from the energy/eu-displacement/mm (blue) curve in In Samples produced different AE signals at different damage stages. These AE signals at different damage stages tended to overlap and presented random and non-stationary distribution . This brIn According to the HHT time-frequency characteristics of AE signals of carbon fiber braided composites in tensile damage mode, on the basis of determining the damage stage and signal type, a two-step method was used to locate the damage source. In the first step, the sound velocity was compensated by the time-frequency analysis results and the anisotropy of the sample. The initial location was performed by the four-point circular arc method. In the second step, after the initial damage location, the Drosophila algorithm in probabilistic neural network was used to calculate the damage location accurately. The Drosophila optimization algorithm was improved by ergodicity of chaotic algorithm and congestion adjustment mechanism of fish swarm algorithm. In this way, the smoothing parameters of probabilistic neural network were optimized, the number of iterations was reduced, and the accuracy of locating the fixed damage source was further improved.Through the analysis of the tensile process and HHT marginal spectrum of carbon fiber braided composites, it can be seen that the acoustic signals of the micro-crack generation stage and the micro-crack propagation stage had obvious continuity characteristics during the tensile process. The acoustic signals in the instability damage stage had obvious sudden characteristics. A number of sensors were fixed on the surface of the object to form a certain geometric relationship, so that the damage source can be initially located by using the time difference of different acoustic waves received by the sensor array. Therefore, the four-point circular arc location algorithm based on the two-dimensional plane location method was used to locate the damage source preliminarily. Four acoustic emission sensors were placed on the sample to form a rectangular plane detection area. Because of the anisotropy of carbon fibre braided composites, the acoustic velocity varies greatly, so it is necessary to correct the acoustic velocity to eliminate or reduce the location error caused by the difference of acoustic velocity . Taking Lead breaking experiments were carried out on the samples. The sample was divided into four quadrants by equal ratio. Two points in each quadrant were taken and 10 lead-breaking experiments were carried out at the same point to correct the sound velocity. The actual location of the acoustic emission source was compared with the calculated location of four-point circular arc location by taking the experimental results of eight different locations. The comparison results are shown in As can be seen from Due to the anisotropy of carbon fiber braided composites, the propagation law of acoustic wave in materials was complex, which made it difficult to locate and detect AE damage. In this paper, Drosophila optimization algorithm was selected to optimize the key parameters of the network, taking full account of the propagation medium material, the geometry of the propagation medium, the interweaving of fibers in three-dimensional space and the convergence accuracy and speed of different intelligent optimization algorithms in the calculation of probabilistic neural networks.The optimization mechanism of the Drosophila optimization algorithm is to simulate the process of Drosophila foraging, which is divided into two stages: olfactory search and visual location. The mechanism of Drosophila optimization algorithm is simple; it has small computational complexity and is easy to implement, and the population can quickly approach the optimal individual, thus ensuring the fast convergence of the algorithm. However, because Drosophila individuals focus on the optimal individuals for random search, it is easy to lead to the premature phenomenon of the algorithm, which reduces the global search ability of the algorithm. Therefore, based on the Drosophila optimization algorithm, combined with ergodicity of chaos algorithm and congestion adjustment mechanism of fish swarm algorithm, an improved Drosophila hybrid optimization algorithm was proposed to locate the damage source of carbon fiber braided composites.When the chaos optimization algorithm is used to improve the shortcomings of random search, chaos generation mechanism can directly replace random number generator mechanism to improve the efficiency of random search. Therefore, the chaotic search method is used to improve the random search method of Drosophila individuals in order to improve the Drosophila optimization algorithm. Logistic mapping is one of the most commonly used chaos generation mechanisms, which has the characteristics of simple form. This paper chose logistic map as the mechanism of chaotic sequence occurrence, as shown in formula (1):μ = 4, the map is full of chaos between 0 and 1. Its chaotic bifurcation diagram and corresponding Lyapunov exponent are shown in μ, the iteration value of Logistic map goes through a period-doubling bifurcation process of one, two, and four periods, and finally enters into chaotic state. When the logistic map is in chaotic state, its corresponding Lyapunov exponent is larger than 0. The advantages of randomness and ergodicity of chaotic sequence make its search efficiency much higher than that of random search method. Logistic mapping was used to generate the chaotic sequence, which replaced the random sequence generated by a random number generator, so as to realize chaotic search.When the parameter x_best, y_best) with probability, and search randomly with small probability.In Drosophila algorithm, all Drosophila have the same behavior criteria and aggregate near the food source. As a result, Drosophila cannot search globally, which reduces the global search performance of the algorithm. In the fish swarm algorithm, similar behavior exists in the foraging process, so the concept of crowding degree is set up. That is to say, the fish cannot gather too much in the same place to prevent the algorithm from convergence too fast. In this paper, the concept of crowding degree in the fish swarm algorithm was introduced into the Drosophila algorithm, so that Drosophila did not have the only criterion of action. The optimal initial position is paid according to random probability. Drosophila individuals search around , and the initial congestion probability (P).Step 1—initialization parameters: The main parameters involved in Drosophila optimization algorithm were initialized. Specifically, it includes the maximum number of iterations (MaxGen), the size of Drosophila population (Size), the initial optimal location of drosophila (Step 2—setting up chaos generation mechanism: It was shown in formula (2). After selecting the initial value, the chaotic initial value of Drosophila optimization algorithm was obtained after 2000 iterations, which can eliminate the influence of the initial value of chaotic mechanism on the search process.x_axis and y_axis were set. There were two ways to set the search direction for the ith Drosophila individual in the population. Random number generator was used to generate random number r between 0 and 1. If r < p, set the random search direction of the ith Drosophila near the locations of x_axis and y_axis. It was shown in formula (3): Step 3—setting random search direction for Drosophila: Random initial r ≥ p, the random search direction of the ith. If x_best and y_best. It was shown in formula (4):Drosophila was set near the positions of i-th Drosophila individual was calculated.Step 4—calculated concentration determination value: The distance between the location and the point of the i) between the location and the point of the first Drosophila individual is calculated, as shown in formula (5):The distance :According to the distance, the concentration determination value i)) of the ith Drosophila: The concentration evaluation function was the optimization function, which was set as fit(iS), the relationship of Smell(i) and fit(iS), as shown in formula (7):Step 5—calculating the concentration :Step 7—preserve the optimal concentration and coordinates: The concentration determination values (bestSmell) and position coordinates of the optimal individual were preserved. Drosophila were directed to fly quickly to that location depending on visual perception, which was shown in formula (9):x_best, y_best) were taken as the optimal initial position for the next optimization, and the probability of congestion degree was reduced, as shown in formula (10):λ < 1, λ is the probability attenuation coefficient of congestion degree.The coordinates  Construction of Optimal FunctionFor probabilistic neural networks, the selection of appropriate smoothing coefficients was to improve the network’s ability to classify samples correctly. Therefore, the fitness function (optimization function) for solving the smoothing coefficients of probabilistic neural networks by using the optimization algorithm was set as follows (11):For known samples, training samples and test samples were divided. The training samples were only used to construct or determine the structure of probabilistic neural networks, not to optimize the smooth coefficients of networks. The test samples were not used for network construction, but only for optimizing network parameters and evaluating network performance.(2) Parameter optimization of probabilistic neural network based on the improved Drosophila optimization algorithmThe specific steps of optimizing probabilistic neural network with the improved Drosophila optimization algorithm proposed in this paper were as follows:s, the parameters to be optimized for probabilistic neural networks were determined to be σ1, σ2, … sσ. In the definition domain of optimization parameters, optimization parameters were initializedStep 1—constructing probabilistic neural network: According to the specific problems to be solved and the number of known samples, a probabilistic neural network model was constructed. If the number of modes was Step 2—setting parameters of the Drosophila optimization algorithm: According to the complexity of probabilistic neural network, the optimum algorithm (MaxGen) was chosen to maximize the number of iterations and the size of Drosophila population. Optimal location parameters and crowding probability values of Drosophila optimization algorithm were set.Step 3—setting up the mechanism of chaos generation: logistic mapping was selected as the generation mechanism of chaotic sequence, and 2000 iterations are carried out to eliminate the influence of initial value selection on the optimization results.Step 4—random search of Drosophila: Drosophila individuals are used for chaotic search.Step 5—calculated concentration determination value: The concentration determination value was calculated.Step 6—calculating individual concentration of Drosophila melanogaster: The fitness function value was obtained to calculate the concentration value of Drosophila individual’s position.Step 7—preserving the current optimal individual: Drosophila with the highest odor concentration was retained as the current optimal individual.Step 8—preserving optimal concentration and coordinates: The concentration determination value (bestSmell) of the optimal individual and its location coordinates were preserved, and the probability attenuation coefficient of congestion degree was adjusted adaptively.Step 9—iterative optimization: Steps 4 to 7 were repeated to determine whether the current optimal concentration is updated or not. The specified number of iterations was reached, or the algorithm was converged.The flow chart of optimizing probabilistic neural network with improved Drosophila optimization algorithm was shown in On the basis of four-point arc preliminary positioning, the initial position is used as the input of probabilistic neural network. The standard Drosophila optimization algorithm and the improved Drosophila optimization algorithm were used to locate the damage source. The results of damage source localization are shown in As can be seen from Furthermore, the HHT spectra of acoustic emission signals of carbon fiber braided composites  were useBy transforming the problem of acoustic emission damage signal identification of carbon fiber braided composites into a mathematical problem of objective function optimization, the problem of damage source localization of three-dimensional braided composites was solved based on “two-step method”. In order to overcome the shortcomings of the Drosophila algorithm in identifying the key parameters of probabilistic neural network, the algorithm was improved based on the idea of chaos optimization and fish swarm optimization. Through numerical calculation and experimental damage source location of different specimens, the results showed that the improved method had significantly improved the identification accuracy of different damage conditions. The optimized probabilistic neural network can locate the damage source accurately in each damage stage of tensile test. In this paper, an improved damage location method was proposed to reduce the error of the conventional location method. This method can locate the damage of carbon fiber braided composite materials in time and accurately, and improve the efficiency of nondestructive testing of composite materials.Through this study, the internal damage signal of carbon fiber braided composites under tensile load can be obtained by acoustic emission detection technology. The time domain and frequency domain characteristics of AE signals were analyzed by HHT, and the corresponding relationship between tensile damage mode and AE signal was established. In the first step, the sound velocity was compensated by combining the time-frequency analysis results with the anisotropy of the sample, and the initial localization was carried out by using the four-point arc method of time difference localization. In the second step, probabilistic neural network combined with the standard Drosophila optimization algorithm was used to locate the target in an error range of about ±5%. The construction of smoothing parameters and functions in probabilistic neural network was optimized to reduce the number of iterations. The positioning accuracy error of the improved Drosophila optimization algorithm was about ±1% and the positioning accuracy was improved.