N2 − N)/4, where N is the number of intentions.With the development of artificial intelligence and robotic technology in recent years, robots are gradually integrated into human daily life. Most of the human-robot interaction technologies currently applied to home service robots are programmed by the manufacturer first, and then instruct the user to trigger the implementation through voice commands or gesture commands. Although these methods are simple and effective, they lack some flexibility, especially when the programming program is contrary to user habits, which will lead to a significant decline in user experience satisfaction. To make that robots can better serve human beings, adaptable, simple, and flexible human-robot interaction technology is essential. Based on the neural mechanism of reinforcement learning, we propose a brain-inspired intention prediction model to enable the robot to perform actions according to the user's intention. With the spike-timing-dependent plasticity (STDP) mechanisms and the simple feedback of right or wrong, the humanoid robot NAO could successfully predict the user's intentions in Human Intention Prediction Experiment and Trajectory Tracking Experiment. Compared with the traditional Q-learning method, the training times required by the proposed model are reduced by ( The research trend of the new generation of robots is to make robots participate in human life, and improve the naturalness and flexibility of interaction between humans and robots through human-robot interaction technology. Robots that can successfully predict user's intention and take appropriate actions according to the intention can effectively improve interaction efficiency and user experience. Researchers have made significant progress in user intention prediction modeling. These studies use a variety of frameworks or models to enable robots to predict users' intentions in specific human-robot interaction tasks. These frameworks or models use a variety of methods, such as probabilistic graphical models, deep learning techniques, and other methods that include extreme learning machine algorithms, etc.There are many studies on the application of the probability graph model to human intention prediction. Song et al.  proposesDeep learning techniques, especially deep long short-term memory (LSTM) neural network, have also been used to predict human intentions. Yan et al.  presentsvia the extreme learning machine (ELM) algorithm, which realizes the robot's learning and prediction of human hand-over intentions in collaborative tasks. The experimental results show that the framework can enable the robot to effectively predict the human hand-over intention and complete the hand-over task. Since the framework enables robots to learn through human demonstrations, it can reduce human manual-programming efforts and improve the efficiency of human-robot collaboration. Lin et al.  mechanisms and the simple feedback of right or wrong.The architecture of the brain-inspired intention prediction model is shown in The dorsolateral prefrontal cortex (DLPFC) is responsible for representing state information  are responsible for action selection  is a critical area for controlling the execution of movement  play important roles in reward cognition  is considered as a critical frontal region for memory formation  The visual information of image category 1 is input into DLPFC, and all neurons representing this category in DLPFC are activated (orange neurons in DLPFC); (b) After the synaptic weight matrix calculation between DLFPC and BG, the neuron representing intention 1 in BG is activated (orange neuron in BG); (c) Thalamus receives the results of BG  and passes the information to PMC to control motor generation (orange neuron in PMC). 2. The positive reward process, if the user gives positive reward into SNc/VTA: (a) The OFC_1 receives the stimulation form Thalamus (blue neuron in OFC_1). The SNc/VTA combines the information from PMC and transmits the stimulation to OFC_2. And all neurons representing this action are activated (green neurons in OFC_2). (b) Then the stimulation of OFC_1 and OFC_2 are transmitted to MOFC (blue neuron in MOFC) and LOFC respectively, and LOFC is simultaneously inhibited by MOFC (green neurons in LOFC). (c) MOFC transmits the information to BG via StrD1 and to DLPFC at the same time. LOFC transmits the information to BG via StrD2 and to DLPFC at the same time. (d) The synaptic weight between DLPFC and BG is updated according to the time difference between the neurons firing. In short, the connection between image category 1 andintention 1 is strengthened, and the connections between image category 1 and otherintentions are weakened. 3. The negative reward process, if the user gives negative reward into SNc/VTA: (a) The OFC_1 receives the stimulation form Thalamus, then transmits the information to LOFC. (b) LOFC transmits the information to BG via StrD2 and to DLPFC at the same time. (c) The synaptic weight between DLPFC and BG is updated according to the time difference between the neurons firing. In short, the connection between image category 1 and intention 1 weakened,while the connections between image category 1 and other intentions remainedunchanged.In order to describe one training process of the model more directly, in a, b, c, and d are dimensionless parameters. If the membrane voltage v is greater than 30 mV, the membrane voltage and the recovery variable are reset according to the Equation (2). I is input, calculated by Equation (3). The Wij is the synaptic weight between presynaptic neuron and postsynaptic neuron, and the Oij is the output of presynaptic neuron. If multiple neurons fire at the same time, the neuron with the largest membrane voltage will inhibit other neuronal firings. The dimensionless parameters c and d are the same in different areas, they are c = −65 and d = 8. And the dimensionless parameters a and b of the neurons in StrD1, StrD2 and other areas are set as a = 0.01, b = 0.01; a = 0.1, b = 0.5; a = 0.02, b = 0.6, respectively.We use the Izhikevich neuron model to build the computational model. The Izhikevich neuron model achieves a good balance in biologically plausible and computational efficiency . In the computational model, synaptic plasticity occurs between DLPFC and BG, and the synaptic weight is fixed between other brain areas.Spike Timing Dependent Plasticity (STDP) is an important learning mechanism in the biological brain, which updates the synaptic weight between presynaptic and postsynaptic neurons according to the time difference between their firing , and to learn new intentions when human intentions change. After 12 gestures and 12 intentions are defined, the user can define the gesture-intention corresponding rules in his mind. The user makes gestures and the robot recognizes the gesture. Then the robot predicts the user's intention and performs the corresponding actions according to the proposed model. The user gives the right or wrong feedback according to whether the robot's action complies with his intentions. The robot can successfully predict the user's intention through multiple interactions. If some of the user's gesture-intention rules change, the robot can continue to learn those changed rules through interaction based on the learned model, and the unchanged rules are not affected, that is, the robot does not need to relearn all the rules.The predefined 12 gestures are shown in The predefined 12 intentions are shown in To make the experiment more intuitive, the initially defined gesture-intention correspondence rules are shown in After learning the gesture-intention corresponding rules, the user modified the corresponding rules , as well as the position information of the left and right hands compared with the left and right shoulders . These neurons determine whether to fire based on the three-dimensional coordinate information of the joint.After obtaining the gesture features, we use an unsupervised learning algorithm based on the STDP mechanism for gesture recognition. When a gesture is detected, the correlation coefficient of neurons firing pattern between the detected gesture and the learned gesture is calculated, and the activated target neuron is determined according to the correlation coefficient. If the correlation coefficient is very small, the gesture is determined as a new gesture, and a new target neuron is activated. The synaptic weights update between the new gesture and the new target are based on the STDP mechanism.The method is an online learning method, and the recognition accuracy increases with the increase of the number of training samples. We define a trial training set that contains 12 types of gestures, each of the gestures is performed once. After the previous trial training ends, the next trial continues learning on the trained model. Test at the end of each trial. The test set consisted of 12 types of gestures performed 30 times each, with a total of 360 samples. When the training of the sixth trial is completed, the gesture recognition accuracy is 98.33%.The method is an online learning method, and the recognition accuracy increases with the increase of the number of training samples. The training set consists of repeated batch training sets. We define a batch training set that contains 12 types of gestures, each of the gestures is performed once. That is, there are 12 samples in a batch training set, which are different types of gestures. A batch training indicates that the model is trained on the batch training set. The online learning method of the model is realized in the following ways: after the previous batch training ends, the next batch continues learning on the trained model. Test at the end of each batch training. The test set consists of 12 types of gestures, each of which is executed 30 times. The test set includes 360 samples. When the training of the sixth batch is completed , the gesture recognition accuracy is 98.33%.The user makes gestures randomly, and the robot predicts the user's intention and performs the corresponding action through the proposed model. Then, the user gives right or wrong feedback according to the robot's action and the initially defined gesture intention correspondence rules. The experiment is repeated many times until the robot could successfully predict the user's intention. The synaptic weights between DLPFC and BG are shown in The Trajectory Tracking Experiment can make the robot learn to walk along the track only through the right and wrong feedback of the remote control.The training scenario and test scenario are shown in We detect the trajectory in the image through traditional image processing methods such as image binarization, edge detection, and Hough transform, and classify the trajectory according to its image characteristics. Finally, we use six neurons to implement an abstract representation of the results. The detection method is simple and effective, which can ensure the robot identifies the trajectory with high accuracy.In the training scenario, the robot completed the training by walking two times along the trajectory clockwise and two times counterclockwise. In the test scenario, the robot can successfully complete the trajectory tracking experiment.First, we test the training times required by the Q-learning method (more details can be found in The result of detailed comparison between Q-learning method and the proposed model is shown in Trainimprove) of the proposed model on training times can be calculated by Equation (6). The TrainQ is the training times required by Q-learning method, and the TrainBIIP is the mode of training times required by the proposed model under the given intention numbers.As can be seen from Finally, we compared the training times required by the two methods when the number of intentions is 1–50. The experimental result is shown in N2 − N)/4, where N is the number of intentions.Based on the neural mechanism of reinforcement learning, we propose a brain-inspired intention prediction spiking neural network model to enable the robot to perform actions according to the user's intention. With the STDP mechanisms and the simple feedback of right or wrong, the humanoid robot NAO could successfully predict the user's intentions in Human Intention Prediction Experiment and Trajectory Tracking Experiment. Compared with the traditional Q-learning method, the training times required by the proposed model are reduced by  and the prediction intention (such as Intention1) to ensure that the user's intention can be correctly predicted when the same visual input is received in the future. Meanwhile, MOFC inhibits LOFC, then LOFC transmits the information to BG via StrD2 and to DLPFC at the same time. This pathway is used to reduce the synaptic weight between the future visual inputs  and the currently predicted intentions (Intention1), avoid new visual inputs to choose the intentions that have been learned (Intention1), and promote new visual inputs to select other unlearned intentions (Intentionothers).Compared with Q-learning method, the direct reason that our model needs fewer training times is the inhibition of LOFC by MOFC in the process of positive reward processing. The positive reward process indicates that the robot successfully predicted the user's intention. MOFC transmits the information to BG We propose a brain-inspired intention prediction model based on the neural mechanism of reinforcement learning. We deploy the model on the humanoid robot NAO, and verified the effectiveness of the model through Human Intention Prediction Experiment and Trajectory Tracking Experiment. The experimental results show that the robot could successfully predict the user's intentions only through the simple feedback of right or wrong. In this way, the robot can quickly learn new rules without interfering with the learned and unchanged intention rules. The proposed model is simple and effective, which can effectively improve the flexibility and simplicity of human-robot interaction.In our future work, we will combine our previous work in affective states recognition , more details could be found at https://github.com/BrainCog-X/Brain-Cog. Further inquiries should be directed to the corresponding author.The python scripts can be downloaded from the GitHub repository of the brain-inspired cognitive intelligence engine at Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences: All authors conceived the initial idea, designed the model, carried out the experiments, and wrote the manuscript.This work is supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (Grant No. XDB32070100), the new generation of artificial intelligence major project of the Ministry of Science and Technology of the People's Republic of China (Grant No. 2020AAA0104305), the Beijing Municipal Commission of Science and Technology (Grant No. Z181100001518006), the Key Research Program of Frontier Sciences, CAS (Grant No. ZDBS-LY-JSC013).The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Mobile communication has achieved enormous technology innovations over many generations of progression. New cellular technology, including 5G cellular systems, is being deployed and making use of higher frequencies, including the Millimetre Wave (MMW) range (30–300 GHz) of the electromagnetic spectrum. Numerical computational techniques such as the Finite Difference Time Domain (FDTD) method have been used extensively as an effective approach for assessing electromagnetic fields’ biological impacts. This study demonstrates the variation of the accuracy of the FDTD computational simulation system when different meshing sizes are used, by using the interaction of the critically sensitive human cornea with EM in the 30 to 100 GHz range. Different approaches of base cell size specifications were compared. The accuracy of the computation is determined by applying planar sensors showing the detail of electric field distribution as well as the absolute values of electric field collected by point sensors. It was found that manually defining the base cell sizes reduces the model size as well as the computation time. However, the accuracy of the computation decreases in an unpredictable way. The results indicated that using a cloud computing capacity plays a crucial role in minimizing the computation time. The Millimetre Wave (MMW) of Electromagnetic (EM) radiation is defined as radiation in the 30–300 GHz range. The high frequency and propagation characteristics make MMWs extremely attractive for 5G wireless communication systems and industrial and imaging applications. Emerging “5G and beyond” communication systems deliver data rates in the order of 100 Megabits to 20 Gigabits per second and lower latency for many simultaneous users. Given the increasing use of the MMW band in emerging telecommunication systems, it is important to determine the likelihood of adverse impacts on the human cornea. The diagnostic imaging applications of MMWs for in vivo assessment of corneal hydration have been also of great interest and is a subject of ongoing study [The cornea consists of 78% of water. Water has a very high dielectric constant that is highly absorptive to high GHz and THz EM radiation. Hence, MMW and THz are sensitive to water content in the tissue. Moreover, the extreme homogeneity and lack of physiological variations of the cornea compared to other organs and tissue in the body allow high GHz and THz imaging to be a promising imaging modality in ophthalmology .The dielectric properties of water in the 6–300 GHz range result in an absorption coefficient  and the Institute of Electrical and Electronics Engineers specifies exposures in this frequency band in terms of the absorbed power density (te (SAR) ,8.The Finite-Difference Time-Domain (FDTD) method has become a preferred method for conducting EM simulations for biological impacts from wireless devices . FDTD ofA smaller model of the relevant tissue may reduce the accuracy of the modeling more than the change in the voxel cell size, since a representative region may not be able to be properly characterized. There is also the consideration of the timestep. A short duration timestep improves the accuracy of the simulation but increases simulation time.c is the speed of light, and The FDTD technique solves the differential time-domain Maxwell’s equation (Equations (1) and (2)), which means that the calculation of the EM field values progresses at discrete steps in time.D method . The timVilagosh et al. have examined absorption by the skin and ear canal using the same computational approach by implementing XFdtd numerical software ,11; hencA high-resolution computational model of the human cornea has been developed in this study, which harvests large quantities of information and is capable of easy optimization. The model incorporates the features of a realistic cornea and surface hydration by including the tear film.Since the cell sizes decrease in three dimensions, the computational demands on representational models increase as the cube of the frequency; a model at 100 GHz will have 37 times more voxel cells than one at 30 GHz. The question of how changing the voxel cell size (expressed in terms of the wavelength) influences accuracy was explored in a set of FDTD simulations. This paper outlines the detail of the elements of the Yee cell meshing and the dielectric properties of the ocular tissues.XFdtd Bio-Pro  is a three-dimensional full-wave EM solver based on the FDTD method. The solver was used as the platform for the anatomical design and the computational implementation of the simulations.The mathematical models are often limited by computational capability, necessitating oversimplified anatomy. The computational complexity and memory requirements of the FDTD algorithm solving Maxwell’s equations is discussed in .In this study, XFdtd Bio-Pro was used utilizing a cloud computing environment .Given the nature of the problem, the direct study of human subjects is hazardous. Designing a mathematical model is a viable and attractive alternative to predicting the cornea’s temperature elevation due to MMW exposure.The cornea is elliptical with non-identical vertical and horizontal diameters. The anterior horizontal diameter is 12 mm, and the anterior vertical diameter is 11 mm. If viewed from the posterior, the cornea appears circular, with horizontal and vertical diameters of 11.7 mm . For theIn the actual eye, the size of the pupil varies as a result of different lighting conditions. However, in this model, the pupil is assumed to have a constant diameter when the light intensity is low. The blood flow inside the choroid is one of the principal heat sources within the eye. Compared to the blood flow inside the choroid, the blood flow inside the iris and ciliary body are relatively small; thus, the thermal model presented in this study has ignored the presence of blood flow in this space. Finally, a buffer cylinder is placed at the model’s rear to diffuse reflected waves. For this purpose, the refractive index is defined to be the same as the back of the model, and the absorption coefficient is set to be higher to minimize internal reflection σ=ε0The FDTD grid defines the E-field at the edges of cells. The Specific Absorption Rate (SAR) value is referenced to the center of the FDTD cell and is formed by summing the contributions of twelve E-fields on the edges of the FDTD cells.SAR is the basic metric for specifying RF heating, which is the rate of energy transferred relative to the tissue mass in (W/kg) and is defined by Equation .(5)SAR=Equation ) is used related , for theThe basis for the simulation is the construction of models that would represent the scenario where the anterior region of the eyeball is exposed to MMW at different frequencies. In XFdtd, an excitation source can be applied externally in the form of incident plane waves for scattering calculations or Gaussian beams for optical frequency calculations.The radiation source is directed toward the anterior eye surface, perpendicular to the corneal eye cross-section. The excitation was set at 3.0 V/m in amplitude at 30, 60, and 100 GHz. A planar E-field sensor  was placed vertically, bisecting the model in the sagittal plane as illustrated in The simulation’s accuracy depends upon the ratio of the wavelength of excitation to the Yee cell (mesh size). Electric material properties are assigned to the appropriate cell edge during the meshing operation. For a homogeneous media, a rule of thumb value of ell size . In thisXFdtd creates the mesh by examining where each piece of geometry in the simulation intersects the grid and applying the material assigned to that piece of geometry to the appropriate Yee cell edge and faces. Assigning meshing priority is a critical step in the meshing configuration. When tissues with different dielectric properties overlap, determining which dielectric properties should be applied to the mesh becomes difficult. To avoid sending a questionable mesh to the calculation engine for simulation, the meshing priority was given to the tear film and the cornea as shown in The cell sizes were manually specified in the Grid editor. In XFdtd, PrOGrid Project Optimized Gridding is a set of Gridding tools that simplify the process of finding the optimal trade-off between runtime cost and accuracy. When the PrOGrid is unchecked, the base cell sizes can be entered manually for each Cartesian direction. This was done to compare the variation in cell sizes and their impact on the duration and the accuracy of the simulations. As noted previously, when defining the cell size for a Grid region, it is essential to specify the cell size much less than the smallest wavelength. Various target cell sizes in the range of 18 to 60 cells per wavelength are manually specified to resolve small geometric features.For adjustments of the mesh properties for individual ocular tissues, the automatic cell size mode was used . This option enables parts with electric and magnetic materials, other than free space, to be included in the mesh. The PrOGrid option increases grid resolution in high permittivity dielectrics. When this option is used, the base cell sizes are determined from the project’s frequency range of interest and the minimum cells per wavelength setting. The values of 18, 25, 30, and 60 were chosen for the minimum cells per wavelength.To ensure that accurate results are obtained, the defined target cell size for a grid region needs to be significantly smaller than the smallest wavelength.A commonly applied constraint is ten cells per wavelength, meaning that the length of any grid edge should be less than one-tenth of the wavelength at the highest frequency  of interest. It is necessary to consider a smaller FDTD grid cell size to provide adequate geometry resolution .As noted earlier, the mesh density (grid cell size) has a major impact on the accuracy of the results. There is an exponential growth of model size and computing time with decreasing mesh cell size. The changes associated with variations in grid cell size in the constructed human eye model are illustrated in Each simulation was run for 10,000 timesteps. The planar sensor begins to gather data approximately at 7 ps into the simulation. Numerical results are shown in The XFdtd enables precisely determining the E-field distribution within biological tissues. In this study, the computational accuracy is assessed by comparing the E-field distribution in the eye model, which is shown in Analysis of the computational results have shown that EM radiation is highly absorbed by anterior eye tissues . The possibility of substantial penetration of MMW radiation of 30–100 GHz radiation into the ocular tissues cannot be ignored . ComputaWhen the PrOGrid (adaptive mode) is on, the minimum cells per wavelength, as well as the wavelength of the project’s frequency of interest, set the base cell size of the grid. Increasing the value of minimum cell per wavelength reduces cell edge length, thus increasing accuracy and computation time. It is worth mentioning that the size of the smallest geometric feature in the project is crucial to resolve, which is known as Minimum Feature Size in the XFdtd PrOGrid option. This is automatically defined when the PrOGrid is checked, which significantly enhances the accuracy of the result. Using the automatic base cell size setting provides reasonable grid cell sizes for the range of frequencies being simulated. It also increases grid resolution in high permittivity dielectrics and captures small geometric features with multiple grid cells. However, this option consumes approximately 10 times CPU time and 40 times RAM when defining smaller cell sizes. For instance, halving the cell size for a radiation source of 30 GHz increases memory requirements 4.9-fold, resulting in a 3.5-fold increase in the computation time.The manual cell size setup, on the other hand, shows a considerable decline in the computation time as well as the required computer resources. The detailed distribution of the E-field in the anterior region of the eyeball  indicateWhen using an adaptive PrOGrid option, small geometric features are captured with multiple grid cells. It seems that manually setting up the cell sizes results in unwanted radiation artifacts, making the simulations inaccurate B. This cChoosing a cell size in the range of twenty cells per wavelength is a good starting point for the current model where small cells are required to resolve small geometric features . Reducing the cell sizes are always at the expense of additional runtime and simulation memory. A smaller cell size such as Since the FDTD method calculates the E-field and H-field at every cell in the whole computational problem space at every timestep, the current constructed model has the capacity to have multiple sensors  inside or outside the model itself. The only limitation of this method is the data storage capacity and the computational time. The computational resource estimations underline the fact that this model with multiple ocular layers is not potentially suitable for desktop computer users.As mentioned earlier, defining accurate base cell sizes for reasonable computations using the FDTD technique requires a high-end computer with several hundred megabytes of computer memory. In this study, the largest simulation with a multi-layer eye model took approximately 153 min using 32 processing cores and 128 GB of memory in a cloud computing environment.The E-field distribution calculation required much higher computation time  using a desktop computer with six processing cores and 48 GB of memory, as compared to a cloud computing environment with 32 processing cores and 128 GB of memory .According to our numerical simulation, using a cloud computing environment with more processing cores and memory can be helpful in reducing the computation time for studying biological effects. Additionally, using an adaptive cell size setup is an effective approach for obtaining accurate computational results.In this study, the human cornea with realistic anatomical layering was simulated using an FDTD solver (XFdtd) to assess the MMW absorption in the anterior region of the eyeball. This study focused on assessing the impact of grid (cell) size on the E-field distribution and E-field strength vs. time. Defining smaller grid sizes leads to more accurate results. However, the computational efficiency is substantially impacted by the need to utilize the timestep corresponding to smaller grid sizes throughout the simulation space. The results of presented simulations suggest that using an adaptive cell size specification  provides fewer radiation artifacts, resulting in more accurate computational simulations.Furthermore, we found that adaptive cell size setup radically increased the required computation time compared to manually specifying the cell sizes. We have utilized a cloud computing environment to process the simulations in an appropriate amount of time.To sum up, using an adaptive cell size specification proved to be not only useful at lower frequencies, but also practical for biological studies at higher frequencies.A future research direction for this paper will be an in vivo experimental evaluation of the result obtained from the computational model discussed in this paper.


As a power source for autonomous underwater vehicles (AUVs), lithium-ion batteries play an important role in ensuring AUVs’ electric power propulsion performance. An accurate state of charge (SOC) estimation method is the key to achieving energy optimization for lithium-ion batteries. Due to the complicated ocean environments, traditional filtering methods cannot effectively estimate the SOC of lithium-ion batteries in an AUV. Based on the standard extended Kalman filter (EKF), an adaptive iterative extended Kalman filter (AIEKF) method for the SOC in an AUV is proposed to address the traditional filter’s problems, such as low accuracy and large errors. In this method, the adaptive update is introduced to deal with the uncertain noise from the lithium-ion battery. The iteration is used to improve the convergence speed and to reduce the computational burden. Compared with the EKF, iterative extended Kalman filter (IEKF) and adaptive extended Kalman filter (AEKF), the proposed AIEKF has a higher estimation accuracy and anti-interference capability, which is suitable for the AUV’s SOC estimation. In addition, based on the second-order equivalent circuit model of the lithium-ion battery, a forgetting factor recursive least squares (FFRLS) method is proposed to deal with the multi-variability problem. In the end, four different methods, including EKF, IEKF, AEKF, and the proposed AIEKF, are compared in computational time. The experiment results show that the proposed method has high accuracy and fast estimation speed, meaning that it has good application potential in AUVs. AUVs are multi-functional underwater vehicles capable of autonomous propulsion. A battery provides the power source for an AUV’s propulsion, its range, and the development of various technologies. Therefore, it is critical to select a high-performance underwater power battery technology for AUVs. Lithium-ion batteries with a low self-discharge rates, high energy ratios, and long cycle life are reliable solutions for AUV energy storage. The SOC of lithium-ion batteries reflects the power consumption condition and remaining capacity of the battery, and SOC estimation is an important function of the battery management system (BMS) ,2,3. BecAt present, many methods for SOC estimation have been proposed. The ampere-hour counting method  is simplIn order to overcome the shortcomings of the traditional EKF in terms of SOC estimation accuracy, an iterative part is added to the EKF. The IEKF updates the Jacobi matrix of the observation equation by bringing the obtained a posteriori estimates into the observation equation, and in each iteration of the calculation process. The estimated value is continuously kept close to the true value by continuously using the observed value during each iteration of the calculation. The absolute value of the difference between the estimated terminal voltage and the measured terminal voltage is set as the threshold value to determine whether to perform the iteration or not. When the voltage error value is less than the threshold value, this process meets the accuracy requirements, and the iteration process is not required. This avoids unnecessary iterative processes and reduces the computational burden. When the voltage error is greater than the threshold value, an iterative process is required. The uncertainty due to system noise affects the accuracy of SOC estimation. To solve this problem, we incorporated the adaptive update scheme. An improved Sage–Husa estimator was used for adaptive updating of process noise and measurement noise, which overcomes the fluctuation of noise due to the influence of external factors. Based on the above, this paper proposes the adaptive iterative extended Kalman filter.4 batteries to verify the reliability of the proposed method. Finally, The rest of the paper is structured as follows. The second-order equivalent circuit model is more accurate than the first-order equivalent circuit model because increasing the number of RC networks can improve the accuracy of the model. The second-order equivalent circuit model has less computational complexity than other higher-order circuit models and better reflects its polarization characteristics . TherefoThe second-order equivalent circuit model equation can be derived from SOC reflects the remaining capacity of the battery and is defined as the ratio of the remaining capacity of the battery to the total capacity with the following formula:According to the second-order RC equivalent circuit model of the lithium-ion battery, the current is taken as the model input, and the voltage Select In order to establish an accurate battery model, multiple unknown parameters in the model need to be identified. In this study, an online parameter identification method based on the FFRLS algorithm was used to reduce the influence of historical data on new data during the recursive process by setting the forgetting factor and continuously updating the model parameters to obtain more accurate battery parameters.The Laplace equation of the battery model is obtained from Equation (1).The above equation is discretized using a bilinear variation, so that Equation (1) can be rewritten as a difference equation.k is Let The recursive least squares formula with the forgetting factor is as follows:The relationship between the battery model parameters and coefficients is:The hybrid pulse power characterization (HPPC) experimental data are used for parameter identification, and the parameter identification results are shown in In this paper, an AIEKF based on the Sage–Husa maximum a posteriori estimator is proposed for battery SOC estimation. The EKF linearizes the nonlinear system by truncating the higher-order terms through Taylor series expansion. Although the EKF improves the algorithm’s ability to handle nonlinear systems and simplifies the computational process, the resulting higher-order loss errors lead to a decrease in estimation accuracy. In order to solve the above problems in EKF, some optimization improvements are made to the EKF algorithm to reduce the influence of the above factors. The iterative idea is introduced in which the posterior estimate is substituted into the observation equation and the Jacobi matrix of the observation equation is updated by means of a multiple iterative measurement update process. By repeatedly using the observations, the estimates are continuously approximated to the true values during each iteration of the calculation. The absolute value of the difference between the measured end voltage and the estimated end voltage is used as the threshold value, and when the voltage error value is less than the threshold value, the iterative process is not required to reduce the computational burden. When the voltage error is greater than the threshold value, an iterative process is required. Because the system in the implementation of the estimation process will be affected by interference and random parameters, the process noise covariance and measurement noise covariance are constantly changing, which usually produces large estimation errors and affects the accuracy and robustness of SOC estimation. Incorporating the Sage–Husa maximum posteriori estimator and adaptively updating the process noise covariance matrix and measurement noise covariance matrix in real-time can improve the estimation accuracy and convergence speed of the algorithm under unknown system noise conditions, and further improve the filtering effect of the algorithm.The nonlinear system model can be described as:Initialization:Time to update:Measurement update:Let the number of iterations be The absolute value of the difference between the measured terminal voltage and the estimated terminal voltage is used as the threshold value, and the threshold value is set to Adaptive update:Adaptive correction for process noise and measurement noise using an improved Sage–Husa estimator.The method uses the measurement data for recursive filtering while estimating and correcting the statistical characteristics of the system process noise and measurement noise in real-time, thus achieving the purpose of adaptive filtering.4 battery with a nominal capacity of 6 Ah was selected as the experimental test object.In order to obtain the experimental data of battery current, voltage and capacity, we set up an experimental platform consisting of the host computer and the NEWARE BTS-4000 battery charge/discharge test system for experimental verification, whose structural configuration is shown in The discharge characteristics of the battery were experimented with by the HPPC test method ,31. The In order to verify the effectiveness and superiority of the proposed method in terms of estimation accuracy, the proposed AIEKF is compared with the AEKF, IEKF, and EKF in this paper. The SOC estimation of the lithium-ion battery was performed under HPPC operating conditions and dynamic stress test (DST) operating conditions, and the estimated results are shown in t, t, and In this paper, the maximum estimation error (MAE) and the root mean square error (RMSE) are used as performance indicators to evaluate SOC estimation. The MAE and RMSE are calculated using Equations (21) and (22). The results of the MAE and RMSE calculations for the two operating conditions are shown in The performance of the lithium-ion battery under operating conditions is analyzed under HPPC operating conditions. In order to further verify the reliability of the proposed method, a more complex DST condition was used to experiment with and analyze the lithium-ion battery. The DST current condition is shown in The results show that the AIEKF-based SOC estimation method proposed in this paper can maintain lower SOC estimation error, higher tracking accuracy and lower fluctuation under two current conditions compared with the AEKF, IEKF and EKF. The MAE and RMSE of AIEKF were calculated as being less than 1%, and the data verify that the estimation accuracy of AIEKF was better than that of AEKF, IEKF and EKF, so AIEKF can effectively improve the SOC estimation accuracy. Three temperatures of –20, 0, and 30 °C were selected to verify the estimation effect of the proposed method at different temperatures under DST operating conditions. The SOC estimation was performed at the three temperatures with and without considering the temperature effect, and the results are shown in By evaluating the SOC estimation performance of different KF family algorithms using model-based algorithms in previous studies with the proposed algorithm in this paper, the comparison results are shown in Although the AIEKF proposed in this paper can improve the accuracy of SOC estimation for lithium-ion batteries, the computational burden is still affected to some extent. The discrete-time model with a sampling period of 1 s was simulated using MATLAB/Simulink R2020b on a desktop with an Intel (R) Core (TM) i7-6500 CPU and 8 GB RAM. The computation times of the three estimation algorithms are listed in The SOC is a valuable parameter for the AUV’s control system to distribute electric energy to the propulsion system. In order to quickly obtain an accurate SOC, an improved AIEKF SOC estimation method for lithium-ion batteries was proposed in this paper. In addition, the FFRLS method was applied to update the parameters of the second-order RC equivalent circuit model of lithium-ion battery online to obtain an accurate model parameter. The computational accuracy was improved, and the unnecessary computational burden was reduced by setting a threshold value. An improved Sage–Husa estimator was used for an adaptive update optimization of process noise and measurement noise to eliminate the effect of noise. The validity of the AIEKF and its good estimation performance were demonstrated by experimental verification analysis with different currents and temperatures. The experimental results show that the RMSE of the SOC estimation results can reach 0.25%.Further research would focus on the battery aging effect on SOC estimation and the prediction of the SOH of lithium-ion batteries in AUVs so that a more intelligent battery energy management system for AUVs can be established to adopt to a long-term voyage in the ocean environment.


In the field of electronics manufacturing, electronic component classification facilitates the management and recycling of the functional and valuable electronic components in electronic waste. Current electronic component classification methods are mainly based on deep learning, which requires a large number of samples to train the model. Owing to the wide variety of electronic components, collecting datasets is a time-consuming and laborious process. This study proposed a Siamese network-based classification method to solve the electronic component classification problem for a few samples. First, an improved visual geometry group 16 (VGG-16) model was proposed as the feature extraction part of the Siamese neural network to improve the recognition performance of the model under small samples. Then, a novel channel correlation loss function that allows the model to learn the correlation between different channels in the feature map was designed to further improve the generalization performance of the model. Finally, the nearest neighbor algorithm was used to complete the classification work. The experimental results show that the proposed method can achieve high classification accuracy under small sample conditions and is robust for electronic components with similar appearances. This improves the classification quality of electronic components and reduces the training sample collection cost. Electronic components are important parts of electronic devices that are significant to signal amplification, power transmission, and Boolean operations. They are widely used in smartphones, laptops, and other electronic products owing to their excellent performance. However, with the advancement of technology, the research and development cycle of electronic products has become shorter, resulting in a faster iteration of electronic products; thus, many obsolete electronic products become electronic waste. These electronic wastes have many electronic components that are still intact. These electronic components contain materials valuable for producing electronic products ,2. ThereWith the development of deep learning, image classification techniques have made great progress owing to the excellent performance of convolutional neural networks (CNNs). They have become the primary method for solving image classification problems and have been widely applied in many fields ,4. Li etIn recent years, several researchers have focused on solving the problem of electronic component classification in the field of electronics manufacturing. Lefkaditis et al. presented a classification algorithm for electronic components which combines support vector machines (SVMs) and multilayer perceptrons (MLPs) with 92.3% accuracy . Atik us(1) The VGG-16 network was selected for the feature extraction part of the Siamese network. The shallow feature maps containing more fine-grained information were fused with the higher-level feature maps containing more semantic information in this model to boost the recognition performance of the model considering limited samples.(2) To improve further the generalization performance of the model, a channelcorrelation loss function was designed to assist the model in learning the correlation between different channels in the feature map.The remainder of this paper is organized as follows: The Siamese network is used to solve few-shot learning ,18,21,22Seventeen different electronic components were captured using a camera to verify the effectiveness of the method proposed in this study, as shown in F. The downsampling layer adopts a 3 × 3 convolutional kernel and stride size of 2. The distance between the flattened F and output of the other branch of the Siamese network is calculated. Specifically, Equation qi=fThe channel correlation loss function can then be defined asTherefore, the total loss is written asDuring training, the cross-entropy loss In this study, all experiments were conducted under the same conditions. The graphics processing unit adopted was GeForce GTX 1060 (6 GB). The model was built using the PyTorch deep learning framework. The total number of training epochs was 120, and the Adam optimizer  was usedlization  and the Since the Siamese neural network outputs the similarity of a pair of input images rather than the category to which the electronic components in the images belong, after the training was completed, we used the nearest neighbor algorithm to classify the images. Specifically, we first collected one sample from each category of electronic components and classified these samples into a new dataset. Then, we compared the test images with the images of each category in this dataset; the more similar the two images were, the higher the prediction score output by the Siamese neural network, and vice versa. Finally, we selected the category with the highest score as the classification result.In our experiments, we performed each set of experiments three times, and in each training we selected the model parameters with the best discriminative performance in the last 10 epochs for classification tests, and finally the model parameters with the best classification performance in these three results were selected.C denotes the number of correct classifications, and T denotes the total number of classifications performed. Additionally, the area under the receiver operating characteristic curve (AUC) was used to measure the discriminative performance of the model. The horizontal coordinate of the receiver operating characteristic (ROC) curve is the false positive rate (FPR), and the vertical coordinate is the true positive rate (TPR), both of which are defined as follows:In this study, the classification performance of the model was evaluated using an accuracy rate defined as follows:To verify the effectiveness of the method proposed in this study, the classification accuracy was tested when using VGG-16, VGG-16-F based on feature fusion, VGG-16 with channel correlation loss, and VGG-16 with both methods included as feature extraction subnets of the Siamese network for different sample numbers. Moreover, experiments with AlexNet , ResNet-The ResNet-50 accuracy was lower than that of ResNet-34 because the former has more parameters than the latter and is prone to overfitting. Given the same number of training samples, the accuracy of GoogLeNet was higher than that of the other three networks because GooLeNet uses a stacked inception block. It extracts features from images at different scales and reduces model complexity, but its accuracy is still lower than that of the proposed method.Therefore, combining the improved VGG-16 model with channel correlation loss improved the model’s classification accuracy while outperforming other outstanding classification networks.Electronic component classification in electronic waste facilitates better recycling. Current deep learning-based classification algorithms require many training samples to obtain high performance and do not provide a fine-grained classification level. This study proposed a Siamese network-based classification algorithm for electronic component classification with high accuracy under small sample constraints to address these issues. First, the low-level feature map of the VGG-16 model was extracted. A continuous downsampling operation was performed to improve the discriminative performance of the Siamese network by fusing the features with the high-level feature map. Second, a channel correlation loss function that allows the model to learn correlation information about different channels in the feature map was proposed to boost the model’s discriminative ability further. For inference, the nearest neighbor algorithm was used for electronic component classification. The experimental results show that the proposed method can achieve 0.94 accuracy with 15 training samples. This is higher than that of other CNNs used as the feature extraction part of the Siamese network, such as AlexNet, ResNet-34, ResNet-50, and GoogLeNet. The results confirm the effectiveness of the proposed method. Although this approach achieves better results, the method still suffers from classification errors for electronic components with similar appearances. In future work, we will apply the attention mechanism to our model to improve the robustness for similar electronic components and investigate the effect of the model on electronic component recognition under different backgrounds and image qualities. Furthermore, our method will be further applied to more types of electronic components and other small sample classification tasks.


It indicates that the prediction results of our proposed method fit more with the basic structure of the given traffic scenario in a long-term prediction range, which verifies the effectiveness of our proposed method.With the rapid development of artificial intelligent technology, the deep learning method is widely applied to predict human driving intentions due to its relative accuracy of prediction, which is one of critical links for security guarantee in the distributed, mixed driving scenario. In order to sense the intention of human-driven vehicles and reduce the self-driving collision avoidance rate, an improved intention prediction method for human-driving vehicles based on unsupervised, deep inverse reinforcement learning is proposed. Firstly, a contrast discriminator module was proposed to extract richer features. Then, the residual module was created to overcome the drawbacks of gradient disappearance and network degradation with the increase in network layers. Furthermore, the dropout layer was generated to prevent the over-fitting phenomenon in the whole training process of the GRU network, so as to improve the generalization ability of the network model. Finally, abundant experiments were conducted on datasets to evaluate our proposed method. The pass rate of self-driving vehicles with conservative driver probabilities of  With the development of artificial intelligence and electric vehicles, in the process of converting from a manual driving transportation system to an autonomous driving transportation system, there will inevitably be a stage of mixed driving with different levels of intelligent carriers. The essence of mixed traffic flow is the coexistence of various driving behaviors, without standards to speak of, but the lack of uniformity will lead to difficulties in decision-making. Since human behavior is the most difficult to predict, autonomous vehicles usually adopt conservative driving strategies in mixed traffic flow scenarios, resulting in them constantly being overtaken by manual driving vehicles in the congested traffic flow, and the efficiency of passage is difficult to guarantee ,2. How tTaking the uncontrolled T-intersection scenario as an example, in the absence of critical guidance facilities such as traffic lights, self-driving vehicles must interact with other vehicles in the main lane if they want to merge safely and efficiently from the T-intersection to the main lane . Since eThe latent traits of human-driven cars are divided into intent estimation and trait estimation . Intent Trait estimation infers the driving characteristics of drivers, such as driving style, driving preference, fatigue status, and degree of distraction, etc. Supervised and unsupervised learning are usually utilized to classify driving characteristics. Morton et al.  learned In recent years, contrast learning has been widely applied to learn feature information from continuous data such as video and pedestrian trajectories . Wang X (1)In order to improve the ability to learn the potential features of different drivers from the trajectory, contrast learning was proposed into the model, which used the minimization of contrast loss to learn the exclusive features of different drivers in the driver trajectory and enhanced the ability of the model to separate different features.(2)We introduced residual modules in the GRU model to capture detailed feature information with strong representational power. These stacked residual units greatly improved the training efficiency, ensuring that the network in the latter layer captured more feature information than the previous layer, reducing information loss. Moreover, the dropout layer was introduced to prevent the over-fitting phenomenon in the whole training process of the gated recurrent unit (GRU) network to improve the generalization ability of the network model.Inspired by contrast learning, this paper improves the VAE + RNN model and introduces contrast loss and the residual network to form the contrastive-ResNet-VAE model (C-ResNet-VAE) so that autonomous vehicles can better avoid people when driving through the uncontrolled intersections. The optimized contrast loss not only enhances the model’s ability to separate different features but also improves the model’s ability to learn the potential features of different drivers from the trajectory. Our main contributions are as follows:Kingma et al.  proposedDue to the fact that the encoder is unable to obtain the prior probability distribution of the hidden variable The VAE model adopts the Kullback–Leibler (KL) divergence  to evaluThe VAE model adopts the encoder to learn the posterior distributed parameter mean terature . SpecifiWe mostly researched two-way two lanes at an uncontrollable T-Intersection, shown in Liu et al. adopted VAE + RNN to infer drivers’ potential features from the trail of vehicles. However, the VAE network is restricted to deducing the abilities of potential features. We introduced contrastive learning and residual modules based on the VAE + RNN network. Meanwhile, we leveraged C-ResNet-VAE + RNN networks to extract the potential states of various drivers from original driving trails and clusters in an unsupervised way. In order to obtain the potential features of every driver from the trails of drivers, having the C-ResNet-VAE + GRU network extract the styles of driving was proposed. The C-ResNet-VAE network is composed of an encoder, contrastive learning classification, and decoder. The encoder squeezes the collected trail The data of drivers’ trails contains generous similar samples, so the VAE module poorly analyzes the drivers’ particular features. If it fails to distinguish the different features of every driver, it will influence the results of autonomous driving in a decision network, and the decoder will rebuild the original inputs from the VAE module. If the exclusive target of the module is to reconstruct sequences, something necessary and remarkable, such as drivers’ personal information, will be neglected. Here, we aimed to use contrastive learning to train the VAE module, which enhances the representational ability of potential variables.We compared the contrastive losses, which the potential variable learns in different drivers by using the contrastive classifier The formulation of contrastive losses is as follows:When the contrastive loss is minimum, i.e., x, the feature mark obtained by residual learning is F(x), and the output value is H(x), then this unit is expressed as:We improved the network based on VAE + GRU and presented a feature extraction network based on residual structure to capture detailed feature information with strong representation ability. Generally speaking, researchers mostly increase the number of network layers to improve the richness of feature information; with more layers or a wider network, the abstract level of feature information will gradually increase. From the initial acquired edge, information gradually becomes more representational semantic information. However, the number of network layers are not deepened infinitely. In the process of deepening the network layers, the model will have a foremost outcome. If the number of network layers continues to increase, the loss will increase accordingly. He et al.  presenteThe structure of residual unit is shown in As shown in This method alternates between the sample data by interacting with the environment and using random gradient rise to optimize the “agent” objective function. Assume The objective function of the PPO algorithm is:t) of The n spaces .n cars moving toward opposite directions in a two-way street and that all of the vehicles that were controlled by the Intelligent Driver Model (IDM) [Our simulation environment was at the uncontrolled T-intersections, and we assumed that there were el (IDM)  never toLet the state of the ego vehicle successfully making a full right turn be −4 and the weight of the KL divergence loss to β 5 × 10−8. The dataset used in this paper is a randomly generated trajectory dataset in Python produced by Liu et al. Because using Python to generate the dataset not only allows one to set the type of dataset needed, such as the number of entries allocated to radical and conservative trajectories in the program, it also saves the cost of obtaining these datasets. It contains approximately 700,000 driving trajectories from two types of drivers. We set the train/test split ratio is 2:1. We trained the policy model with 466,667 random trajectory data and tested with the other 233,333 trajectory data. We set the decaying learning rate to 5 × 10In this paper we proposed a network of C-ResNet-VAE + GRU and compared it with the VAE + GRU network  and GRU We trained two methods for 500 epochs and then utilized a set of test trajectories as inputs to act as encoders to measure if the latent trait effect was good or bad. The unsupervised classifier results of both methods are shown in The red areas represent aggressive drivers, the blue areas represent conservative drivers, and the middle, which is not fully separated out, usually contains very short trajectories and trajectories with vague front clearance. The lines closer to the boundary represent vehicle trajectories where the car has moved out of the range and we cannot see. As shown in Therefore, trajectories with different potential characteristics were mostly gathered together and not classified successfully. Study  utilizedFrom the results in (1)The supervised learning with labels proposed by Ma X et al. , which t(2)The strategy of ,18, and We used two baselines as comparative experiments:In addition, we used a reinforcement learning policy directly trained with truth labels as a baseline. We ran experiments with different proportions of two types of drivers and tested four models with 500 random cases. The percentage of auto-vehicles successfully taking a right turn to merge into the upper lane, colliding with surrounding vehicles and completing overtime, were calculated, respectively, where overtime refers to a situation where the car failed to make a right turn within the allotted time and did not crash. The results are shown in p is the probability for each surrounding driver to be conservative. The task difficulty decreased as p increased. From the experimental results, the collision rate of the proposed potential state feature extraction method was lower than the other three methods, and the successful completion of the task accounted for a higher proportion, which is closer to decision-making under the real label training. The main reason was that our method effectively extracted the trait differences of the surrounding drivers, which made better use of the reinforcement learning for the decision-making of the ego vehicle. Our policy was able to utilize the existing trait representation and focused more on the decision-making of the ego vehicle, which led to better navigation performance. The model in [p value became smaller. The reason is that the latent representation did not distinguish between different traits and only provided very limited useful information to learners. For Ma et al. [model in  had a loa et al. , both sta et al.  had limiAt time t, it can be seen from t + 1, it can be seen from At time t + n, it can be seen from n was set to 50 s.At time In this paper, the C-ResNet-VAE network was proposed to improve the existing deficiency of the VAE + RNN model and learn the potential characteristics of drivers from vehicle trajectories. Then, the potential characteristics and vehicle status were used to learn the trajectory prediction of autonomous vehicles at uncontrolled T-intersections. The introduction of contrast loss better learned the exclusive characteristics of the drivers from the vehicle trajectory and ensured the personalized and distinctive characteristics of the drivers; the residual network was added to the latent variable of feature extraction to improve the ability of feature extraction and prevent the gradient from disappearing. Experiments showed that the proposed method better separated the potential characteristics of drivers with different styles, received more exclusive characteristics of drivers from the trajectory, and improved the collision probability at uncontrolled intersections. However, the fuzzy driving trajectory was not successfully distinguished, and lane change and turning were not studied, which is the direction of our future research.


A recently developed contactless ultrasonic testing scheme is applied to define the optimal saw-cutting time for concrete pavement. The ultrasonic system is improved using wireless data transfer for field applications, and the signal processing and data analysis are proposed to evaluate the modulus of elasticity of early-age concrete. Numerical simulation of leaky Rayleigh wave in joint-half space including air and concrete is performed to demonstrate the proposed data analysis procedure. The hardware and algorithms developed for the ultrasonic system are experimentally validated with a comparison of saw-cutting procedures. In addition, conventional methods for the characterization of early-age concrete, including pin penetration and maturity methods, are applied. The results demonstrated that the developed wireless system presents identical results to the wired system, and the initiation time of leaky Rayleigh wave possibly represents 5% of raveling damage compared to the optimal saw cutting. Further data analysis implies that saw-cutting would be optimally performed at approximately 11.5 GPa elastic modulus of concrete obtained by the wireless and contactless ultrasonic system. Concrete is one of the most popular materials in civil engineering. Along with the increasing need for structural inspection, ultrasonic methods have been employed to evaluate the quality of materials in situ. Ultrasound is beneficial for characterizing the soundness of materials because the wave parameters such as velocity and amplitude are directly related to the mechanical properties of the materials . HoweverRecent advances in non-contact ultrasonic sensing technology have broadened the scope of nondestructive inspection of various engineering materials including concrete . Zhu et On top of inspection applications , the contactless ultrasonic system enables the provision of practical information on early-age material characteristics, such as the setting time. Conventional methods to determine a setting time are based on pin penetration tests, where a physical pressure load is applied to the surface of a concrete specimen. However, the methods recommend wet sieving to filter out coarse aggregate such as gravel, which is a significantly labor-intensive and time-consuming procedure. Choi et al., 2016) first demonstrated that the initiation of leaky Rayleigh waves is a useful tool to determine the final setting of early-age cementitious materials without physical contact with the specimens . Moreove first deIn the characterization of early-age concrete, the contactless ultra-sonic system showed great potential for field applications. Tran and Roesler 2020) applied the contactless ultrasonic system to the saw-cutting initiation time in concrete pavements and suggested the framework to predict the relationship between the saw-cut initiation time and final setting time . Saw cut20 applieIn this study, the optimal saw-cutting time is investigated using the concept of a contactless ultrasonic scheme. The uniqueness of this study over previous research lies in (1) the improvement of the sensor network of the contactless system using wireless data transfer for possible field applications, and (2) the development of a signal processing and data analysis scheme to determine the optimal saw-cutting time. Moreover, we optimized the critical testing parameters of the contactless ultrasonic system for application to saw-cutting time, such as the incident angle, measurement time intervals, and the number of sensor array elements. The theoretical background of leaky Rayleigh wave propagation in early-age concrete is supported using numerical simulations. Moreover, the developed hardware and signal processing algorithm were experimentally validated and compared to the conventional methods, such as the pin penetration and maturity methods.Rayleigh wave is one of the surface waves, propagating at the surface of a medium. The wave motion is elliptical rotation, which is caused by the surface guidance of body waves. The portions of longitudinal and shear waves generate the unique behavior of wave propagation. The relationship between material properties and Rayleigh wave velocity is given by’s ratio .At the joint-half space of air and concrete, small portions of the Rayleigh wave en0.ergy are leaked into the air . A contactless ultrasonic measurement system measures the leaky Rayleigh wave without a physical coupling procedure to the surface of a concrete element. In non-destructive evaluation, the measurement of leaky Rayleigh waves has the great benefit of non-invasive material characterization.Concrete is the composite material consisting of cement, and fine and coarse aggregate. The early-age concrete mixture is in a liquid state when water is added, and the hydration reaction makes the mixture harden in time. Conventionally, the degree of hardening at the initial stage is defined by the initial and final setting, generally representing the initiation of chemical adhesion and the physical stiffening of the mixture, respectively. The solid-state of concrete is considered as the time after the final setting. Leaky Rayleigh waves are measurable when the medium enables stress transfer, implying the propagation of both longitudinal and shear waves inside the medium. Therefore, the initiation of leaky Rayleigh waves represents the critical phase of shear resistance development in the medium, which is more relevant to the definition of the final setting.The stiffening scenario with respect to leaky Rayleigh wave propagation is numerically demonstrated using the finite element method. The commercially available software, COMSOL Multiphysics, was used to analyze the propagation of the leaky Rayleigh wave at the joint-half space. The Multiphysics model included the air and concrete, where the ultrasonic excitation and sensing locations were positioned in the air. The details of the numerical simulation are described in E = 0.1 GPa) shown in E = 1 GPa), Rayleigh waves propagate at the surface of concrete, and the corresponding leaky waves are observed in the air. In the hardened concrete case (E = 25 GPa) seen in The simulation results presenting wavefield snapshots are shown in Based on the results of numerical simulations, the direct relationship between the leaky Rayleigh wave velocity and elastic modulus of concrete was derived as shown in To measure the leaky Rayleigh wave responses from concrete pavements, a wireless and contactless ultrasonic system was developed. The system includes two printed circuit boards (PCBs) encapsulated by an aluminum frame. Each PCB is designed for sensing and data acquisition parts, respectively. The sensing board consists of 16 MEMS  ultrasonic sensor elements placed with 5 mm spacing. The data acquisition board is composed of several components, including a field-programmable gate array (FPGA), multiplexer, amplifier, and Wi-Fi module. The details of the wireless system are presented in The acoustic pressure is measured using the MEMS sensor array, and the collected analog signals are amplified 2000 times at the center frequency of 50 kHz. Then, the analog signals are converted to digital signals with a sampling rate of 2 MS/s and a dynamic resolution of 14 bits. The data acquisition process is repeated 16 times through each MEMS sensor element of the array, and the acquired data are wirelessly transferred to a personal computer. The stored data can be checked in real-time with programmed software, and measurement parameters such as the number of time-averaging can be set by an operator. The developed system is significantly improved in terms of the overall structure of the sensor network compared to the previously applied wired unit . Second,In addition to the hardware development, we propose a signal processing approach to computing the leaky Rayleigh wave velocity from measured signals. An overview of the proposed signal processing approach is illustrated in t-x) to frequency–wavenumber (f-k) domains, given byf-k domain signal set, and j is the imaginary number. Next, a region of interest in the f-k domain is highlighted by multiplying a filtering mask First, bandpass filtering is applied to the measured signal set f and k values at the maximum amplitude of Note that the filtering operation shown in Equation (3) maintains wave components with the Rayleigh wave velocity of typical concrete while suppressing other unwanted components including noise. Then, peak frequency  was derived from the simulation results shown in To characterize the state of early-age concrete, conventional methods such as maturity and pin penetration methods were also performed. The pin penetration method (ASTM C403) is a standard test method to determine the setting time of concrete mixtures . In thisDespite its wide acceptance as a means to determine the setting time of concrete, the pin penetration method has several limitations. First, performing the pin penetration test is highly labor-intensive. As the pin penetration method is not applicable directly to concrete with coarse aggregate, wet sieving needs to be carried out to extract the mortar fraction of the concrete mixture. Also, the test results can vary depending on the proficiency of the operator. To obtain reliable setting times, the operator needs to be trained for the test procedure. Moreover, the pin penetration test is typically carried out in a controlled laboratory environment that can be different from that of an actual construction site .Q the activation energy divided by the ideal gas constant, and The maturity method (ASTM C1074) is a standard practice to estimate concrete strength using measured temperature data . Two matLike the pin penetration test, the maturity method has its own limitations. First, the laboratory curing temperature may not represent that of the concrete element on the actual construction site. Also, the strength of concrete cannot be estimated directly from the temperature measurement data; a calibration curve needs to be constructed before actual strength estimation is carried out, which is time-consuming. The maturity method does not capture local strength variations within a concrete element.The developed wireless and contactless ultrasonic hardware system and proposed signal processing approach were validated through experiments. All the early-age concrete specimens used in this study were produced with the mixture design proportions summarized in Once the specimens were prepared, contactless ultrasonic tests and maturity tests were carried out. For the contactless ultrasonic measurement tests (both the wireless and wired systems), a contactless air-coupled transmitter  with a center frequency of 48 kHz generated ultrasonic Rayleigh waves in the tested specimens, and the corresponding leaky Rayleigh wave responses were measured using an array of MEMS microphones. Measured analog signals were digitized at a sampling rate of 2 MS/s using a data acquisition device . An incident angle was set to 5° for the air-coupled transmitter. To improve the signal-to-noise level, 50 times time averaging was applied. Leaky Rayleigh wave responses were automatically measured every minute and stored in laptop computers.Pin penetration tests were carried out following ASTM C403. More details about the test method can be found in . For theLeaky Rayleigh wave signals measured using the contactless ultrasonic measurement systems are presented in Despite the good agreement in the leaky Rayleigh wave initiation time and wave-front development, the wired and wireless measurement systems show differences in signal-to-noise level in the B-scan images . The sigThe results from the conventional methods are presented in The equivalent age of the mixture is shown in Raveling damages caused by a saw cut that is too early were observed on the top surface of the slab specimen as shown in The quantified saw-cut areas as a function of elapsed time are presented in Based on the proposed signal processing scheme described in DI values closer to zero represent the smaller extent of raveling damage. To evaluate the degree of raveling damage, a damage index ((1)The wireless and contactless ultrasonic system measures the initiation of leaky Rayleigh waves and their velocity increments during concrete hardening. The performance of the developed hardware was nearly identical to that of the wired system.(2)The proposed signal processing scheme provides the velocity of leaky Rayleigh waves and the corresponding elastic modulus of concrete, which is a conservative prediction compared to theoretical Rayleigh wave velocity.(3)Graphical analysis efficiently shows time-dependent raveling damage, and a damage index is calculated to assess the damage level with respect to the development of elastic modulus.(4)The initiation of leaky Rayleigh wave with a 5° incident angle was observed two hours later than the final setting time defined by the pin penetration test, which represents 5% of raveling damage compared to the optimal saw-cutting time.(5)The correlation between the damage index and elastic modulus implies that the saw cutting would be optimally performed when the elastic modulus of concrete reaches approximately 11.5 GPa.In this study, a wireless and contactless system was applied to evaluate the optimal saw-cutting time based on leaky Rayleigh wave measurements. The development of sensor networks and the proposed signal processing approach were numerically and experimentally validated. Based on the results, the following conclusions were drawn:


The method of using millimeter-wave radar sensors to detect human vital signs, namely respiration and heart rate, has received widespread attention in non-contact monitoring. These sensors are compact, lightweight, and able to sense and detect various scenarios. However, it still faces serious problems of noisy interference in hardware, which leads to a low signal-to-noise ratio (SNR). We used a frequency-modulated continuous wave (FMCW) radar sensor operating at 77 GHz in an office environment to extract the respiration and heart rate of a person accustomed to sitting in a chair. Indeed, the proposed signal processing includes novel impulse denoising operations and the spectral estimation decision method, which are unique in terms of noise reduction and accuracy improvement. In addition, the proposed method provides high-quality, repeatable respiration and heart rates with relative errors of 1.33% and 1.96% on average compared with the reference values measured by a reliable smart bracelet. Using the millimeter wave Doppler sensor  to non-cAt present, increasing the SNR level of the signal to improve the accuracy of the respiration and heart rate is a research focus of numerous researchers. Multiple channels of the MIMO radar were adopted to collect data, as covered in paper , and theThis paper demonstrates the feasibility of measuring human vital-sign signals using a 77 GHz FMCW sensor. We extracted the respiratory and heart rate using the phase information of the IF signal, a mixture of transmitted and received waves. Then, an algorithm called Iterative VMD Wavelet-Interval-Thresholding was proposed to reduce the noise in the measurement process. The experimental results show that the proposed algorithm can improve the SNR of respiration and heartbeat signals by 1.89 dB and 1.44 dB. We analyzed the principle of the FMCW radar detecting human vital-sign signals in our previous paper . The modAfter the IF signal is sampled by ADC, the data acquisition board exports the sampled data to form a two-dimensional matrix. The sampled data in each chirp constitute the matrix’s columns, and the matrix’s rows are composed of different frames. It should be noted that we set only one chirp in each frame. The sampled data are preprocessed mainly through static signal-clutter removal to eliminate the background interference in the environment for the position information of the measured target. The fast Fourier transform (FFT) is applied to the column vectors of the matrix successively to obtain the precise range window of the measured target, and then we further extract the phase of each frame in the range window. The vital signs of the target can be obtained from the phase information. To eliminate the influence of the DC components of the I and Q channels on phase extraction, the DC offset compensation operation is carried out to obtain the original phase. The phase difference operation plays a significant role in enhancing the heartbeat signal components. Then the iterative VMD Wavelet-Interval-Thresholding algorithm is proposed to filter the impulse noise generated by the differential process. In the vital signs detection stage, respiration and heart rates are extracted successively. Among them, the respiration rate is obtained by the FFT-CZT mixed spectrum estimation algorithm, while the heart rate is estimated by FFT-CZT and peak-seeking in the time domain method separately. Then we combine the confidence degree for thresholding judgment, and one of the results obtained by the two methods above is selected as the final result. The details of each step are elaborated in the sections as follows.The 1D-FFT in the distance dimension is performed on the two-dimensional matrix formed by sampling the IF signal to obtain the range slow-time matrix. Due to the complex background in the actual detection environment, the reflected static clutter signals will affect the selection of the target range bin and introduce some phase noise, which is not conducive to the extraction of the cardiopulmonary information. Therefore, the interference of static clutter should be removed. In the same test scenario, the clutter signals caused by other stationary objects basically remain unchanged with slow time, so subtracting the mean value of the slow-time dimension of each range window from the data in the slow-time dimension corresponding to each range bin can remove the background noise and retain the movement information of the measured target. Assuming that Because of the limitations of the manufacturing process of the hardware circuit in the receiver, the sensor used may have problems of insufficient mixer isolation or incomplete matching between the receiving antenna and the transmitting antenna. Therefore, a certain DC offset still exists after solving the static signal clutter caused by the fixed objects in the detection environment. DC offset will introduce more phase noise, resulting in the phase’s distortion extraction of the echo signal by the arctangent operation. The extracted phase can be expressed as:After DC offset compensation, we need to extract the phase information of the complex data. Using the arctangent algorithm to extract the phase will have a breakpoint problem, leading to phase drift exceeding ±π. Assuming that the real and imaginary parts of the complex data above are expressed as I(n) and Q(n), respectively, we apply the extended differentiate and cross multiply (DACM) algorithm  to extraIterative VMD Wavelet-Interval-Thresholding is mainly used to remove the impulse noise generated after the phase difference operation. This process refers to the EMD-CIIT described in paper . CompareThen, The algorithm can be divided into the following steps: first, initialize We set According to the above theoretical inference, VMD decomposes the original signal into the sum of multiple IMFs, where each IMF is a narrow-band amplitude and frequency modulation (AM-FM) signal. After comparing the energy and spectrum of each IMF with the original signal, we find that the first IMF basically does not contain useful signal components, so it is considered to contain noise components, and the remaining IMFs are regarded as signal components. The noise reduction process is shown in (1) VMD expansion: Perform the VMD decomposition of the original noisy signal y to obtain lgorithm , in whic2) Wavelet-thresholding: The first IMF obtained, which is called Select the appropriate wavelet basis function and determine the layers of the wavelet decomposition;Obtain a set of wavelet decomposition coefficients in the wavelet domain by wavelet decomposition of  Randomly alter: Considering the randomness of the noise contained, we chose to randomly alter the noise component (4) Signal reconstruction: Add the noise component (5) Interval-thresholding: Perform the VMD decomposition of jth unit of the ith IMF; Considering the general characteristics of the IMF obtained by VMD decomposition, we define the waveform between adjacent zero crossings as a unit of the IMF. These characteristics include that the difference between the number of extreme points and zero crossings in the whole sequence must be no more than one, and the envelope mean value in any time range must be zero. Then, the threshold detection is carried out for each unit. If the amplitude of the extreme value in one unit is greater than the set threshold, the unit is considered a valuable signal to be reserved. In comparison, if the amplitude is less than the threshold, the unit is considered a noise signal and set to zero. It can be expressed as:ρ = 2.01  for A = (6) Iterate: Iterate A-1 times steps (3)–(5) to obtain the corresponding noise reduction versions;(7) Results of processing: Add and average the noise reduction versions generated after each iteration. The respiration rate is obtained by extracting its spectral peak value through the FFT-CZT hybrid algorithm. The algorithm mainly refers to our previous work , and ref(1) Filter out the respiratory signal through a band-pass filter: Considering the general heart-rate range, we select an elliptical band-pass filter with a passband of 0.7–3.3 Hz. The finite transmission zeros in the stopband of the elliptic filter reduce the transition region, and then an extremely steep attenuation curve can be obtained. Thus, it has a very narrow transition bandwidth;(2) Filter out the respiratory harmonic interference through notch filter banks: The nonlinearity of the FMCW radar will lead to the generation of respiratory harmonics, in which the second to fourth harmonic components are within the range of humans’ heart rate, which will affect the extraction of the heart rate. Therefore, the notch filter banks designed in paper [in paper  are used(3) Heart rate judgment: Due to the intermodulation between the breathing and heartbeat signals and the characteristic of multiple scattering points of humans’ chests, there are still frequency components other than heartbeat frequency after removing breathing and its harmonic components. There are two extraction methods:The FFT-CZT hybrid algorithm mentioned abovePeek-seeking in the time domainIf The above two methods have their advantages and limitations. Combined with the confidence of the FFT method, the threshold decision scheme of heart-rate estimation is given: if the confidence of FFT exceeds the threshold and the difference between the results of the two methods above is less than the threshold, the FFT-CZT hybrid algorithm is used for heart-rate estimation; otherwise, the peak-seeking in the time-domain is adopted. Suppose that We chose the radar AWR1642 from Texas Instruments (TI) for the experiment. It is an integrated FMCW millimeter-wave sensor operating at 76–81 GHz. It integrates a high-performance DSP subsystem and ARM processor for post-processing, which can realize programming and signal processing on the chip. However, in order to further observe the experimental results and analyze the data, we hope to obtain the original data for the algorithm verification. Therefore, we used the AWR1642 evaluation board to test the target and the DCA1000 acquisition board to collect the raw radar data. Then, we imported the collected data into MATLAB for signal analysis. The parameter configuration of the experiment is shown in A total of 11 subjects were selected for this experiment and divided into two groups. All measurement processes lasted for 25.6 s, and then the stable data were intercepted from them with a length of 12.8 s for subsequent processing.As shown in Scenario #1 in the first group: the subject sat on a chair at a distance of 0.8 m in front of the radar and kept the respiration and heartbeat signs stable before the test with the respiratory and heart rate at 24 and 75 times per minute, respectively, obtained by the smart bracelet;Scenario #2 in the second group: the subject sat still on the chair at a distance of 1.3 m in front of the radar with a deliberately increased respiratory rate at 35 times per minute;Scenario #3 in the second group: the subject sat on the chair at a distance of 1 m in front of the radar with some routine behavioral patterns and performed aerobic exercise before the measurements to obtain a deliberately increased respiratory and heart rate at 31 and 103 times per minute.The breathing and heartbeat signals extracted from the denoised phase difference signals by VMD decomposition operation are shown in (d) and (e) in The effect of the Iterative VMD Wavelet-Interval-Thresholding is analyzed below. We redefined the SNR for respiratory and heartbeat signals in the frequency domain for quantitative analysis. Taking the SNR of the respiratory signal as an example, assuming that the spectral line position corresponding to the peak of its spectrum is in paper , with anBecause the amplitude of the chest cavity vibration caused by respiration is relatively large, it is not easy to be disturbed by noise. Thus, we only analyzed the detection accuracy of the heartbeat rate here. This paper introduces a high-precision vital-signs detection algorithm based on millimeter-wave radar sensors. In practical situations, however, the monitored person does not stay completely still. So, next we discuss the applicability of the proposed algorithm in the non-stationary cases. It is mainly divided into two situations: small-scale micro-movements and large-scale and fast random body movement (RBM). The former includes coughing, sneezing, and arm swinging, while the latter refers to behaviors such as walking around. The difference between the two is whether there is a significant change in the displacement of the chest wall over time. Meanwhile, deep learning-based technology has also been studied and applied to the detection of vital signs’ information in order to adapt to complex and changeable scenarios. At present, the more mature application is mainly to distinguish whether the target’s respiratory or heart rate is abnormal with high precision. Yin. W et al. proposed using convolutional neural networks independent of each other to learn the signals collected by ECG and the UWB radar and then performed the comprehensive analysis in a cascaded way to achieve mutual complement of biological and physical characteristics of the heartbeat . Li. J pIn this paper, we discussed a systematic method for respiration and heartbeat monitoring based on a FMCW millimeter wave radar. Two scenarios with 11 human subjects at different distances were performed and studied. With the 25.6 s short-time monitoring in each experiment, a window of 12.8 s was randomly selected for signal processing and vital signs’ information extraction. The results show that the average accuracy of respiratory and heart rate estimation is not less than 98% using the FMCW sensor and the proposed processing algorithm in the test scenario where subjects are stationary. The possible noise contained in the test process and the corresponding noise removal measures were discussed. The experiments showed that the SNR of the vital signals can be increased by using the Iterative VMD Wavelet-Interval-Thresholding algorithm. In addition, the decision method based on spectrum refinement and peek-seeking in the time domain also helps to improve the accuracy of rate estimation. We envision that this approach could be applied to various practical situations, such as home health-monitoring for the elderly, medical assistance for burn victims, and driver monitoring systems.