Freestyle skiing U-shaped field is a snow sport that uses double boards to perform a series of action skills in a U-shaped pool, which requires very high skills for athletes. In this era of deep learning, in order to develop a more scientific training method, this paper combines multitarget tracking algorithm and deep learning to conduct research in freestyle skiing U-shaped venue skills motion capture. Therefore, this paper combines the convolutional neural network and multitarget tracking algorithm in deep learning to study the human action recognition technology, and then uses the LSTM module to study the freestyle skiing U-shaped venue skills. Finally, this paper designs the training method of the action recognition algorithm of the freestyle U-shaped skiing skills multitarget tracking algorithm based on deep learning. This paper also designs multitarget tracking dataset experiments and model updating experiments. Based on the data of experimental analysis, the training method designed in this paper is optimized, and finally compared with the traditional training method. Compared with the traditional freestyle U-shaped skiing skills training method, the experimental results show that the training method of the freestyle U-shaped skiing skills multitarget tracking algorithm action recognition algorithm is based on deep learning designed in this paper and this improves the skill score by 14.48%. Most professional students are very satisfied with the training method designed in this paper. Action recognition is a popular research topic in computer vision, which refers to the automatic identification and classification of human actions in videos by computers. It is affected by a series of conditions such as body shape, camera angle, moving objects, and background. Action recognition is essentially a video classification task, where many techniques and methods are used from the fields of image recognition and text retrieval. Research on action cognition is valuable for development in video surveillance, human–computer interaction, and other fields.With the continuous improvement of the modern competitive level and the increasingly intensified competition, the development level of special physical characteristics, which is one of the main components of competitive ability, is becoming more and more prominent in modern competitive sports. China's freestyle U-shaped skiing skills have achieved good results in international competitions, and the competitiveness of the players is also rapidly improving. It is expected that the Winter Olympics will become a favorable event for China to win gold medals. Research basic theories and methods to develop the physical strength of freestyle skiing U-shaped field skill athletes, seek the best physical training theory and practice mode, and make the athletes' physical training process more scientific, systematic, and optimized.In recent years, techniques including machine learning and deep learning have gradually emerged, and deep neural networks are already a highly topical subject of investigation and have made progress in various fields such as computer vision. Being an overarching subset of deep neural networks, convolutional neural networks are commonly employed in areas such as videographic processing and phonological identification. Its emergence also offers a fresh response to the issue of identifying human movements that we study. Therefore, this paper combines the application of deep learning and multitarget tracking algorithm in freestyle U-skiing skills, which can allow athletes to train scientifically and effectively.The innovation of this paper lies in the use of convolutional neural networks in deep learning, and then through the research on human action recognition technology. This paper combines the LSTM module and the freestyle skiing U-shaped field skills, and finally constructs the application of the multitarget tracking action recognition algorithm based on deep learning in the freestyle skiing U-shaped field skills. This paper also designs multitarget tracking dataset experiments and model update experiments to optimize the algorithm designed in this paper.Recent developments in machine learning, particularly deep learning, have facilitated the recognition, classification, and quantification of medical image models. Deep learning has been rapidly gaining ground as a frontier, boosting the capability of all kinds of pharmaceutical operations. Shen D described the fundamentals of deep learning approach and reflected on previous achievements in registration of images, detection of anatomical and cellular structures, division of organizations, computer-aided disease prevention, and remediation . AlthougDeep learning  is a treWith the invention of machine learning models such as Support Vector Machines  and DeepA typical feature of deep learning compared to shallow learning is that training requires large training datasets. The collection of datasets is a large workload, usually using data expansion methods , 13. ForBecause features have a great impact on classification results, extracting important features representing objects is an important part of recognition. Traditional feature extraction is manually extracted, and features are mainly selected based on experience and professional knowledge. Deep learning is to automatically extract features through autonomous learning, effectively strengthening the firmness of human action cognition. Now, there are many deep learning models such as Scalable Networks (DBN) and Collapsible Neural Networks (CNN).Deep learning extracts features through multiple layers, and the effect of multilayer feature extraction is better than that of single-layer or shallow feature extraction. Each level extracts different features according to the scale of the level. The low level is handed over to the high level after extracting the feature map. The high-level extracts the feature map from the feature map provided by the low-level again and passes it to the other party to extract the most effective recognition and features that improve the recognition accuracy. Convolutional neural networks have been mainly applied to tasks such as identity separation, monitoring, and discrimination. Schematically, in a convolutional neural network, a convolutional level and a subsampling level could constitute a characteristic selector . In the Convolution is a classic concept in image processing. Calculate the local area of the image by using a two-dimensional matrix and obtain the corresponding convolution value. The convolution kernel, called an image filter, is a square matrix with weights. Filters can be applied to image areas. It usually uses multiple filters simultaneously . For exaIn a convolutional layer, the input data are convolved with several filters. The result of applying a filter to an image is called a feature map, and the number of feature maps is the same as the number of filters. If the preapplicator layer is a convolutional layer, the filter is applied to the feature map. This is the same as inputting feature maps and outputting other feature maps. Intuitively, when the weights are spread over the whole image, the features do not depend on the location, and multiple filters can detect different features separately.Image preprocessing . When inDetection of moving objects . Moving Feature extraction. After the human domain detection is completed, the next step is feature extraction. The purpose of feature extraction is to extract a part of feature information from a single or a series of image data, which is characterized by human behavior, and the features are extracted from previous experience. General static feature extraction methods include: Histogram of Oriented Gradients (HOG), Local Binary Patterns (LBP), and Haar-like feature methods. Common dynamic extraction methods include spatiotemporal interest point algorithm (STIP), histogram of directional optical flow (HOF), and moving boundary feature (MBH).Analysis and identification of functions. After the feature extraction is completed, the obtained feature is a high-dimensional feature vector. In order to realize the classification of human actions, the feature vector needs to be analyzed. Commonly used feature analysis methods include human body model-based feature understanding and statistical model-based feature understanding. Among them, the human body model-based feature understanding method obtains results by comparing the obtained human action performance with existing action models. The feature understanding method based on statistical model establishes the feature data template of each action, compares the acquired feature data with the action template, and calculates the similarity to determine the action type. According to the above steps, the classification results of human actions can be completely obtained.The basic flow of the human action recognition algorithm is shown in The SSD model  is an enSSD methods mainly use a series of small convolutional filters to predict class scores and position offsets for a fixed set of default bounding boxes on feature maps. . Generate predictions at various scales from feature maps at different scales and correctly predict various aspect ratios. The detection speed of SSD is almost as fast as YOLO, and it is as accurate as the detection method using region proposal technology and ROI pooling. The principle of SSD target detection is shown in The SSD model approach is also based on a feedforward neural network that integrates all computation, feature extraction, detection, and normalization processes into one network. The network structure of the SSD model is shown in As shown in the figure, when the SSD network is trained for detection, six feature layers are extracted as the final detection features, and the scale of these feature maps is gradually reduced, so that multiscale features can be extracted and the final detection results are more accurate. Applying a set of convolutional filters to each feature layer produces a corresponding fixed-size prediction.γ, θ, and ω denote the input gate, the forgetting gate and the output gate, respectively, and the subscript h denotes the input of the LSTM module. Without loss of generality, it is assumed here that an LSTM module contains only one internal state Z. The LSTM network also processes a sequence a of length S, the forward operation starts at s = 1, and the backward operation starts at s = S.Like ordinary recurrent neural networks, LSTM networks are trained using the BPTT method , and theForgotten gate:Input gate:Internal state:Output gate:Status output:Similarly, it is not difficult to derive the backward processing process of LSTM recurrent neural network according to the BPTT method. Here, the definitions of the two quantities are given as follows: the backward processing process of the LSTM network on the state output, output gate, internal state, forget gate, and input gate, respectively, is given.Status output:Output gate:Internal state:Forgotten gate:Input gate:Sports model:In the U-shaped field competition of freestyle skiing , on the The stage of the U-shaped field is divided into two stages. One is the U-shaped ski field and the stage where the action is done in the air. The stage where the action is done in the air mostly reflects the athlete's somersault. The sliding stage of the swivel-capable field on the U-shaped ski resort covers the athlete's take-off and landing control ability. For take-off, speed and take-off angle are the two most direct physical parameters, because the athlete obtains the ideal height to complete high-quality movement, so choosing the most reasonable take-off angle as the premise of a specific speed is the core issue. The secret to a successful landing is to control the body with a certain initial speed as the premise, and at the same time to achieve high-quality movement, enter the U-shaped field again and land successfully. This stage is the way to maintain speed and balance. On the U-shaped ski field, athletes can reasonably use the sliding speed of the accelerated slope to perform somersaults, twists, swings and various action switching functions. The most obvious performance feature of this technique is the difficulty of the athlete's movement, the completion of the movement flight height, and the aerial movement posture.The freestyle U-shaped track skier obtains the height of entering the groove through the acceleration sliding of the landslide, from the initial acceleration of the landslide until entering the U-shaped groove, that is, the kinetic energy is converted into the gravitational potential energy. During the take-off process in the slot, the kinetic energy is converted into gravitational potential energy, while the gravitational potential energy decreases and the kinetic energy increases during the falling process. The sliding speed determines the slotting angle of different action techniques. The perfect application of the speed lies in the precise control of the athlete in the sliding, and its speed characteristic is embodied as “steady.”In the 5–6 take-off movements of freestyle U-shaped skiers in the groove, different movement techniques have different out-of-the-groove angles. The slotting angle determines the landing area of the action, and the correctness of the landing zone directly affects the success or failure of a single action, so the slotting angle is characterized as “accurate.”If freestyle U-shaped skiers do not have good sliding ability, they will not have high-quality take-off speed, which will affect the flying height and the completion of high-quality difficult movements. Therefore, the flight altitude characteristic of this project is expressed as “high.”In the complete set of movements of the freestyle U-shaped field skiing project, there must be a basic movement in the rules, and then the smoothness of the connection between the movements, and the application of the difficulty of the movements. When other factors are satisfied, the use of action difficulty determines the final success or failure of the game, and its action difficulty characteristics are embodied as “beautiful” and “difficult.”This paper uses OTB100 for the assessment of the dataset. OTB100 consists of a grand total of one hundred recorded tracking video clips, which are integrated from various datasets. The evaluation mark uses one-pass valuation (OPE) profile as an assessment indicator. By definition, the OPE profile is the outcome of just a single trace, that is, the current tracking chain is ended when the tracking fails, and no reinitialization is necessary. The OPE profile has two types of assessment, as shown in The selection of traditional correlation filter-based individual objective pursuit algorithms is presented in this chapter and compared with deep learning-based individual objective pursuit algorithms, as shown in MOT2015 is a multiobject tracking dataset containing 22 video sequences. The collection devices include both stationary and mobile camcorders, and the acquisition views consist of elevated, overhead, and downward views, as well as varying meteorological situations and varying illumination requirements. The video sequence has 11286 images, of which the control collection consists of 5500 images, 500 objects, and 39,905 frames. The training session consisted of 5500 images with 721 tracking targets and 61440 tracking target frames. The dataset offers inspection frames gained from fused passage characteristics (ACF). The specific distribution of the data is shown in Tables Among them, the detector of MOT2016 is replaced by a deformable component model (DPM), which has a higher target localization accuracy than the fusion channel feature detector used in MOT2015. So in general, the same algorithm, the indicators on MOT2016 will be slightly higher than the results on MOT2015. The annotation files of MOT2015 and MOT2016 are also different. MOT2015 provides the 3D information of the target, but only labels people. MOT2016 not only abandons 3D information but also labels cars, bicycles, motorcycles, etc. The specific distribution of the data is shown in Tables A simple comparison shows that STAM has fewer missed detections (FN) and label switching (IDSw), but more false detections (FP) when the overall effect (MOTA) is not much different. STAM and AMIR, respectively, represent the idea of two different multitarget tracking algorithms. Among them, STAM can retrieve the lost target by constructing a matching convolutional neural network, that is, less missed detection. However, due to the algorithm itself, the missed detections retrieved may not be correct, so more false detections are brought about. AMIR can eliminate a large number of false detections by building an association model based on recurrent neural network, but there is no way to make up for lost targets , 26.The two most important parts of the tracking process are the tracking strategy for the model. After initializing the parameters of the network model by a certain method, the input of the first frame can be accepted to predict and locate the target object in the subsequent frames. However, during the tracking process, the target object is constantly changing, and will experience deformation, occlusion, blurring, etc., resulting in a larger and larger difference between the target object and the initial state. The ability of the initial model to fit the features of the target object decreases. If the model is not updated according to the latest state sampling of the target object, the target will be transferred or lost. The subject has been tested, and the experimental results of the test model fixed and online update in the OTB2015 data set are shown in It can be seen from the experimental results of the graph that the accuracy and success rate of the algorithm are higher than those when the model parameters are fixed when the model is updated.Through the test of the data set, the action recognition algorithm of the freestyle U-shaped skiing skill multitarget tracking algorithm based on deep learning designed in this paper is optimized . In ordeAs can be seen from the figure, the skill score of the multitarget tracking algorithm for freestyle U-shaped skiing skills based on deep learning designed in this paper after the training of the action recognition algorithm reached 92.96 points. The freestyle U-shaped skiing skills based on traditional training methods scored only 81.2 points. This shows that the training method of the multitarget tracking algorithm for freestyle U-shaped skiing skills based on deep learning designed in this paper improves the skill score by 14.48% compared with the traditional method. Moreover, 27 people were very satisfied with the training method of the multitarget tracking algorithm action recognition algorithm for freestyle U-shaped skiing skills based on deep learning designed in this paper, and 13 people were quite satisfied with this training method. This shows that the training method of the multitarget tracking algorithm for freestyle U-shaped skiing skills based on deep learning designed in this paper has a great improvement compared with the traditional training method and is very popular among professional students.This paper mainly studies the application of skill motion capture technology combined with multitarget tracking algorithm in the U-shaped field of freestyle skiing under the background of deep learning. Therefore, this paper combines the research of convolutional neural network in deep learning with multitarget tracking algorithm, and then selects the LSTM module to study the U-shaped field skills of freestyle skiing in the context of human action recognition technology. Finally, this paper designs a training method for the action recognition algorithm of freestyle U-shaped skiing skills multitarget tracking algorithm based on deep learning. Then this paper designs a multitarget tracking data set experiment, analyzes the data set, and improves the algorithm based on the results of the data set analysis. Then this paper updates the model locally through model update experiments. Finally, this paper uses experiments to verify the role of the training method of the deep learning-based freestyle U-shaped skiing skills multitarget tracking algorithm action recognition algorithm in professional students.


This study proposes an innovative method for structural health monitoring of simply supported PCI girder bridges based on dynamic strain and edge computing. Field static and dynamic load tests were conducted on a bridge consisting of a span with newly replaced PCI girders and numerous spans with old PCI girders. Both the static and dynamic test results showed that the flexural rigidity of the old PCI girders decreased significantly due to deterioration. To improve the efficiency of on-site monitoring data transmission and data analysis, this study developed a smart dynamic strain gauge node with the function of edge computing. Continuous data with a sampling frequency of 100 Hz were computed at the sensor node. Among the computed results, only the maximum dynamic strain data caused by the passage of the heaviest vehicle within 1 min were transmitted. The on-site monitoring results indicated that under routine traffic conditions, the dynamic strain response of the new PCI girder was smaller than that of the deteriorated PCI girder. When the monitored dynamic strain response has a tendency to magnify, attention should be paid to the potential prestress loss or other deterioration behaviors of the bridge. The development of structural health monitoring (SHM) technology for bridges is a major goal in civil engineering. The SHM of a bridge involves using different types of sensors, such as accelerometers, inclinometers, displacement sensors, strain gauges, load cells, and thermometers, for monitoring the response of the bridge to external stimuli ,2. AnalyAccording to the sampling frequency used, SHM can be categorized as static monitoring and dynamic monitoring. In general, in static monitoring, the sampling frequency is relatively low and data acquisition is performed at a specific time interval, such as every 1 s, 1 min, 10 min, or 1 h. The sampling frequency in dynamic monitoring is considerably higher than that in static monitoring. For bridge structures with natural frequencies of vibration between 0.2 and 20 Hz, a sampling frequency of 100 Hz  is sufficient.SHM technologies for bridges have been developed for more than 30 years. Unfortunately, the influences of environmental temperature changes on the physical behavior of bridges make it difficult to use common indicators such as bridge vibration frequency, static displacement, and static strain for examining the structural health of bridges ,13,14. TThe structural strain of bridges is sensitive to external forces; thus, strain monitoring may be a suitable approach for the SHM of bridges. However, the traditional static strain monitoring method is severely affected by environmental temperature changes. In general, a strain gauge measures the apparent strain, which includes the strain caused by temperature changes and the mechanical strain. Theoretically, the mechanical strain required for structural analysis can be obtained if the temperature-induced strain and apparent strain are known. However, because current commonly used strain gauges, such as resistance strain gauges, vibrating-wire strain gauges, and optical fiber strain gauges are affected by long-time temperature changes, the measured strain exhibits a drift phenomenon, which makes signal analysis extremely difficult ,22,23,24In recent years, some studies have used dynamic strain devices to measure the strain responses caused by vehicles passing through bridges ,26,27. SThis study performed field static and dynamic load tests on a bridge consisting of a span with newly replaced PCI girders and numerous spans with old PCI girders to demonstrate the vehicle-induced strains can be used to evaluate the deterioration of the old PCI girders. Subsequently, the dynamic strain gauges were deployed to monitor the relative strain change caused by the dynamic load of the vehicles. In general, a vehicle requires only a few seconds to pass through a prestressed concrete bridge. The temperature can be considered to be constant during this time. Therefore, the relative strain change caused by the passage of a vehicle is only affected by the weight of the vehicle and the flexural rigidity of the PCI girder. Thus, the dynamic strain examined in this study is unaffected by temperature. Since the flexural rigidity of a PCI girder decreases due to deterioration including concrete cracking, prestress loss, etc., the relative strain change exerted by a vehicle passing over the PCI girder can serve as an indicator of the deterioration of the PCI girder.Dynamic strain measurement requires the use of high-cost data acquisition equipment, and a considerable amount of data must be recorded and transmitted for dynamic strain monitoring. Therefore, on-site dynamic strain monitoring systems have a high workload, making them power-intensive and prone to system instability. Consequently, the on-site application of these systems is difficult. To overcome the problem caused by the need to transmit considerable data generated through dynamic measurement, this study adopted the edge computing technology, which is used in Internet of Things (IoT) frameworks. In edge computing, the data computation and analysis processes for an application are migrated from the central node in the network (cloud server) to edge nodes (the sensor nodes); thus, the sensor nodes only have to transmit key computed data. The main strength of edge computing is that it reduces the load on the network and servers so that each sensor can be directly equipped with a narrowband IoT (NB-IoT) wireless communication module which is a Low Power Wide Area Network (LPWAN) radio technology. This data monitoring process is highly effective and independent and does not require an accompanying host or gateway on site. The combination of the dynamic strain measurement and edge computing can form a simple, straightforward, and effective method for evaluating the degradation of simply supported PCI girders.The flexural rigidity of the old PCI girders decreased significantly due to deterioration.The vehicle-induced strains of old and deteriorated PCI girders were noticeably larger than those of the newly replaced PCI girders.This study developed a smart dynamic strain gauge node with edge computing functions to improve the monitoring system efficiency.The vehicle-induced strain responses were stable under routine traffic conditions. When the monitored dynamic strain response tends to amplify, attention should be paid to the potential prestress loss or other deterioration behaviors of the bridge.The important findings of this paper are as follows:This paper first presents the results of static and dynamic tests performed on a PCI girder bridge. Secondly, the development of a smart dynamic strain gauge node with edge computing functions is introduced, followed by the field application of the smart dynamic strain gauge node. Finally, based on the results obtained in this study, discussions and conclusions are given.The field static and dynamic tests were carried out on a bridge located near the west coast of Taiwan, which was completed in October 1995. In the static loading test, two 24-ton vehicles were placed in parallel on the bridge deck at the mid-span of the simply supported PCI girders, as shown in The static vehicle test was performed by slowly moving the first vehicle to a designated location at the midpoint of the span, and then moving the second vehicle. These two vehicles were placed in the specific locations and lasted for at least 100 s. h) between the bottom and upper strain gauges, the measured strains can be used to determine the curvature (κ) at the cross-section where the strain gauges are installed as follows:Knowing the distance (EI) as follows:The curvature (κ) can also be related to the moment (M) acting on the cross-section and the flexural rigidity .Since the vehicle load applied to each span was the same in the static test, the moment (M) acting on the investigated cross-section of the simply supported PCI girder can be regarded as a constant. Thus, In the dynamic test, a 25-ton vehicle passed over each span at a speed of 25 km/h, as displayed in As illustrated in The resistance value in a resistance strain gauge changes with the deformation of the measured object. Due to the small change in the resistance (approximately 0.1%), the resistance was converted into a signal with a small relative voltage difference by using a Wheatstone bridge circuit. When the internal resistance ratio (R1/R2) is equal to R3/R4, a fixed excitation voltage between Ve+ and Ve− is applied to the aforementioned bridge. The voltage difference between Signal+ and Signal− at this moment is 0. If R1 is the resistance of the measuring device , R1 varies with the deformation of a measured object. At this moment, according to the laws of resistance, the resistance is directly proportional to the length and inversely proportional to the cross-sectional area (Ω ∝ L/A) of the strain gauge. When a linear elastic material is compressed, gauge length decreases and cross-sectional area increases. Thus, R1 decreases, and the balance of the bridge is broken. The voltage of Signal+ is higher than that of Signal−. A positive voltage difference exists between Signal+ and Signal−, and the magnitude of this difference is linearly proportional to the deformation of the measured object. Therefore, the strain of a measured object can be determined from the magnitude of the output voltage difference between Signal+ and Signal−.EX) of the bridge circuit and the reference voltage (VREF) of the ADC are designed with rail-to-rail. The VEX and VREF voltages are supplied by a voltage reference IC, which can reduce power consumption and self-heating effects simultaneously. The novel design greatly reduces the wire interference and can reduce VEX and VREF to 2.5V. Calculated by a general 120 ohm bridge circuit, the supply current only needs 20.8 mA. The smart node can perform the analog-to-digital conversion of the strain gauge signal. All the output signals are digital signals; thus, wireless signal transmission can be achieved using the IoT technology. The main advantage of using smart nodes is that the signal transmission is free from the interference of ambient noise, which ensures high signal quality. The sampling resolution and sampling frequency of the smart nodes used in this study were 16 bits and 100 Hz, respectively.Due to the impedance of the wire, the voltage signal is susceptible to loss during transmission in the wires. The longer the transmission line, the higher is the resistance of a wire. Therefore, shorter wire lengths are more suitable for the strain gauge. Consequently, a smart node can be placed adjacent to a strain gauge. This smart node contains a data acquisition unit, and a microcontroller unit (MCU). Monitoring the dynamic strain of a bridge enables the determination of the bridge response caused by the passage of various types of vehicles on the bridge and the mechanical behavior of the bridge, so as to carry out effective safety management. However, continuous dynamic strain monitoring has rarely been performed in practice. In addition to the relatively high cost of dynamic measurement instruments, considerable data must be recorded in dynamic strain monitoring, which results in a high load on the on-site monitoring system, which can lead to high power consumption and system instability; thus, conducting continuous dynamic strain monitoring is difficult.This study used the technology of edge computing to solve the problem of large amounts of data transmission in existing dynamic strain monitoring systems. Edge computing is based on a distributed computing architecture in which the data computation task of the cloud server is shifted to edge nodes (sensor nodes). The main strength of edge computing is that it reduces the load on the network and servers so that decisions can be made within the edge network, and the latency is reduced. The main advantage of edge computing is that it reduces the load on the network and server. A distributed sensor is equipped with an MCU that contains central processing units, memory, a timer, and various input–output interfaces. It can be placed inside a device; however, the storage capacity of its memory is low, which limits the freedom of MCU firmware development and increases the difficulty of the temporary storage and processing of dynamic data. The MCU used in the study is Texas Instrument MSP430. The MCU’s clock is synchronized with the Network Time Protocol (NTP) server every hour.The main concept of edge computing for the SHM of a bridge is explained in the following text. Vehicles passing over a bridge cause the bridge to exhibit transient strain response; therefore, dynamic strain gauges can be installed at the midpoints of a bridge span to monitor the maximum dynamic response caused by the passage of vehicles, which serves as an indicator for the SHM of the bridge. Two methods exist for outputting the data of a dynamic strain gauge. The first method is the traditional method that involves transmitting all the recorded strain data to the server, which then interprets the data. The second method involves processing and interpreting the data at the sensor nodes through edge computing and then transmitting only key data  to the server. The difference between the aforementioned two data output and analysis methods is illustrated in The second output method only requires the outputting of key data within a certain time range. For example, for the dynamic strain data depicted in The basic algorithm of edge computing was to find the local maximum and minimum values of the vehicle-induced strain data by adopting the sliding window and moving average methods. The vehicle-induced strain can then be obtained by subtracting the local maximum value from the local minimum value. The comparison process of each vehicle-induced strain within 1 min can find the relative maximum strain per minute, which is the so-called key data in this study.Since the amount of dynamic strain data to be transmitted is considerably reduced through edge computing, an LPWAN NB-IoT communication module can be used for the wireless transmission of data by the monitoring system. In the dynamic strain monitoring system used in this study, a dynamic analog-to-digital converter, an ultra-low-power microcontroller, edge computing firmware, backup memory, and communication technology are combined and integrated to create an ultra-low-power dynamic sensor network system. The excitation voltage of the bridge circuit is 2.5 V and the equivalent resistance of the bridge is 120 ohms, so the current during continuous measurement is 20.8 mA, and the power consumption is 52 mW. The average current consumption of NBIoT transmission every 10 min is 0.83 mA, which leads to an average power consumption of 2.74 mW when the supply voltage of NBIoT is 3.3 V. The power consumption of the microcontroller and the signal processing circuit is about 1.3 mW and 1.4 mW, respectively. Therefore, when the sensing nodes of this system function in combination with its communication nodes, the power consumption of each sensing node is less than 60 mW.To verify the functionality of the smart nodes installed on site for edge computing, two smart nodes designed with wireless dynamic strain gauges were simultaneously installed in a prestressed box girder bridge, as displayed in Based on the dynamic test results and the edge computing technology presented in the previous sections, a new SHM method for the PCI girder bridge is proposed. The time-domain dynamic strain response was used to determine the relative strain change caused by the passage of a vehicle in a few seconds, thereby ignoring the temperature effect. This makes the signal analysis of the proposed new SHM method relatively simple, straightforward, and effective.The bridge mentioned in To obtain the dynamic strain response of the PCI girders, a 38.46-ton dump truck was used to conduct a moving vehicle test. The dump truck passed the fifth and seventh spans at 14:02 on 13 August 2020. This section discusses the major achievements, limitations and challenges based on the research results presented in this paper.First, the major achievements of this study include verifying that the flexural rigidity of deteriorated PCI girders can be evaluated by performing constant-weight moving vehicle test, proposing that the vehicle-induced dynamic strain responses can serve as an indicator for assessing the degree of bridge deterioration, and developing a smart node with edge computing functions to avoid transmission of massive data generated through dynamic measurement. It is worth mentioning that the dynamic strain response caused by the passage of a vehicle needs just a few seconds, so that the temperature effect can be ignored. This makes the signal analysis of the proposed new SHM method relatively simple, straightforward, and effective.However, the bridges investigated in this paper are PCI-girder bridges. Further studies are needed to determine whether the proposed SHM method is still applicable to other types of bridges, such as prestressed concrete box-girders.Although the monitoring method proposed in this study can improve the measurement benefit of vehicle-induced dynamic strain in terms of processing data volume, the biggest challenge is the development of firmware for edge computing at smart nodes because the processing clock of the microcontroller is only 1 MHz and RAM is only 2 kBytes. In addition, the question of how to apply the measured vehicle-induced dynamic strain to bridge safety management is an important and difficult challenge.In this study, static and dynamic tests were conducted on site on a PCI girder bridge. Because of severe deterioration of the seventh northward span, the PCI girder of this span was replaced by new PCI girders, which could serve as the control group in the vehicle loading test. Moreover, this study developed a smart dynamic strain gauge node with edge computing functions. This node only transmitted the relative maximum dynamic strain data per minute, which considerably reduced the original 6000 output data entries per minute to only one output data entry per minute; thus, this node can be paired with an NB-IoT wireless communication module to develop a high-performance dynamic strain sensor capable of independent operation.The results of the static test showed that the EI/M ratios of the deteriorated spans relative to the newly replaced girder of the seventh span range between 0.693 and 0.730. The static test of the bridge did find that the flexural rigidity of the old PCI girders was significantly reduced.The results of the dynamic test indicated that the relative maximum dynamic strain of a newly installed PCI girder was significantly smaller than that of old and deteriorated PCI girders. Thus, the flexural rigidity of the old and deteriorated PCI girder was significantly smaller than that of the newly installed PCI girder.Based on the dynamic test results and the edge computing technology presented in the study, a new SHM method for the PCI girder bridge is proposed. The time-domain dynamic strain response was used to determine the relative strain change caused by the passage of a vehicle in a few seconds, so that the temperature effect can be ignored. This makes the signal analysis of the proposed new SHM method relatively simple, straightforward, and effective.The on-site monitoring results showed that under routine traffic conditions, the dynamic strain response of the new PCI girder was smaller than that of the deteriorated PCI girder. The dynamic strain responses of the two spans were stable under routine traffic conditions. This indicates that no significant structural deterioration occurred during the monitoring period. When the monitored dynamic strain response has a tendency to magnify, attention should be paid to the potential prestress loss or other deterioration behaviors of the bridge.The following conclusions are drawn from the results of this study:This paper introduces the application of smart dynamic strain gauge nodes with edge computing functions in bridge dynamic strain monitoring, which solves the problem of massive data transmission and makes the field application of dynamic strain monitoring more feasible. Subsequent further research can include monitoring and tracking the neutral axis position of bridges, applications in bridge weight in motion, and the combination of some computational intelligence techniques such as model updating using evolutionary computation or data-driven machine learning tools.


The face has a large amount of information that is useful for humans in social communication. Recently, non-invasive methods have been used to investigate human brain activity related to perception and cognition processes. Electroencephalography (EEG) and magnetoencephalography (MEG) have excellent temporal resolution and reasonably good spatial resolution. Therefore, they are useful to investigate time sequences of human brain activity related to the face perception process. In this review, we introduce our previous EEG and MEG studies of human face perception that demonstrated the following characteristics of face perception processing: (1) Event-related components in the temporal area related to the activity in the inferior temporal (IT) area, corresponding to the fusiform face area (FFA), are evoked approximately 180 msec after the presentation of a face. The activity in the IT area plays an important role in the detection processing of a face, and the contours of a face affect the activity in the IT areas. (2) Event-related components in the temporal area related to the superior temporal sulcus (STS) activity are larger when eyes are averted than when directly looking into the eyes. (3) The direction of features of a face affects the face perception processing in the right hemisphere. On the other hand, the matching of the direction between the contours and features of a face affects the processing in the left hemisphere. (4) Random dots blinking (RDB), which uses temporal changes in patterns of many small dots to present stimuli without a change in luminance during the presentation of a face, is a useful visual stimulus method to investigate the brain activity related to face perception processing in the IT area using EEG and MEG. In this review, we discuss the following characteristics of face perception processing based on our previous electroencephalography (EEG) and magnetoencephalography (MEG) studies:–Event-related components related to the activity in the inferior temporal (IT) area are evoked after the presentation of a face. In addition, the activity in the IT area plays an important role in the detection processing of a face and the contours of a face affect the activity in the IT area.–Event-related components related to the superior temporal sulcus (STS) activity are larger when eyes are averted than when looking straight.–The direction of the features of a face affects the face perception processing in the right hemisphere. On the other hand, the matching of the direction between the contours and features of a face affects the processing in the left hemisphere.–Random dots blinking, which presents stimuli without a change in luminance during the presentation of a face, is a useful visual stimulus method to investigate face perception processing.Facial information, such as sex, familiarity, and expression, plays an important part in our daily lives. By judging facial information, especially expressions, one can read and sympathize with others’ emotions through social interaction. In psychology, human face perception is one of the main topics. Therefore, many studies have been performed and models of human face perception were suggested e.g., .In addition, there are many studies of human face perception using non-invasive methods for human subjects. Functional magnetic resonance imaging (fMRI) has an excellent spatial resolution that is sufficient to detect activated areas due to an increase in blood flow in the brain areas. Previous fMRI studies on human face perception revealed a specific area related to face perception, termed the fusiform face area (FFA)  has an excellent temporal resolution. In addition, high-density recordings can provide reasonably good spatial resolution. Therefore, EEG detects time sequences of human face perception. A well-known component related to human face perception is a negative component that occurs at approximately 170 msec after the presentation of a face, which is termed N170  the inferior temporal (IT) activity evoked by a face ,b, (2) tA previous study using MEG  investigDuring MEG recording, 1M and 2M components were identified for face with opened eyes and face with closed eyes in both hemispheres. The 1M component was identified approximately 132 msec after the presentation of each stimulus and the activity in the occipital area related to primary visual perception represented by a change in luminance was estimated from the 1M component. The 2M component was identified approximately 179 msec after face stimulation (face with opened eyes and face with closed eyes) in ten subjects for each stimulus in the right but in only five subjects in the left hemisphere. In addition, there was no significant difference in the 2M latency between face with opened eyes and face with closed eyes. The activity in the IT area around the FFA related to face perception was estimated from the 2M component.Another previous MEG study  investigIn this study, 2M components were recorded for all subjects for each condition from the right hemisphere . On the The above studies ,b suggesThe above studies ,b revealThe waveforms were recorded from 204 gradiometers by a 306-channel biomagnetometer in a representative subject following stimulus onset in CDL  and the Previous ERP studies reported that it takes longer to recognize an image of the eyes only than the whole face . MoreoveIT activity reflected the detection process of a face during face perception in our previous studies ,b, but wTherefore, we investigated whether eye direction indicates attention to others using faces with different eye directions in our previous study . FourteeThe superior temporal sulcus (STS) plays a role in the perception of gaze direction, eye and mouth movement, and body movement according to previous studies e.g., . An fMRIIn previous psychophysical studies e.g., , people Our MEG study  was perfThe evoked components  by each condition were identified, with the 2M component being the largest and most major among these components from the right and left hemispheres to LHF and RHF stimulations in representative subjects .2M and 3M latencies were longer for the inverted condition than for the upright condition in the right hemisphere and shorter in the left hemisphere , althougIn multiple source modeling, the 3-dipole model  was the most appropriate for upright and inverted conditions, but not for the objects condition.A previous study  revealedThere was a difference in the face perception process between the right and left hemispheres; however, it is unknown why there is such a difference and how upright and inverted faces were processed. We investigated the effects of inverting facial contours (hair and chin) and features  on processing for face perception using MEG .We examined ten normal adult subjects with normal or corrected visual acuity and presented three conditions as follows : (1) uprIn both hemispheres, ECDs were located in the IT area, corresponding to FFA, for all conditions from all subjects. The waveforms were recorded from 204 gradiometers by a 306-channel biomagnetometer in a representative subject for the U&U condition and the waveforms in all conditions at representative sensors, where the largest component was identified for the U&U condition in each occipital or temporal area of the same subject, are shown in Concerning the peak latency of the ECD, the latency was significantly longer for U&I and I&I than for U&U in the right hemisphere and significantly longer for U&I than for U&U and I&I in the left hemisphere . On the In the U&U condition, the spatial relationship among the facial features  was intact and the features were upright; however, in the U&I and I&I conditions, the features were inverted but retained their spatial relationships even though the contours (hair and chin) of the face were upright or inverted. The U&U condition was considered “easier” to encode because it fits a normal template. In this study , the ECDIn previous studies that investigated face perception, the luminance differed after visual stimuli were presented and the occipital area related to the primary visual perception was activated by this change in luminance. Therefore, the IT area related to face perception cannot be completely separated from strong- and long-lasting activities in the primary visual areas.A useful visual stimulus method, random dots blinking (RDB), uses temporal changes in patterns of many small dots to present stimuli without a change in luminance during the presentation of an object , which reduces activities in the occipital area related to the primary visual perception, and detects only activity in the IT area related to visual form perception, such as faces and letters . In the We investigated the face perception process using the RDB method in ten normal adult subjects with normal or corrected visual acuity . We presIn a previous study using RDB , a clearThe grand-averaged waveforms of evoked components after stimulus onset are shown in This study  revealedOne previous study using intracranial ERP  reportedIn this review, we discussed the following characteristics of face perception processing based on our previous EEG and MEG studies.First, event-related components related to activity in the IT area, corresponding to FFA, were evoked after the presentation of a face, and the detection process for a face was shorter than that for eyes because the face is important for social communication. The facial contours affect and shorten this process; therefore, they are one of the most important parts of face perception.Second, event-related components related to the STS activity were larger when eyes were averted than when straight. This suggested that the direction of gaze, especially eye aversion, is an important indicator of an individual’s focus for automatic and unconscious attention.Third, the face perception process was shortened by an upright face in the right hemisphere, whereas it was shortened by an inverted face in the left, revealing hemispheric differences in the face perception process. In addition, the direction of the features of a face affects the face perception processing in the right hemisphere, whereas matching of the direction between the contours and features of a face affects the processing in the left.Lastly, using RDB without a change in luminance during the presentation of a face, the IT activity related to face perception was specifically detected without interference by activity in the occipital area related to the primary visual perception process. According to this method, the face perception process in the IT area is shorter for an upright face rather than for an inverted face and non-face stimuli.KM and SW designed and performed the experiments, and performed data recording and statistical analyses in our previous studies. YT made the program used for the stimulus presentation in our previous studies. KM wrote this review. RK made suggestions about this review. KM had the overall responsibility for this review. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Gnathonemus petersii specimens. Robustness analyses of input variability were performed to discard overfitting and assess validity. Results show that the set of SPI patterns is consistently reproduced reaching a dynamic balance between synaptic properties in the network. This model can be used as a tool to test novel hypotheses regarding temporal structure in electrogeneration. Beyond the electromotor model itself, the proposed methodology can be adapted to fit models of other biological networks that also exhibit sequential patterns.Mormyridae, a family of weakly electric fish, use electric pulses for communication and for extracting information from the environment (active electroreception). The electromotor system controls the timing of pulse generation. Ethological studies have described several sequences of pulse intervals (SPIs) related to distinct behaviors . Accelerations, scallops, rasps, and cessations are four different SPI patterns reported in these fish, each showing characteristic stereotyped temporal structures. This article presents a computational model of the electromotor command circuit that reproduces a whole set of SPI patterns while keeping the same internal network configuration. The topology of the model is based on a simplified representation of the network with four neuron clusters (nuclei). An initial configuration was built to reproduce nucleus characteristics and network topology as described by detailed morphological and electrophysiological studies. Then, a methodology based on a genetic algorithm (GA) was developed and applied to tune the model connectivity parameters to automatically reproduce a whole set of patterns recorded from freely-behaving  Pulse mormyriforms, a group of weakly electric fish, produce electric pulses with high temporal precision. These fish have the ability to polarize their body in fast voltage transients whose deflection in the fish surroundings is detected by the fish using a specialized electric organ Caputi, . The eleInformation is encoded in the fish signal using a multiplexed temporal coding , simpler neural control motor systems with rhythmic outputs , with the development of a multiobjective fitness function for this system, was applied for synaptic parameter setting. Patterns recorded from Pulse mormyrids generate a wide variety of electrical activity patterns using different sequences of pulse intervals (SPIs) and resting IPIs range from ~100 to 300 ms. Previous studies have described several stereotyped SPIs related to typical social behaviors  are well-defined behavioral firing patterns, although they show a large temporal variability  not only between patterns but also between samples of the same pattern. Examples of this variability in SPI displays, especially regarding accelerations, are shown in Carlson and Hopkins . Also, pFour different neuron ensembles in the electromotor command chain were modeled. They were the medullary command nucleus (CN), the mesencephalic precommand nucleus (PCN), the adjacent thalamic dorsal posterior nucleus (DP), and the dorsal region of the ventroposterior nucleus (VPd). Each nucleus was simulated using the neuron model developed by Izhikevich . This siwith an auxiliary after-spike resetting:v represents the neuron's membrane voltage and u is a voltage variable representing the combined action of ionic current dynamics. The parameters a, b, c, and d set the working regime of the neuron model. isyn is the model external input.where a, b, c, and d, as shown in Izhikevich . Model parameters were also selected to ensure that the intrinsic dynamics could reproduce the characteristic temporal structure observed in the resulting pattern evoked after activation of each nucleus . This was an assumption based on physiological results in network behavior. However, the intrinsic characteristics of the neurons in each nucleus are not known in detail, neither under spiking nor step function inputs. Also, there was considerable variation in the timing of the first spike of the burst relative to CN, intraburst firing rate, burst duration, and the number of spikes per burst from VPd units, so none of these characteristics was a priori selected. Finally, the CN model was configured to a phasic spiking firing regime as it mainly integrates inputs from DP and PCN  during a pulse (tf < t < tr) and after the pulse (tr < t) was calculated as follows:We reproduced neural projections using a model of chemical synapses as chemical inter-nuclei communication takes place in the real electromotor command network. When the pre-synaptic target generates an action potential in these synapses, a certain amount of neurotransmitters are released and bind to the postsynaptic receptors. The mathematical description used to model this behavior is based on a description of the receptor bindings  was detected when the presynaptic neuron's membrane potential crossed a given threshold. The time between tf and tr was defined as the maximum release time (tmax). Both threshold and tmax were tuned as a parameter of the synapses  is the post-synaptic potential at time t, and Esyn is the synaptic reversal potential at the same time.where DP, ESPCN,ISDP, ISPCN, ESCDP in ad hoc iterative tuning of the parameters of the model was performed to match previously described dynamics in the electromotor command network: (i) DP/PCN units firing sporadically before CN, (ii) DP/PCN remaining silent for tens to hundreds of milliseconds after an action potential from CN, and (iii) VPd firing a burst of action potentials during DP/PCN silence starting ≈1–8 ms after CN activation.The topology of the model was set up using a standard configuration of the model of chemical synapses  was developed and applied to evolve the synapses' parameters to reproduce the variability of the electromotor command system patterns. Both synthetic and recorded sets of patterns were used during GA optimization, resulting in two different configurations of the model: S-GA (optimized to synthetic patterns) and R-GA (optimized to recorded patterns).I in the population of the GA was conformed by a set of 20 parameter values: α, β, g, tmax for each of the 5 synapses of the model , each one corresponding to a target SPI: Acceleration (Sacc), scallop (Ssca), rasp (Srasp), cessation (Scess). Simulation cases (Spat) established the current inputs required to reproduce the pat pattern. Each individual I was modeled under all four simulation cases. The fitness function of the overall individual (f(I)) was defined as the sum of the fitness results under each case:Each individual, defined by a set of values for the previously specified parameters, was evaluated being simulated under a predefined set of 4 different simulation cases  where pi is each IPI arranged successively. The output of a simulation case was also defined in terms of an ordered sequence of IPIs. Sequences were normalized to the same duration  and regularly interpolated  to detect the pattern shape. This was done to compare and subsequently differentiate them. The SPI patterns are better defined by the increasing/decreasing slopes between IPIs better than the absolute timing values  were defined in terms of an ordered sequence of IPIs (refer to  thus obtfpat(I) was calculated by comparing the target SPI patterns after these transformations (MSE) as follows:Finally, the fitting value fpat(I) for each pattern  was calculated for each pattern this way, the global fitness function was calculated as defined in Equation 6.Mean squared error (Equation 7) was calculated for all target SPI pattern examples . Then, only the target example with the minimum MSE  was considered to calculate VPd, InDP, InPCN in Since the model presented here is multi-objective , a set of four different simulation cases (each one related to a distinct SPI pattern) was determined. Each simulation case was defined by the input current values received by the model . Input values for each SPI case are described in Analysis for all the different configurations of the model was conducted to assess robustness to input variability and discard overfitting to the predefined stimulation cases. The step functions used as inputs in the simulations are meant to reproduce the nuclei activation associated with each SPI refer to . Consequn in the p pattern simulation case ) was given by:A set of different simulations was conformed by modifying the intensity and duration of the predefined model inputs  = 0, which is the case in the central point of robustness charts that we will discuss later  > 0. Note that the fitness function was used again to measure the distance to target patterns but no GA was employed.The robustness of the model is evaluated below in terms of the relative increments and decrements of this value. Strong robustness is defined as the ability to maintain ΔG. petersii specimens (R-GA) was analyzed  for the ESPCN synapse . Also, the corollary discharge has a larger maximum release time than any other of the synapses in the model, which is coherent with the idea that it might be an indirect pathway. These characteristics are always present in all best individuals obtained from the GA, and they are also shared with another configuration of the model which was fitted to synthetic patterns using the GA . It is worth noting that, contrary to what happened in the initial model, in both R-GA and S-GA configurations CN integrated several DP spikes before firing , better complying with the accelerations from the fish  as described in Equation 9. Results did not show relevant decreases in the fitting value for changes up to 50% in the intensity and duration of the default simulation values. Conversely, they showed a limited increase of the fitting value when a slight decrease in duration was balanced with a slight increase in intensity. The result was the same when slight increases in duration were balanced with slight decreases in intensity. These results show that overfitting to simulation timings has been avoided when considering all the patterns at the same time. To also discard overfitting to each SPI pattern, a robustness analysis disaggregated by pattern was also performed ; similar fit (−100 < Δf < 100); and better fit (Δf > 100). It is shown that even in the worst fit results, the internal temporal characteristic of each SPI pattern was reproduced: the sudden drop to the shortest IPI followed by an almost immediate recovery in the scallop; a series of almost regular short IPIs  in accelerations; a sudden drop followed by an acceleration-like tail of regular IPIs; and a long pause of hundreds of milliseconds in EOD production during cessations. Rasps were the least robust SPI pattern in R-GA, but even those with the worsts fitting results kept a recognizable rasp shape.In The robustness analysis also showed that the differences between simulations of distinct SPIs were larger than the differences between simulations of the same SPI with variated inputs refer to . To measApteronotus leptorhynchus to study the electric organ signal and its spatiotemporal features in wave-type fish. In Shifman and Lewis  play an important role in reproducing the whole set of SPIs observed in experimental data.A multi-objective genetic algorithm (GA) was developed and applied to tune the synaptic parameters of the model. The proposed GA optimization method allowed us to readily obtain different parameter configurations, which optimize the ability of the model to reproduce all target patterns under different activations in the electromotor command network.Automatic synaptic parameter optimization allowed to prove that it is possible to reproduce the four pre-selected SPI patterns using the same intrinsic dynamics and connectivity reaching a dynamic balance between the synaptic properties in the network. In the resulting model each nucleus is represented by a single , sometimes they also lead to losses of SPIs internal structure but, as long as the proper nucleus/nuclei is/are stimulated, it does not occur in such a critical way as it happens with changes in the synaptic parameters refer to . RobustnThe implemented model and its computational efficiency also enable novel closer-to-natural stimulation techniques to perform more realistic closed-loop experiments with the real system  is made availableThe raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.The animal study was reviewed and approved by Ethics Committee of Universidad Autónoma de Madrid (TIN2017-84452-R/CEI-88-1661).The model was conceived and designed by AL and FR. Literature review was done by AL, FR, and PV. AL performed the experimental recordings and runs both the simulations and the GA optimization. AL and FR analyzed the results. AL, FR, and PV wrote the manuscript. All authors contributed to the article and approved the submitted version.This study was supported by AEI/FEDER grants TIN2017-84452-R, PID2020-114867RB-I00, and PGC2018-095895-B-I00. The funders had no role in study design, data collection, analysis, decision to publish, or manuscript preparation.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


The high-speed hand-piece bur is one of the methods to perform tooth sectioning during the minimally traumatic extraction of impacted mandibular third molars. During tooth sectioning, the breakage of the bur might take place when it is improperly used. Three cases of the breakage and displacement of a high-speed hand-piece bur during extraction are reported, aiming to remind dental surgeons of this complication.The bur fragment in case 1 was embedded in the mandibular bone under the previously removed crown of tooth 48 and distal to tooth 47. The bur fragment in case 2 was embedded in the lingual edge of the socket and partly beneath the mucosa on the lingual side. The position of the bur fragment in case 3 was similar to that of case 1 but was completely embedded in the spongious bone. The three cases were performed by first-year residents, and all of the bur fragments were successfully removed by attending doctors after accurately locating them by radiological examination.In order to avoid breakage of the high-speed hand-piece bur, the number of uses of the bur should be monitored and the integrity and state of the bur should be carefully checked. Moreover, light pressure with little lateral force should be used during tooth sectioning. If bur breakage and displacement occur, the retrieval protocol should be determined based on the imaging findings and conducted as soon as possible to avoid serious consequences. Surgical removal of impacted mandibular third molars is one of the most common surgical procedures performed in dentoalveolar surgery. Nowadays, minimally traumatic tooth extraction is widely applied, since it improves the clinical outcome, including reducing healing time, discomfort and inflammation , 2. FurtThe first patient was a 24-year-old male who was undergoing extraction of a right impacted mandibular third molar tooth 48). According to the classification of Pell and Gregory . Accordi, it was The second patient was a 30-year-old female who was undergoing extraction of a left impacted mandibular third molar (Fig. The third patient was a 25-year-old male with a left impacted mandibular third molar Fig. . This waDespite efforts to perform tooth removal carefully, accidents may happen sometimes. Among the tooth extraction instruments, the dental elevator and high-speed bur have a relatively thin working tip. The breakage and migration of their working tip might take place, although this has rarely been reported. If the pieces of broken instrument are left in the body, symptoms such as swelling and pain associated with infection may follow . da SilvTooth sectioning related to impacted tooth extraction is a daily procedure in oral and maxillofacial surgery. High-speed dental hand-piece burs have a high cutting efficiency and might be acquired in every dental clinic to become one of the minimally traumatic methods for tooth sectioning. However, dentists should always pay attention to the probability of overheating, subcutaneous emphysema and/or breakage , 14. SurBetween December 2018 and July 2020, there were three cases of breakage and displacement of a high-speed hand-piece bur in the Department of Oral and Maxillofacial Surgery at the Affiliated Stomatology Hospital of Southwest Medical University, as described above. The incidence rate was around 0.06%. All cases in this report involved breakage of a surgical fissure bur, with the same broken point. The burs used in all cases were the same model products from the same company. According to the product instructions, the bur is not disposable and can be reused after strict high-pressure steam sterilization. On the other hand, the three cases were performed by three first-year residents, and the removal of bur fragment was performed by attending doctors. Based on these three cases, the following measures should be taken to avoid breakage of a high-speed hand-piece bur.First, the number of uses of the bur used should be monitored. Although the high-speed bur used in dental clinical practice may be subjected to fatigue from sterilization, it might also become worn and break after using it several times. It is advisable to retire the bur after more than 20 uses and to use reliable brands and products with good quality . The intIf breakage and migration of the bur do occur, the position of the broken fragment should first be confirmed by imaging. In the three cases, a panoramic radiograph was first used to confirm the existence and overall information of the bur fragment, since the bur fragment might be sucked in by the saliva ejector. Moreover, the reason for using a panoramic radiograph was its cheaper price and less radiation compared to CBCT. On the other hand, CBCT might be the optimum imaging examination, since it provides a three-dimensional image. However, the accurate location of the segment might be impaired by metal artifacts . DentistMoreover, it is wise to remove the fragment as soon as possible in order to prevent it from migrating into a neighbouring space. The delayed removal of the fragment might carry the risk of infection, thrombosis, erosion into the carotid artery or one of its branches and interference with some cranial nerves . The firIn conclusion, the following learning points about the prevention and treatment of bur breakage could be drawn. Firstly, the number of uses of the bur should be monitored, and it should be retired within 20 used during tooth sectioning related to impacted tooth extraction. The integrity and state of the bur should be carefully checked to avoid excessive wear and tear. Secondly, adequate irrigation during the operation is important and helps to reduce the temperature of the bur and its surrounding tissue, as well as prevent breakage of the bur to a certain extent . Thirdly


The surface spectral reflectance of an object is the key factor for high-fidelity color reproduction and material analysis, and spectral acquisition is the basis of its applications. Based on the theoretical imaging model of a digital camera, the spectral reflectance of any pixels in the image can be obtained through spectral reconstruction technology. This technology can avoid the application limitations of spectral cameras in open scenarios and obtain high spatial resolution multispectral images. However, the current spectral reconstruction algorithms are sensitive to the exposure variant of the test images. That is, when the exposure of the test image is different from that of the training image, the reconstructed spectral curve of the test object will deviate from the real spectral to varying degrees, which will lead to the spectral data of the target object being accurately reconstructed. This article proposes an optimized method for spectral reconstruction based on data augmentation and attention mechanisms using the current deep learning-based spectral reconstruction framework. The proposed method is exposure invariant and will adapt to the open environment in which the light is easily changed and the illumination is non-uniform. Thus, the robustness and reconstruction accuracy of the spectral reconstruction model in practical applications are improved. The experiments show that the proposed method can accurately reconstruct the shape of the spectral reflectance curve of the test object under different test exposure levels. And the spectral reconstruction error of our method at different exposure levels is significantly lower than that of the existing methods, which verifies the proposed method’s effectiveness and superiority. The visual system is important for humans to sense the external environment. The eye can sense light radiation in the visible light range in the visual system. The light stimulation photoreceptor cells receive will eventually form a comprehensive vision in the brain. By simulating visual perception, digital cameras use color filter array sensors and image signal processing technology to record the radiation spectrum of natural scenes as a color image that conforms to human visual perception. However, color imaging technology has a limited ability to reproduce colors and characterize objects, and the information contained in the visible spectrum is far beyond the RGB data. The metamerism problem of color imaging is the key issue of its use in high-fidelity color reproduction and high-precision material characterization .It is well known that spectral reflectance is the “fingerprint” of color information, which can effectively overcome the influence of light source and observer on color reproduction and object characterization. It is often used in agriculture, cultural relic protection, skin health monitoring, and other fields . In addiReconstructing multispectral images of scenes from RGB images has been widely researched in many fields. Spectral reconstruction is one of the ill-conditioned inverse problems . The samCurrent spectral reconstruction methods are mainly divided into two different classes: machine learning and deep learning-based methods . The macIn recent years, with the rapid development of deep learning in the field of computer vision, classical network models, such as convolutional neural networks and generative adversarial networks, have been used in spectral reconstruction. Deep learning-based spectral reconstruction models usually use a large number of data sample pairs as support to establish the mapping relationship between RGB images and multispectral images. For the deep learning-based spectral reconstruction, An optimized deep learning-based spectral reconstruction method is proposed based on a lightweight convolutional neural network (CNN) to address the issues arising from the current deep learning-based spectral reconstruction methods, such as a large number of model parameters and sensitivity to exposure changes of test images. Moreover, some optimized measures are integrated into the network. First, the input RGB training image is randomly multiplied by an exposure adjustment coefficient during model training so that the model can see images with more exposure levels. According to the exposure adjustment coefficient of the training image, the reconstructed multispectral image (RMSI) is reciprocally corrected in the loss function to make the model learn the exposure invariant spectral features that correspond to the training images. Secondly, the dense connection mechanism in the original model not only helps to alleviate the problem of model gradient disappearance but also greatly reduces the number of model parameters. Finally, an attention mechanism is introduced into the model to improve spectral reconstruction accuracy by adaptively weighting the feature channels. Experimental results show that the proposed method not only achieves the performance of exposure invariant but also exhibits better spectral reconstruction accuracy than existing methods.The imaging process of a color digital camera involves three factors, light source, object, and camera. The light sources are usually characterized by their spectral power distribution. When the light source irradiates the surface of the object, the object will selectively absorb some wavelengths of energy and reflect the rest to form a radiance spectral that integrates the light source information and the reflection characteristics of the object. The radiance spectra are focused and incident on the camera sensor through the lens. After photoelectric conversion and analog-to-digital conversion, the radiance spectral forms a raw format digital image on the camera sensor, and then the raw image undergoes a series of image signal processing  to form the visually pleasing color image .The imaging mentioned above process of a digital camera can be generally divided into a linear imaging stage and a nonlinear processing stage. The linear imaging stage is from the radiance spectral to the raw image, and the nonlinear imaging stage is from the raw image to the final visual pleased color image. However, because different brands of cameras usually use different image signal processing algorithms, and because they are all the company’s intellectual properties, it is hard to accurately and uniformly express the nonlinear stage. The current research on spectral reconstruction is carried out in the linear imaging mode , as showdi represents the response of the i-th channel of a pixel in the image, l(λ) is the spectral distribution of the lighting source, r(λ) is the spectral reflectance of a point on the surface of the object, t(λ) is the overall transmittance of the camera lens optics, fi(λ) is the transmittance of the i-th channel filter of the camera, s(λ) is the spectral sensitivity function of the camera sensor, λ indicates the wavelength, ni represents the noise signal of the i-th channel of the digital camera, im = l(λ)t(λ)fi(λ)s(λ) represents the overall spectral sensitivity function of the i-th channel of a digital camera , t(λ), fi(λ), and s(λ), and r represents the N × 1 dimensional spectral vector of a pixel.where Because of the high correlation between RGB images and the corresponding multispectral images, learning-based methods can be used to model the mapping between RGB and multispectral images. In recent years, with the success of deep learning in many computer vision tasks, CNN based methods have gradually been applied to spectral reconstruction. Assuming that an RGB image and its corresponding multispectral image are given, the mapping between RGB and multispectral image can be described as follows:d is the digital response value of any pixel in the RGB image, r is the spectral reflectance corresponding to the pixel, and f(⋅) is the mapping model from RGB values to multispectral reflectance. When the spectral reconstruction model f(⋅) is established, the corresponding spectral reflectance r can be obtained by Eq. 3 for any given pixel response value d. The multispectral image corresponding to the RGB image is obtained.where As mentioned in the introduction section, neural networks can learn the mapping relationship from RGB images to multispectral images, and convolutional networks and generative adversarial networks are gradually being used in spectral reconstruction. However, existing deep learning-based spectral reconstruction models usually need to train millions of model parameters. And during model training, convolution operators are used to extract deep image features. Multiple sets of convolution operators are usually superimposed to improve the network’s performance, making the deep learning spectral reconstruction methods more complex than machine learning-based ones.Relu, Prelu, and Sigmoid functions) and non-zero bias terms are always included in the model. Therefore, obtaining the corresponding linear output is difficult for the existing deep learning-based spectral reconstruction model when a set of linear inputs is given. In other words, the existing deep learning-based spectral reconstruction models are exposure sensitive and cannot guarantee the correctness of the reconstructed spectral curve  is the activation function. Without considering the influence of exposure change on spectral reconstruction, the corresponding output y can always be obtained for any given input x. However, considering the exposure influence on spectral reconstruction . The training process is shown in The proposed method is based on the neural network model developed by In addition, the attention mechanism is introduced into the original model to further improve the spectral reconstruction accuracy. Currently, the commonly used attention modules include channel attention, spatial attention, and non-local networks . If the For the existing attention modules, The convolutional block attention module (CBAM) is a tandem hybrid attention module . It lears is the compression parameter. In this article, the values of s are 4, 32, and 16, respectively. After the feature map output is by the first fully connected layer, it is activated by the Relu function and restored to the original dimension by the second fully connected layer. The Sigmoid function processes the feature information processed by the second fully connected layer. At this stage, the weight calibration of the original features is completed. Then, the spatial attention mechanism module takes the output of channel attention as input and uses average pooling and maximum pooling to integrate channel feature information. After that, the two parts of the features are combined using 1 × 1 convolution for dimensionality reduction, and finally, the required mask is acquired through the Sigmoid activation function. The overall expression for CBAM attention is expressed as Eqs 6, 7:The feature channel information is first compressed into a real number through the average pooling and max pooling layers for the channel attention module. Then the feature weights are extracted through two fully connected layers, including the activation function. Among the fully connected layers, the first fully connected layer can reduce the feature dimension to 1/s of the input, where x, δ, and MLP are the input, activation function, and multilayer perceptron, respectively. C is the output channel attention feature, and S is the output spatial attention feature. Based on Eqs 6, 7, the weight of each channel can be adaptively learned and used to weigh the corresponding channel features to improve spectral reconstruction accuracy.where This is plotted in After shallow feature extraction and a densely connected network, a feature information map with 128 layers is obtained, which is input into the reconstruction layer that includes three layers of convolution. The kernel size of each layer of convolution in the network is set at 3, the activation function is Relu, and the CBAM module is added to the shallow feature extraction and reconstruction layer to further improve the robustness of the network. Finally, we obtain the RMSI corresponding to the input of the RGB image.To test the effectiveness and superiority of the proposed method, we carried out the verification experiment. The proposed deep learning-based spectral reconstruction framework is implemented in TensorFlow and is trained using the platform of Intel Xeon and Tesla V100. The database for the experiments is NTIRE2018 , which iIn addition, in the NTIRE2018 challenge, 53 multispectral images with the same spatial and spectral resolution were added to further expand the dataset, so the experiment finally used 256 multispectral images as training data in this per, and their corresponding RGB images were acquired using the same method as NTIRE2018 and NTIRE2020 , 2020. Dn represents the spectral bands, i represents a pixel in a multispectral image, IR represents the reconstructed multispectral images, and IG represents the ground-truth multispectral image.In the experiment, the spectral root-mean-square error (RMSE), the mean relative absolute error (MRAE), and the spectral angle mapping (SAM) error are used to evaluate and compare the spectral reconstruction accuracy of different models. The smaller the value of the evaluation metrics, the closer the RMSI is to the ground truth and the better the performance of the method. Calculation of the evaluation metrics is shown in Equs 8–10, where k means the exposure adjustment coefficient k adjusts the exposure level of the test image before it is fed into the framework. The value of k is used in 1, 0.25, 0.5, 2, and 4 during the testing stage. Additionally, when calculating the spectral reconstruction error, the RMSI is corrected to 1/k of the output.Using the experimental conditions mentioned above, we tested the effect of the proposed deep learning-based spectral reconstruction method compared with several current advanced methods, such as k is equal to 0.25, 0.5, 2, and 4. The proposed method is significantly better than the HSCNN+ and other methods. The overall average spectral reconstruction error of all the tested exposure levels is 0.38, 2.43, and 1.66%, respectively, which is significantly better than the existing advanced deep learning spectral reconstruction algorithms.It can be seen from the experimental results summarized in In addition, according to the standard deviation of each method under different test exposure levels, there is no significant difference in the spectral reconstruction error of the proposed method under different test exposure levels. However, the standard deviation of the compared methods under different tested exposure levels is quite large. On the one hand, the experimental results in The RMSE map of the RMSI and the GMSI of each method are plotted in From the results in In addition, the verification of the proposed method in reconstructing the non-uniformity illuminated images is also tested. As shown in The spectral reconstruction errors of the original method developed by In addition, although the improvement in the spectral reconstruction accuracy is not so significant when introducing the attention mechanism into the proposed network, the spectral reconstruction accuracy of the proposed method does improve at some specific tested exposure levels. For example, at the tested exposure level of 0.5, the error of the evaluation metric MRAE is reduced by about 8% compared with the only use of data enhancement in the proposed method. At the tested exposure level of 2, the error of the evaluation metric MRAE is reduced by about 6% compared with the pure data enhancement. Moreover, at the tested exposure level of 4, the error of the evaluation metric SAM is reduced by about 5% compared with the pure data enhancement in the proposed method. The proposed method may be further improved based on data enhancement.At last, the number of learning parameters for each model in this article is counted and plotted in Multispectral image acquisition is the prerequisite for its applications. In view of the problems of the existing deep learning-based spectral reconstruction methods, such as a large number of parameters and the exposure invariant. An optimized lightweight neural network for spectral reconstruction is proposed in this article, and the data augmentation and attention mechanisms are introduced into the original method to make it more efficient and exposure invariant. The optimization of the proposed method makes it more robust in practical applications in an open environment with variable light sources and non-uniformity illumination. The shape of the reconstructed spectral reflectance curve of the target can be well preserved using the proposed method under different exposure levels, which provides the foundation for high-precision multispectral image acquisition in an open environment. However, improving the module attention mechanism proposed in the proposed method does not significantly improve the spectral reconstruction accuracy. More research will be carried out to further improve the method in the future.The original contributions presented in this study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.JL: methodology, data collection and analysis, and writing-reviewing. LX: methodology, data collection and analysis, and writing. ZZ: data collection and analysis. JZ and AL: data collection. HL: data analysis. XH: methodology, funding acquisition, and writing-reviewing. All authors contributed to the article and approved the submitted version.