Modern hip implants have a modular design. In case of wear or other damage it allows surgeons to change the tribological partners, i.e., the acetabular liner and femoral ball. In both revision and primary surgery, the secure joining of the implant components is important for the success of the operation. The two components, namely the ceramic liner and hip cup, are connected via a conical press connection and should be concentrically aligned to avoid chipping. Malseated liners can reduce the life span in situ. The amount of the joining force, which is usually applied via a hammer, depends on the surgeon. In this study, an alternative joining method for acetabular ceramic liners in hip cups was investigated, which intends to make the process more reproducible and thus safer. For this purpose, a handpiece was used to apply a defined force impulse of 4 kN. For the concentric alignment of a ceramic liner in the hip cup, an adapter was developed based on findings via a qualitative finite element (FE) analysis. Insertion and pushout tests of the acetabular cup–liner connection were performed in the laboratory with the new instrument (handpiece with the connected adapter) to evaluate the functionality of the instrument and the reproducibility of the new insertion method. For comparison, liners and acetabular cups were joined using a testing machine according to the standard. The presented results demonstrate the technical proof-of-concept of the new joining method under laboratory conditions. They meet the acceptance criteria of established manufacturers, which proves the equivalency to proven methods for joining modular implant components. To verify the improvement of the new joining method compared to the conventionally used joining method, an application-oriented study with different surgeons and the new joining instrument under clinical conditions is necessary. Due to aging societies , increasModern hip implants are modular, i.e., the joint-forming implant component consists of at least three separate parts. Through revision surgery, friction partners such as liner or head can thus be changed without having to replace the implant components located in the bone , which preserves the remaining bone material. The modular implant components are connected to the elements anchored in the bone via a conical press connection . The acetabular liner  is joined by an impactor instrument and several impacts ,6,7. TheMetal abrasion at the junctions of modular implants is assumed to be a reason for failure and associated revision surgery in total hip arthroplasties ,10,11. AMcAuley et al. investigated the factors influencing fractures in ceramic liners and were able to simulate through practical tests that misaligned ceramic liners increase the risk of fracture. At the same time, the quality of the connection is improved via intense and repeated hammering .Previous studies describe the status of the joining methods for implant components and problems resulting from misaligned liners. They require careful surgical technique, but current joining methods allow for mistakes. A reproducible and controlled insertion method of hip implant components can lower failure rates due to metal abrasion (by reducing relative motions due to insufficient joining forces), chipping or ceramic fractures. Therefore, this work aims to test the technical feasibility of a reproducible insertion method of modular acetabular ceramic liner using a novel instrument. Our hypothesis is that joining over the edge surface of the line is as good as established joining methods. An adapter developed for this instrument aligns the ceramic liner axially. The impaction force is applied via the edge of the liner using an commercial handpiece with a reproducible force impulse of 4 kN . The eva® [In order to implement the new reproducible insertion method, a new instrument has been developed in a collaborative research project called “Smart-I” together with manufacturers of implants and surgical instruments (see the Acknowledgments). The instrument consists of a handpiece designed by endocon GmbH , which is already available on the market under the name safeConnect® . It applFor the concept design of the instrument’s adapter, the first step was examining the conical locking mechanism between the cup and liner using static mechanical FE analysis with the Ansys 2021 R2 software . This is a numerical method used for different physical tasks like strength and deformation testing. For this purpose, the force application angle and location were varied.The joining of the liner and acetabular cup is based on a friction-locked connection, which is realized via a conical press fit. The liner is inserted into the cup with force pulses of hammer impacts. Currently, available instruments, transfer force to the concave surface inside the liner via a ball ,6,7,26. With the help of FE analysis, the conventional joining method was analyzed for varying force introduction angles and locations with an impaction force of 4 kN. In addition to the axial load introduction (0°) and two tilted introduction angles (14.5° and 29°) were studied to quantify the influence of the force angle on the stress distribution in the ceramic liner and the hip cup and the resulting push-out forces.In comparison, an alternative joining method was analyzed in which the force is not introduced to the liner’s curved surface but via the planar upper edge surface of the liner. The FE analysis is intended to verify whether the applied impaction loads are distributed more evenly in the liner through this joining method and whether occurring regular forces are distributed more homogeneously in the joining interface between the liner and cup. In addition, a uniformly applied load at the liner edge could counteract tilting. The influence of the force application location and its direction were investigated using the following model:Material: The material models for the ceramic liner made of ATZ (Alumina Toughened Zirconia Ceramic “ceramys” ) and theContact conditions: The contact conditions for joining the liner and hip cup were realized via frictional contact. The friction coefficient was calculated according to Equations (1)–(3)  based onThe friction value was determined from the axial impaction force Mesh: Tetrahedral elements with a quadratic shape function were chosen for the calculation. A mesh convergence study was performed, where the refinement of the mesh size from 1 to 0.5 mm resulted in only a 0.11% change of the maximum stress in the liner. This is the reason why a mesh size of 1 mm was chosen. The entire calculation mesh contains 93,380 elements and 153,210 nodes with an average skewness of 0.36. Identical contact meshing was carried out in the contact area. The mesh settings were set globally.2 (radius of 2.5 mm). For the alternative insertion method, a constant surface load of 4 kN was applied to the upper side of the liner. The hip cup was fixed at the centric hole in the bottom.Initial and boundary conditions: For the force application see , the surFindings: To evaluate the joint connections of the different load application angles and the new load application method, the push-out force thod see . With anFurthermore, the loads on the liner during conventional joining are very high due to the small force application area see . With ax® [The preliminary numerical investigation shows the importance of axial alignment of the instrument, as an angled load application decreases the resulting push-out force. Furthermore, the load on the liner can be significantly improved by shifting the force applied to the edge surface of the liner. However, according to the current state of the art and to the authors’ knowledge, there are no adapters for pulse-controlled surgical tools such as the safeConnect®  with axiThe simulations showed that the force application via the edge surface of the insert achieves equivalent push-out forces with significantly lower loading of the ceramic insert. However, these are not validated models, which were only used for qualitative load analysis of the joining process. Further experimental validation would be required to use the models for the component design of the cup or insert.In order to test the developed alternative joining method, i.e., the adapter for its suitability for joining ceramic liners in hip cups, these components were joined manually with the new instrument and compared using the testing machine according to ASTM F1820-13. It is standard procedure to measure the attachment strength between the hip cup and liner, specifying the test setup for axial disassembly or the test speed. To evaluate the achieved connection quality, the respective push-out forces according to ASTM F1820-13 were determined and compared. Due to a lack of samples, the number of specimen deviates from the standard. For the two test series (joining via instrument and via testing machine) six repetitions were performed, with one component pair available per series. Due to the expected plastic deformation of the components during the first insertion process, which leads to higher push-out forces compared to the following tests, the first test data were not included in the evaluation. Five evaluable test datasets were used to calculate the mean value of the resulting push-out force.An acetabular hip cup  with a cup diameter of 58 mm manufactured by Aristotech Industries GmbH  and a 32-ceramic liner (ceramys) by Mathys Orthopädie GmbH  were used.When joining with the instrument, the force was transferred to the components via an impulse triggered in the handpiece see a. For thStraight pick up of the liner;Hold the liner;Align the instrument to the hip cup;Transmit forces.An adapter was developed for the instrument used to join the liner and cup, which had the following functions:The implemented preferred concept for the adapter introduces the force via the upper liner edge. The liner is held on the instrument during the implantation process by the clamping force of an upset O-ring on the concave inner surface of the liner. The liner is aligned straight to the instrument by the contact surface on the liner edge. The developed adapter enables one-handed placement in the cup during surgery. For axial alignment, the adapter is guided straight into recesses in the hip cup with the help of lugs on the liner without retaining the hip cup see a. Thus, PPSU  was used as material for the adapter. It is a high-performance plastic often used for surgical instruments, that needs to be resistant to typical sterilization methods. The adapter was manufactured by Mathys Orthopädie GmbH  with the close adjustment of the manufacturing tolerances. It consists of three parts: the main body with a pressed-in threaded sleeve and the O-ring see b.The measured push-out forces for removing the liner from the cup, joined by the instrument and the testing machine, are listed in p-value of 0.15 (t(4.92) = −1.68). However, all tests fulfill the internal acceptance criteria of Mathys Orthopädie GmbH  with 350 N  is 0.233 for the testing machine and 0.212 for the instrument.The quality of the connections of the hip implant components significantly influences the success and life span in situ of operative care with total hip arthroplasty. The push-out force of the modular components is influenced by the surgical technique, like the applied impacting force ,25. The In a study by Lee et al. , ceramic®) already on the market and a newly developed adapter. The push-out forces were compared to the components joined according to ASTM F1820-13. The push-out forces for the joined components were above the acceptance criteria. The mean values for both joint connections do not differ significantly when comparing the push-out forces of the components joined with the instrument and the components joined with the testing machine. It is likely that the new instrument and thus the new joining method over the liners edge had no significant influence on the connection. It is equal to proven methods.Therefore, this work aimed to evaluate the feasibility of a new insertion method with reproducible forces to achieve the press fit connection of the ceramic liner and acetabular cup. For this purpose, these components were joined using an instrument consisting of a handpiece , the testing machine ensures that the target force is still reached. The instrument, on the other hand, triggers an impulse once. Should the components slip during the insertion process, this is not compensated for by readjustment. Furthermore, the testing machine almost ideally applies the forces straight to the components. In contrast, the instrument may have a slight angular misalignment of the axes due to the clearance fits chosen for guiding the lugs of the adapter in the recesses of the cup. This would also explain the greater variation in the values of the push-out forces.A greater number of samples in a larger study would statistically validate this investigation. This data could be used to validate the FE analysis, because the analysis performed in this study was not validated through experiments. But the quantitative statements made were sufficient for the design of the adapter and therefore sufficient for this investigation. Another limitation is that the tests were carried out under almost ideal but simplified conditions in the laboratory. Further studies should still verify the improvement compared to currently used instruments and under real conditions in the operating room.The application of force via impulse, the force level, and the surrounding tissue influences the achieved push-out forces. Although ceramic liner fractures are well known, they are rarely reported in the literature. But there are studies that describe the influencing factors when joining the hip stem and ceramic head. As this connection is also a conical press fit, the dependencies could be relevant for the joining of ceramic liner and acetabular cup as well. From a technical point of view, the connection mainly differs in the length of the cone, which tends to have a positive effect on self-centering and is therefore less conducive to tilting.We compared the push-out forces of impulsively joined components with quasistatically joined implant components. In a study by Wade et al. , six insThe influence of the surrounding tissue and the force level were not considered in this study. Krull et al.  showed tSimulations showed that load application via the liner edge is gentler on the ceramic than conventional load application via the inner concave liner surface. As a result, an adapter was developed which, in addition to the alternative load application, also enables axial guidance and thus straight load application. The small deviations of the tests with a testing machine and the achievement of the acceptance criteria of established implant manufacturers show that when comparing the push-out forces, the new method is equivalent to proven insertion methods. The functional proof-of-concept of the instrument, including the adapter, by joining the hip implant component ceramic liner to the cup was demonstrated.In future research, mechanical tests should be carried out with a larger number of components to validate the FE analyses performed. These tests can answer the question as to whether this new insertion method and the developed instrument could lead to less chipping or fewer ceramic fractures.To show the clinical relevance and the benefit of the new instrument, a comparison with conventionally used instruments with qualified personnel from a clinic is still outstanding. The influence of the bony environment on the achieved joining force or the access routes has not yet been sufficiently investigated, nor have the advantages for liners made of other materials, like PE. All patients who need a hip implant benefit from safe joining method and a longer life span in situ.


The COVID-19 pandemic has underscored the need for real-time, collaborative virtual tools to support remote activities across various domains, including education and cultural heritage. Virtual walkthroughs provide a potent means of exploring, learning about, and interacting with historical sites worldwide. Nonetheless, creating realistic and user-friendly applications poses a significant challenge. This study investigates the potential of collaborative virtual walkthroughs as an educational tool for cultural heritage sites, with a focus on the Sassi of Matera, a UNESCO World Heritage Site in Italy. The virtual walkthrough application, developed using RealityCapture and Unreal Engine, leveraged photogrammetric reconstruction and deep learning-based hand gesture recognition to offer an immersive and accessible experience, allowing users to interact with the virtual environment using intuitive gestures. A test with 36 participants resulted in positive feedback regarding the application’s effectiveness, intuitiveness, and user-friendliness. The findings suggest that virtual walkthroughs can provide precise representations of complex historical locations, promoting tangible and intangible aspects of heritage. Future work should focus on expanding the reconstructed site, enhancing the performance, and assessing the impact on learning outcomes. Overall, this study highlights the potential of virtual walkthrough applications as a valuable resource for architecture, cultural heritage, and environmental education. The COVID-19 pandemic has wrought an unprecedented and far-reaching global impact, prompting governments worldwide to implement sweeping measures to contain the spread of the virus. These measures, including school and workplace closures, gathering and travel restrictions, and stay-at-home orders, have drastically altered people’s mobility and lifestyle patterns [In this sense, collaborative eXtended Reality (XR) technologies, integrating Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), have gained increasing popularity in the educational field as they largely proved their potential to improve learning quality and training performance  by proviAdvanced technologies and techniques in data acquisition, processing, representation, and analysis can improve three-dimensional (3D) digitization and comprehensive recording of valuable architectural and cultural sites. Innovative solutions like photogrammetry-based virtual reconstruction can tackle the issues of morphological and material complexity commonly encountered in cultural heritage sites  while enVirtual walkthrough  applicatThis study develops and tests a virtual walkthrough prototyping application for collaborative exploration and learning purposes of an alley within the Sassi of Matera, a UNESCO World Heritage Site in Italy. It combines photogrammetric reconstruction and HGR to give the virtual visitors an interactive and engaging learning experience. Hand gestures are used to navigate through the 3D scene replicating the real-world scenario and interact with virtual dashboards displaying educational content, according to the didactic needs.The study aims to contribute to the growing body of research on the use of XR technologies for cultural heritage and education and explore the potential of integrating different techniques to create more immersive and engaging virtual walkthroughs that facilitate access to cultural heritage sites and support remote learning activities. The specific scope is to provide users with an integrated and multidimensional educational tool that supports their visit experience prior to the physical visit or during remote learning activities and can be easily accessed. Through the case study, this research demonstrates the potential of such technologies to create a rich and authentic experience that is not limited by physical barriers, allowing for greater accessibility and valorization of sites of historical, archaeological, and architectural interest.The case study was conducted in Matera, a city in the region of Basilicata, Italy. This location was selected due to its unique and culturally rich history ,30.Civita, the Sasso Caveoso, and the Sasso Barisano, and the overlooking Gravina canyon, whose geomorphology influenced its development [grabiglioni) that shaped the housing structures of the Sasso Barisano and Sasso Caveoso built on overlapping terraces. The Sassi, meaning stones, form a complex urban landscape carved and constructed from calcarenite  and arranged in a vertical succession of levels that incorporate natural terraces and cuts. The streets often pass above the roofs of other homes. The urban ecosystem [The city is renowned for its ancient urban center, formed from the elopment . The Civcosystem  is knownThe location provided a challenging and complex urban environment for the virtual reconstruction process, which aimed to recreate the intricate details of the site with accuracy and precision. The irregularity of the structures, combined with the presence of overhanging elements, hollows, and caves, creates a complex three-dimensional geometry that is difficult to accurately reconstruct through manual 3D modeling  and requThus, the reconstruction of a portion of the urban environment was carried out using photogrammetry with the software RealityCapture . This software was chosen because it can process images taken from multiple viewpoints to generate a 3D representation of an object or scene to be used in various applications such as architectural and industrial design, video production , games , and culA total of 1937 photographs  of an alThe output generated by the software was a point cloud , which wSimplify and Filter Selection tools. The first tool reduced the triangle count of the selected reconstructed mesh by specifying the target triangle count while preserving its overall shape and structure. The latter tool created a new model from the previous one by removing specified triangles that were manually selected by basic modes  and via the Advanced Selection tool by different criteria . The simplification process helped to balance the trade-off between the desired level of detail and performance.The reconstructed mesh was highly complex, with a total of Texture Reprojection tool of RealityCapture, which preserves the illusion of a high-quality reconstruction with a mesh of minimal triangle count. The final textures, generated by aligning the input photos and projecting them onto the 3D model, were reduced to a total of 5 with a resolution of To improve the photo-realism, a total of 6 high-quality textures with a resolution of The figures that follow provide a detailed visualization of the reconstructed mesh within the RealityCapture user interface. The comparison presented in The reconstructed model was subsequently exported in the fbx format and imported into a project made with Unreal Engine (UE) 4.27  , which hEmissive Color in the materials of the object.The lighting strategy adopted in UE aimed to prevent incorrect rendering of lights and shadows. This was achieved by exclusively illuminating the scene with the SkyLight Object, which serves as an ambient light source. In addition, the textures were utilized as The 3D model obtained through the workflow described earlier served as a virtual replica of the real-world alley to be used as the environment for the virtual walkthrough’s prototypical application.1.Input capture: a frame is acquired via a general-purpose monocular RGB camera;2.MediaPipe Hand ,39 proce3.x and y components of each hand–knuckle coordinate.Hand gesture prediction: A Feed-Forward Neural Network (FFNN) is used to predict the current hand gesture from the The HGR system used in this study was adapted from the work presented in . Figure The use of a general-purpose monocular RGB camera as an input source has been shown to achieve good hand tracking . AdditioMediaPipe Hands is a well-established approach due to its reliance on single camera-independent RGB frames as input, in combination with its real-time inference speed and high prediction quality ,38,39. Fx, y, and z components, where the z component is obtained using relative depth with respect to the wrist, and for this reason, the coordinate spaces are referred to as MediaPipe Hands ,39 uses  as 2.5D .x, y), defined in the pixel space, of each of the 21 predicted landmarks as input. For the specific task of interest, the third coordinate z was not informative and thus could be disregarded.The FFNN is provided with A dictionary of 13 hand gestures was defined based on their potential use in an interactive virtual urban environment. These gestures, derived from state-of-the-art research ,42,43,44The proposed gesture set is divided into two categories: 6 static gestures  and 7 dynamic gestures , shown in Static gestures are fixed hand poses predicted directly by the FFNN. The FFNN inputs the coordinates provided by the MediaPipe framework, based on the current camera frame. The system considers a gesture valid if it outputs the corresponding gesture representation.Dynamic gestures are defined by hand movement and further divided into single and combo gestures depending on the number of hands tracked. Single dynamic gestures are performed by a single hand, whereas combo dynamic gestures are performed by two hands. The system uses static activation gestures, predicted by the FFNN in a similar manner to static gestures, to initiate the tracking of dynamic gestures. It then employs assertions on the landmarks over subsequent frames to determine the type and direction of the dynamic gesture performed by the user.The distinction between static and single or combo dynamic gestures allows for a more nuanced understanding of the user’s intention and can lead to a more intuitive interaction experience.The FFNN architecture features a single hidden layer with 32 units and employs a rectified linear unit (ReLU) activation function on the hidden layer and a Softmax function on the final layer. The choice of the FFNN among other DL approaches and its implementation strategies were carefully selected to achieve an optimal balance between computational cost, accuracy, and real-time performance, as detailed in .x and y values of 21 hand landmarks, provided by the MediaPipe processing step. As MediaPipe also supports egocentric hand tracking, the samples were taken and labeled in both egocentric and non-egocentric modes, allowing the system to be used with a simple RGB camera placed in front of the user  or mounted on XR devices. The FFNN was trained for 2000 epochs using the Adam optimizer [To train the network, a dataset of y of 98% .A virtual walkthrough prototyping application of the alley within Matera’s Sassi was developed for collaborative exploration and learning purposes. It combines the photogrammetric reconstruction and the HGR system to give the virtual visitors an interactive and engaging learning experience. The virtual environment was augmented into the UE project with the capability of supporting interactions and movements within the virtual space through the HGR system. Hand gestures are used to navigate through the 3D scene replicating the real-world scenario and interact with virtual dashboards displaying educational content. In selected points within the reconstructed alley, as shown in The dashboards—accessible through the mentioned 3D information stands placed in the scene—were triggered by a pre-defined gesture as popup windows refer to . The infThe above-described steps were performed on a high-performance computer workstation equipped with a Microsoft Windows 10 Pro 64-bit operating system, an Intel Xeon W 2275 (Cascade Lake-W) processor with a clock speed of The immersive virtual walkthrough application developed in this study combined photogrammetric reconstruction and a deep learning-based HGR system to offer a highly interactive and collaborative experience for multiple users to explore the historical and cultural site of Matera’s Sassi. The virtual environment created by the application provided precise representations of Matera’s complex historical locations, enabling users to interact with the virtual environment using natural hand movements recognized by the HGR system. The application included educational content on cultural heritage and the environment, incorporating real-time data, static information, and visual media.The application was tested on a sample group of 36 participants who were representative of both genders and composed of high school students. The participants were asked to perform various tasks within the virtual environment, such as exploring the alley, interacting with the virtual dashboards, and navigating the space using the gestures defined in The subjective feedback was collected using questionnaires that asked participants to rate their experience on several parameters, such as level of immersion, ease of use, level of engagement with the educational content, overall satisfaction, and recommendation to others. When rating their experience, participants used a scale of 1 to 5, with 5 representing the highest score. The results of the questionnaires showed that the majority of participants found the application easy to use and engaging. The use of hand gestures for navigation and interaction was praised for being intuitive and enhancing the sense of presence in the virtual environment. The average score for the ease of use was The results of the questionnaires are illustrated in The study did not include task completion time, as it prioritized the users’ experiential and qualitative aspects over performance measures. This allowed users to freely explore the environment at their own pace and in their preferred way.These results indicate that the virtual walkthrough application for collaborative exploration and learning was well-received by the participants. The participants found the HGR easy to use and the created virtual environment detailed, representing accurately the real-world location. The HGR accurately recognized in real time a variety of hand gestures with inputs provided by a general-purpose monocular RGB camera  and translated them into actions in the virtual environment. Our findings showed that having multiple gestures for the same action in a gesture-based navigation system can enhance user experience and accessibility; the approach enables users to choose the most comfortable and instinctive gesture, can provide redundancy in case of incorrect recognition, and can reduce the likelihood of mistaken gestures. The students were able to explore the virtual environment together and interact with virtual objects and elements in a natural and intuitive way, improving the learning experience, with many expressing a desire for continued use. However, it is important to acknowledge that the sample size of the study was limited, and further research with a larger and more diverse participant group would be beneficial to confirm the generalizability of these findings.Overall, this study provides valuable insights and demonstrates the potential of virtual walkthrough applications using photogrammetric reconstruction and HGR to enhance the user experience in collaborative virtual learning environments.The aim is to provide users with an integrated and multidimensional educational tool that supports their visit experience prior to the visit or during remote learning activities  and can be easily accessed.The findings indicate that this approach is effective in promoting the cultural landscape of Matera, which encompasses various aspects such as archaeological, historical, and natural heritage. The immersive nature of virtual environments can offer an engaging and accessible learning approach, and incorporating hand gestures can enhance the experience’s intuitiveness and user-friendliness.The user test led to positive feedback, highlighting the advantages of integrating advanced techniques  to provide precise representations of complex historical real-world locations as well as additional narratives in immersive applications for architecture, cultural heritage, and environmental education. The proposed application has the potential to promote the cultural landscape of Matera and be applied in various different domains, including but not limited to preservation, gaming, and tourism .Future work should aim at exploring new ways to make the virtual experience even more realistic, as well as improving the performance and scalability of the system. This could involve expanding virtual walkthroughs to include additional historical and cultural sites. Additionally, further studies may explore the use of other forms of input, such as voice recognition, to enhance the experience even further. However, deeper analysis is necessary to investigate the impact of these technologies on learning outcomes and user experience; qualitative and quantitative assessments , such as


This paper presents a sound source localization (SSL) model based on residual network and channel attention mechanism. The method takes the combination of log-Mel spectrogram and generalized cross-correlation phase transform (GCC-PHAT) as the input features, and extracts the time–frequency information by using the residual structure and channel attention mechanism, thus obtaining a better localizing performance. The residual blocks are introduced to extract deeper features, which can stack more layers for high-level features and avoid gradient vanishing or exploding at the same time. The attention mechanism is taken into account for the feature extraction stage in the proposed SSL model, which can focus on the most important information on the input features. We use the signals collected by microphone array to explore the performance of the model under different features, and find the most suitable input features of the proposed method. We compare our method with other models on public dataset. Experience results show a quite substantial improvement of sound source localizing performance. Microphone array-based SSL has received a lot of attention from researchers, where DOA estimation is an important research direction in multichannel audio analysis2. DOA is usually represented by two relative angles: azimuth and elevation. In most practical cases, SSL is simplified as a DOA estimation problem. Although SSL is a long-standing and extensively researched topic, it remains a challenging problem to date3.Sound source localization (SSL) refers to estimating the position or direction of arrival (DOA) of the sound source through multi-channel signals. This technology has been well developed over the past few decades and has achieved big progress4 and generalized cross-correlation phase transform (GCC-PHAT)5. Traditional SSL algorithms are based on ideal signal model, so their robustness usually is not very good. With the rapid development of deep learning, more and more researchers are trying to use deep learning methods to solve the SSL problem6. Deep learning-based SSL is often formulated as a classification or regression problem. The classification problem divides the space into different regions. For different inputs, the neural network will output its probability values in different regions. The regression problem, on the other hand, estimates the location coordinates or direction of the sound source directly from the inputs10. With the popularity of deep learning methods, a large number of network architectures and input features are proposed every year, such as convolutional neural network (CNN) and convolutional recurrent neural networks (CRNN), and input features such as short-time Fourier transform (STFT) and generalized cross-correlation (GCC)14. Many studies have shown that deep learning methods possess good performance16. Xiao first proposed DOA estimation by neural networks to obtain sound source angle information using a fully connected perceptron17. Since then, the SSL technology based on neural network has developed rapidly. Hirvonen uses CNN to extract features from multi-channel amplitude spectrograms, and then classifies the audio source position through four full connection layers18. Chabarty proposed a classification method based on CNN to predict the angle of speakers19. The input feature is multi-channel short-time Fourier transform phase spectrograms, and the system consists of three continuous convolutional layers and three fully connected layers. Adavanne presented a pioneering work using convolutional recurrent neural networks for SSL and showed good performance20. The CRNN consists of multiple convolutional layers and recurrent layers. The convolutional layer has been proved to be suitable for extracting information from various input features. The recurrent layer is suitable for learning time information. Therefore, CRNN is often used for SSL.Many traditional SSL methods based on signal processing have been proposed, such as steered-response power phase transform (SRP-PHAT)In the field of SSL, CRNN is the most popular architecture. However, the existing CRNN-based localization models face a dilemma. It is difficult to extract effective high-level features when the model uses too few convolutional layers, while as the number of convolutional layers deepens, it leads to the problem of gradient vanishing or exploding. Therefore, a sound source localization method based on residual network and channel attention module (SSL-RC) is proposed in this paper. We bring in the residual blocks to extract deeper features, which can stack more layers for high-level features and avoid gradient vanishing or exploding at the same time. Meanwhile, we introduce the attention mechanism into the feature extraction stage of the proposed SSL model, which can focus on the most important information on the input features. As for the input features, we select the original audio captured by the microphone array system as the dataset to train the neural network, and compare and evaluate its localization performance under different input features to find the best input features for our model. Experiments compared with other SOTA SSL models on publicly available datasets show that our model get better performance.The problem of SSL can be analyzed from the perspective of array signal processing, and can also be solved by using the idea of deep learning. Deep learning methods can find the relationship between multi-channel signals and sound source locations. Nowadays the SSL method based on deep learning has gradually become a research hotspot. Generally, the audio signals in different channels received by the microphone array is different, because the distance from the sound source to each microphone are different. Deep neural network (DNN) perform SSL by learning this difference and the complex relationship between input features and sound source location. Many existing researches prove that deep learning methods are feasible for SSL.21. Lu integrated some additional convolutional layers and replaced the Bi-GRU layers with bidirectional long short-term memory (Bi-LSTM) layers, and this method improved the DOA estimation accuracy22. In recent years, more and more researchers have proposed improving methods for classical convolutional recurrent neural networks. Guirguis used temporal convolutional network (TCN) instead of bidirectional recurrent layers, which reduce the computational stress and improve the model inference and training speed23. Naranjo improved feature utilization by adding residual squeeze excitation (SE) blocks to the convolutional recurrent neural network. The results show that the introduction of residual SE blocks can obtain better results than the baseline system24. Grumiaux proposed a network model with more convolutional layers and pooling layers, which reduced the loss of information and improved the performance25. Chakrabarty proposed a CNN based supervised learning method to estimate the DOA of the sound source. The phase component of the STFT of the sound signal was used as the input feature for training. At the same time, the impact of the convolutional layer on the localization performance was evaluated26. He used convolutional layers and residual blocks to extract high-level features from the input features for localization, and showed good performance27. Komatsu used gated linear units (GLUs) instead of convolutional layers to enhance the learning ability of the frequency dimension, and the model reduces the angular error28. Deep learning-based SSL models need to extract useful information from the input features for inference learning, but it contains some information that is not important for the final result. How to make good use of the important information in the features to improve the localization effect of the model is a problem worth studying.Researchers have proposed many neural network-based methods for SSL in recent years. Among these approaches, the majority of model rely on the learning of time–frequency domain features of the acoustic signal. Adavanne implements DOA estimation directly through a set of convolutional layers, a set of bidirectional gated recurrent unit (Bi-GRU) layers and a set of feedforward layersIn general, the convolutional module has a direct impact on the performance of the sound source location network model. More and more researches are focused on the optimization of convolutional modules. We will improve the classical convolutional recurrent neural network in this paper. In order to extract high-level features and useful information, we use the residual structure and the channel attention mechanism to improve the information utilization effect.We first describe the architecture of SSL-RC, then the specific feature extraction process of the network architecture is described.Figure Our model takes the combination of log-Mel spectrogram and GCC-PHAT as the input features. Log-Mel spectrogram is widely used in audio processing. This is found by observing the human ear, which is extremely sensitive to signals in some specific frequency bands. In order to extract log-Mel spectrogram, we first divide the audio signal into overlapping frames, then calculate Fourier transform and apply Mel-scale filter in frequency domain. Finally, the energy in each sub-band is calculated and its logarithm is taken to get the log-Mel spectrogram. GCC-PHAT features are based on the traditional time delay estimation algorithm. For the same sound source, sound waves arrive at two microphones at different times due to sound propagation. Based on the above phenomena, the DOA of sound source can be estimated. GCC-PHAT algorithm uses the correlation between the signals received by two microphones from the same sound source to calculate the cross-correlation function. By maximizing the cross-correlation function, the time delay between the sound source and two microphones can be estimated. It can be seen that the log-Mel spectrogram and GCC-PHAT features of audio signals contain rich time and frequency domain information.20 which has three convolutional layers followed by two bi-directional GRU layers and two fully connected layers. We note that the traditional convolutional recurrent neural network has too few convolutional layers, which is not enough to extract high-level features, so we add residual layers to enhance the ability of features extraction in time–frequency information. Generally, the more layers the network has, the stronger the expression ability will be. However, more layers may cause gradient vanishing, which degrade the network performance. It is also difficult to avoid these problems by using some regularization and other optimization methods. The residual structure can avoid the problems caused by the increase in the depth of the convolutional layers, so a deeper network can be designed27. We choose 2D convolutional layer with residual structure to extract features in our model. As shown in Fig. The baseline architecture used for the experiments in this paper is CRNN29. It is a local cross-channel interaction strategy without dimensionality reduction, and effectively avoids the effect of dimensionality reduction on the learning effect of channel attention. The module involves only a few parameters but has a significant effect gain, and the proper cross-channel interaction can significantly reduce the complexity of the model while maintaining performance. Considering the outstanding performance of ECA, we introduce this module into the neural network model of SSL. ECA attention mechanism uses one-dimensional convolutional to efficiently realize local cross-channel interaction and extract the dependencies between channels. Firstly, the input features are subjected to global average pooling operation. Secondly, one-dimensional convolutional operation with convolutional kernel size K is performed, and the weight W of each channel is obtained through Sigmoid activation function. Finally, the weight is multiplied by the corresponding element of the original input feature to obtain the final output feature. For important channel, the output of the sigmoid function is close to 1. While for unimportant channel, the output of the sigmoid function is close to 0. Through this module, we carry out channel weighting on the extracted features, so as to efficiently use the information extracted from the residual structure. The ECA module is shown in Fig. In the traditional convolutional recurrent neural network, the features extracted by the convolutional layer are directly transported to the recurrent layer for time-scale learning, however, the time–frequency features of different channels after convolution may have different influence on the localization performance. We need to learn the weight distribution for different channels, amplify the useful time–frequency features and attenuate the useless time–frequency features. So, we introduce an Efficient Channel Attention (ECA) module before the recurrent layerTo validate the performance of our proposed network architecture, we set up two experiments. In the first experiment, we compare the proposed method with other model on a public dataset. In the second experiment, in order to find the best features and explore the effect of the model in a real environment, we train the neural network using a real dataset and compare its localization performance using different features.30. Convolutional recurrent neural network baseline architecture is very popular in the field of sound source localization. It usually consist of three 2D convolutional layers, two bidirectional LSTM or bidirectional GRU recurrent layers, and fully connected layers20. We compared SSL-RC with the baseline architecture and other models. At the same time, in order to clarify the influence of channel attention mechanism and the number of residual structures on the performance, the influence of the existence of channel attention mechanism and the number of residual blocks on the localization accuracy and error is compared. The input features used in the model are the combination of log-Mel spectrogram and GCC-PHAT. The number of Fourier transform points is set to 1024, the window length is 1024, the overlap rate is set to 50%, and the window function is a Hanning window. The model is trained using the Adam optimizer. The early stop method is used during training, and the process will be stopped if no improvement in validation loss is observed within 30 epochs. We calculated the percentage of correct prediction angles of the model when the error range is 5°, 10° and 15° respectively, that is, the ratio of the number of correct predictions within the corresponding angle range to the total number of samples in the test set. The mean localization error of the model on the test set was also calculated. The results are shown in Table We conducted a comparative experiment using a publicly available SSL datasetThe results in Table Table 32. However, synthetic data cannot fully reflect the real environment. We want to use data from real environments for SSL studies. Most SSL algorithms rely on the signal collected by the microphone array. We use a microphone array to acquire audio signals from a real environment, hope to get a dataset for SSL studies. The acquisition process is shown in Fig. The input feature selection experiments use audio signals captured by microphone arrays as training data to explore the localization accuracy of the proposed model with multiple input features and find the best-performing features. Deep learning is a data-driven technology, and it usually need a large amount of data to train model. Currently in the field of deep learning SSL, supervised learning is still the dominant approach, which requires a certain amount of labeled data. Most of the current studies use synthetic data to develop neural network models6. The Fourier transform parameter settings in feature extraction are consistent with the first part of the experiment. First, we compare the accuracy of each input feature in terms of azimuthal classification, i.e., the ratio of the number of correct classifications to the total number of samples. Figure By synthesizing the analysis of existing literature, we extracted seven input features such as multichannel phase spectrograms, multichannel amplitude spectrograms, log-Mel spectrogram, and GCC-PHAT that are applicable to microphone array audio signalsTo validate the performance of SSL-RC, we first compare it with the baseline architecture and other improved models on a publicly available dataset, and the results show that our model outperforms the other models in terms of localization accuracy and mean error. In the first experiment, the 3Conv-ECA and 3Conv-Res models show the improvement effect in localization accuracy and mean error, demonstrate that residual structure and attention mechanism are helpful to the improvement of SSL performance. In the second experiment, the classification accuracy of the model under different features is investigated. The experimental results show that the combination of features of log-Mel spectrogram and GCC-PHAT outperforms other features, and its average accuracy is 0.76–19.46% higher than the rest of features, as shown in Table This paper presents a SSL model based on residual network and channel attention module. The input features are extracted by the residual network, and then the channels are weighted by the attention module, so that the model can use the time–frequency information more effectively. In order to illustrate the reliability of the proposed model, we compared the proposed model with the popular baseline architecture based on convolutional recurrent neural network and other improved models using the public dataset. Our model shows the best performance in terms of localization accuracy and error. Meanwhile we use the audio signals collected by microphone array in a real environment to study the performance of the model with different input features. The experimental results show that the combination of features of log-Mel spectrogram and GCC-PHAT get the best performance.


With the gradual development of and improvement in earthquake early warning systems (EEWS), more accurate real-time seismic intensity measurements (IMs) methods are needed to assess the impact range of earthquake intensities. Although traditional point source warning systems have made some progress in terms of predicting earthquake source parameters, they are still inadequate at assessing the accuracy of IMs predictions. In this paper, we aim to explore the current state of the field by reviewing real-time seismic IMs methods. First, we analyze different views on the ultimate earthquake magnitude and rupture initiation behavior. Then, we summarize the progress of IMs predictions as they relate to regional and field warnings. The applications of finite faults and simulated seismic wave fields in IMs predictions are analyzed. Finally, the methods used to evaluate IMs are discussed in terms of the accuracy of the IMs measured by different algorithms and the cost of alerts. The trend of IMs prediction methods in real time is diversified, and the integration of various types of warning algorithms and of various configurations of seismic station equipment in an integrated earthquake warning network is an important development trend for future EEWS construction. Seismic intensity measurements (IMs) serve as indicators of the spatiotemporal variation in seismic intensity when an earthquake occurs. Seismic networks are used to detect earthquakes and quickly estimate the source location and magnitude at the data center . Subsequently, based on the estimated source parameters, the ground motion model (GMM) is used to calculate the IMs field to describe the distribution of the IMs field, which is centered at the epicenter. After an earthquake occurs, the real-time estimation of the IMs field enables individuals to rapidly assess the impact of the earthquake, and the intensity of the impacted area can be categorized according to the IMs strength. Individuals who receive warning information can evaluate the potential severity of the consequences of their decisions according to the intensity of the earthquake at their location, and thereby, they can implement more effective measures. This also furnishes a crucial framework for subsequent rescue operations and disaster evaluation.With the development of earthquake warning technologies and methods, scholars have gradually deepened their understanding of earthquakes and have refined earthquake early warning systems (EEWSs), which has led to better earthquake warnings. This has resulted in operational EEWSs around the world that have been tested or implemented ,2,3. TheP wave information from the near-source stations, and then the distribution of the IMs at different source distances is obtained by using the GMM, and this information is used to alert individuals of the potential damage zone of the earthquake. The regional warning system relies on abundant station information to provide a more accurate estimate of the earthquake source (ES) parameters. However, this system typically requires a longer processing time and cannot provide timely warnings in areas close to the epicenter. Conversely, the on-site warning system uses the initial part of the P wave waveforms observed by a single or few adjacent stations to predict subsequent IMs values at the same site. Therefore, the on-site warning system is commonly deployed for critical targets, with surrounding stations used to predict the intensity of the impending seismic waves. Due to its straightforward deployment, it can provide early warnings to the area around the epicenter [EEWSs can be mainly classified into two types of warning systems based on the alert methods used: regional warning systems and on-site warning systems. In the network-based EEWSs, the regional warning occurs after the basic earthquake parameters are obtained from the initial few-second picenter .P wave information in the initial few seconds has some drawbacks. Large earthquakes can cause saturation problems with magnitude estimates. Moreover, the GM along the direction of the fault rupture (FR) will be stronger. The longer the FR, the stronger the GM at shorter distances from the fault. Regarding the 2011 M 9.0 earthquake off the Pacific coast of Tohoku, the predicted earthquake intensity in the Kanto area of Tokyo was lower than the actual observed intensity. The reason for this may be that the amplitude–magnitude saturation of seismic events with Mw > 8 that reach the upper limit of seismometer measurements cause the magnitude to be underestimated. In addition, the inversion of the ES process indicated that the fault extended into the offshore Ibaraki Prefecture near Kanto, which led to an underprediction of the intensity [The core infrastructure of a regional EEWS can be categorized into four modules: earthquake event detection and localization, estimation of magnitude, forecast of peak ground motion (GM) at the target location and issuing alert notifications . Among tntensity ,18. Thisntensity . It alsontensity . Variousntensity ,23,24.In this paper, we will focus on the real-time prediction methods of IMs for EEWSs in recent years and track the latest advancements. Firstly, we will investigate whether the ultimate magnitude scale can be predicted based on the initial rupture behavior of the earthquake, which is in line with various theories supported by EEWS methods. We then summarize the progress of regional and on-site warnings that are based on IMs predictions. The methods developed in the last decade are then summarized in terms of finite faults (FF) and simulated seismic wave fields, and the development and characteristics of these different algorithms are described in detail. Finally, the accuracy and timeliness of different algorithms are discussed, and user tolerance and warning cost are considered to evaluate the EEWS algorithm performance.The question of whether large and small earthquakes exhibit different characteristics during early rupture is central to understanding the evolution of rupture over time. For point source EEWSs that rely on information from the first few seconds of source rupture to rapidly estimate the source parameters, the ability to quickly and accurately calculate magnitude is crucial. In the past decade, researchers have made efforts to investigate whether the initial rupture is deterministic, but the results remain inconclusive. P wave is proportional to the final magnitude of the earthquake, which allows for an estimate of the magnitude before the completion of the rupture process [P wave peak displacement evolves differently with time for different earthquakes in the early stages of the rupture process. On the other hand, Rydelek and Horiuchi [Pd) of a progressively longer time window and assuming a time-dependent saturation of the linear relationship between log10 (Pd) and magnitude (Mw). The results indicated a universal growth pattern in the evolution of the Pd with time after the initial rupture of the fault, which was inconsistent with the deterministic model of seismic rupture.The concept of deterministic assumption refers to the variation in the final magnitude of earthquakes due to different nucleation processes, and the waveform of the initial few seconds can predict the final magnitude. Some scholars have held that the frequency or amplitude of the initial  process ,35. Colo process  discoverHoriuchi  questionHoriuchi  analyzedHoriuchi  compiledMw 7.0 to 8.5. The results indicated that the average seismic moment rate was significantly different in the first 10 s, whereby it was much smaller than it was during the duration of the earthquake. This suggests that a self-similar slip pulse is formed soon after the rupture starts. So, purely deterministic rupture mechanisms may be ruled out. Nonetheless, in some cases, weak or probabilistic forms of deterministic assumption can still be observed through a detailed analysis of seismic or geodetic data. Goldberg et al. [In recent years, the focus of earthquake research has shifted from deterministic rupture nucleation to a concept known as rupture weak determinism, which concerns the ability to infer the final earthquake size after the nucleation. Melgar and Hayes  proposedg et al.  used seig et al.  observedg et al.  suggesteP wave signals forms the basis of deterministic assumption. When an earthquake starts, most of the rupture sliding of the fault is limited to a small area near the ES. Seismic energy is released in a brief period, and small-to-moderate earthquake events align with this assumption. However, predicting the final magnitude is difficult when the rupture extends hundreds of kilometers across the fault before it is complete, as the time available to reach the P wave window to predict the final magnitude is limited. The EM estimate is likely not yet stabilized after the arrival of the S wave at the station. Furthermore, high-pass filtering is carried out for strong motion records to minimize long-period drift during velocity and displacement integration, which diminishes the low-frequency content in seismic records [Mw 7.0, the EM prediction suffers from a saturation of the magnitude estimate [Most of the regional warning methods based on seismic networks use point source algorithms (PSAs), which treat an earthquake as a point source, calculate the source parameters of the earthquake and predict future IMs based on the GMM. The implicit assumption of these EM estimates that are based on  records . Hence, estimate . AnotherP wave. Wu and Zhao [R), magnitude (M) and peak displacement (Pd) based on the Pd of the initial P wave in the first 3 s. The period algorithm was first proposed by Nakamura [τp of the seismic wave. While the single parameter-based magnitude estimation had a high uncertainty, the multiparameter combination method combined multiple parameters for magnitude estimation, which effectively improved the accuracy of the magnitude prediction. Huang et al. [τc, Pd was more effective than the single τc method and could be used as a new EM estimation method. The virtual seismologist method proposed by Cua and Heaton [P wave feature parameters from waveforms. Zhang et al. [When it comes to regional warnings, to predict the IMs parameters, the seismic magnitude needs to be calculated first. The traditional methods for EM estimation are mainly carried out by using amplitude algorithms, period algorithms, multiparameter combination methods and Bayesian-based methods. The amplitude algorithm estimates the magnitude according to the amplitude parameter within a few seconds of the initial and Zhao  establisNakamura , who argg et al.  found thd Heaton  was an Ed Heaton  used a dg et al.  proposedGMMs are mathematical models that predict how the intensity of IMs varies with earthquake size, distance and other factors, and they play an important role in earthquake engineering and seismic damage assessments. When researching the attenuation models of IMs, models generally consider the effects of three aspects: source characteristics, propagation medium and site conditions. The source characteristics cover the magnitude, fault type and plate location of the earthquake; the propagation medium focuses on the geometric dispersion and energy dissipation and absorption of seismic waves, which are usually expressed by the epicenter distance and fault distance; and the site conditions focus on the influence of the site type such as bedrock or soil fields on the IMs. In areas with abundant earthquake observational records, attenuation relationships are empirical formulas obtained from statistically analyzing strong motion observation records. A very well-known GMMs project is the Next Generation Attenuation (NGA) project led by the Pacific Earthquake Engineering Research Center (PEER), which shows the trends of the next generation of attenuation relationships in the global digital network with abundant strong motion data . The firP wave observed by seismic stations. ElarmS estimates the EM by using data from at least the first four stations of the seismic wave arrival network; it takes the two parameters τc and Pd of the initial P wave and then predicts the IMs values by using the GMM. This information is later integrated and updated in real time to generate IM prediction maps [τp, Pd, Pv and Pa are within the set range to exclude any abnormally small or large amplitude detriggers. A “range post-trigger” parameter (R) is introduced to ensure that the signal is not a single pulse or a rapidly shifted nonseismic signal. Finally, the horizontal-to-vertical amplitude ratio is checked to prevent S wave triggers from entering the system [Among the EEWSs currently in operation, the ShakeAlert warning system in the U.S. is a typical regional warning system. The ElarmS algorithms in the ShakeAlert system calculate the seismic position, size and other source parameters by using the limited number of parameters extracted from the first few seconds of the initial ion maps ,49. The ion maps . Since Ee system . After ce system .P wave observed by the seismic station. Compared with the regional warning method, the on-site warning method has fewer stations and has a relatively lower accuracy when estimating the source parameters. However, on-site warning only needs to predict future IMs values at the current location without considering the source parameters, which bypasses the uncertainty of source parameter estimation and the uncertainty of predicting the IMs field by using the GMM. This can provide a longer warning time for the area around the epicenter. Most of the current on-site warning systems use the Pd and τc of the initial P wave to estimate the IMs parameters. For example, the on-site algorithm in the ShakeAlert system provides warnings to locations up to 30 km away, and it provides 6 s of warning time to locations 50 km away [The on-site warning system predicts IMs based on the characteristic parameters of the initial  km away . The Ita km away .Pd and τc mentioned above, other characteristic parameters of the P wave information, such as the squared velocity integral (IV2) and cumulative absolute velocity (CAV), can also be used for on-site warnings. Wurman et al. [Pv and Pd to estimate the magnitude. Odaka et al. [P wave amplitude. Festa et al. [In addition to the n et al.  used Pv a et al.  proposeda et al.  proposeda et al.  investiga et al.  proposedP wave warning parameters to quickly estimate the IM parameters is an important issue in on-site warning research. The GMM that considers P wave characteristic parameters and IM parameters is constructed in the on-site early warning system so that the earthquake warning information is issued in time when the set threshold value is reached.Determining how to use the initial P wave amplitude, frequency, IV2 and other related parameters can be used with the IM parameters to construct a GMM. For each of the P wave warning parameters, the basic GMM can be expressed asP wave parameters . A and B are the coefficients to be obtained by the fit. For the threshold warning process, the implementation steps include . Using these parameters, the GM damage potential and range in the vicinity of the station could be quickly estimated. In contrast to seismic network monitoring, the threshold warning method utilizes single or multiple stations in proximity to the ES region for monitoring. An alert is promptly issued to the epicentral area upon detecting an earthquake that exceeds the predetermined threshold. This approach circumvents the uncertainties associated with predicting IMs by using the GMM. The practical applicability of the threshold warning method has been demonstrated worldwide, such as in China [τc and Pd, in magnitude estimation and proposed a threshold evolutionary magnitude estimation method based on τc and Pd. They recommended utilizing Pd within a range of 10 km to predict events in the larger magnitude range and suggested the joint use of τc to reduce the underestimation error for larger earthquakes.Caruso et al.  employedin China , Japan [in China  and Italin China . All of in China  analyzedP wave information. By extracting multiple P wave initial characteristics, we can establish more relationships between the P wave parameters and IM parameters or magnitude, which leads to more accurate IMs predictions [P wave over time, joint Bayesian estimates of the EM, ES and station distance are produced. Additionally, the correlation between the initial P wave parameters and the IM parameters is explored. Peng et al. [P wave parameters and IM parameters and thereby improved the accuracy of on-site warning systems. Wang and Zhao [P wave estimation of IMs using the Wenchuan database. They selected eight characteristic parameter attributes of the P wave in a 1 to 3 s time window and established relationships with four IM parameters, and then they subsequently analyzed the accuracy of their estimation.An additional approach to constructing GMMs is multiparameter joint prediction, which predicts IMs by making full use of dictions . Meier edictions  proposedg et al.  used filand Zhao  investigP wave warning parameters to construct a simple GMM for fitting is no longer necessary. Multiple P wave feature parameters can be extracted from the raw waveform data by using machine learning to construct an artificial-intelligence-based GMM to establish a more complex relationship between the feature parameters and the IMs. Hsu et al. [P wave feature parameters from the first few seconds of an earthquake at a single station and used support vector regression methods to build a regression model to predict the PGA based on these features. Song et al. [P wave. Chiang et al. [P wave and the strong motion is used to predict whether the GPA of subsequent waves exceeds a predetermined threshold. To reduce the complexity of multiparameter calculations, Hsu et al. [P wave signals and used an artificial neural network algorithm with the introduction of different Site Characteristic Parameters for PGA prediction.Artificial intelligence techniques provide another new research idea for on-site warning systems; using one or two u et al.  extracteg et al.  used theg et al.  proposedu et al.  selectedP wave information. Hsu and Huang [P wave data of the first 3 s of an earthquake detected at a single station by using the CNN approach. Jozinović et al. [P wave of the stations of neighboring earthquakes, a CNN model was used to predict the intensity of earthquakes at more distant stations, which could provide an estimate of the IM within 15–20 s after the occurrence of an earthquake. Compared with the GMM method, the CNN outperformed the GMM in terms of the residuals of the data. The CNN required no ES parameters, the receiver was located at the epicenter and its uncertainty was lower than that of the GMM. However, the weak interpretability of machine learning models can lead to false positives or false negatives, which makes it difficult to assess incorrect predictions [Moreover, artificial intelligence techniques are used to predict the IMs of the target location directly from the observed nd Huang  performeć et al.  introducdictions . The datP wave frequency or amplitude in relation to the EM will saturate [Treating the rupture as a point source in a large earthquake, the saturate . The FF saturate . Therefosaturate . The EM b based on the maximum value of the GPA recorded by the dense seismic network. The interpolation matrix b is classified for near and far stations to generate the binary image f. In the classification, if the PGA of the interpolated grid point is greater than the threshold GPAtr, the grid point is assigned a value of 1, whereas others are 0, as shown in Equation (3). Then, f is matched with the near-field template to determine the best position of the template in the data:Böse et al.  proposedRjb), a template library g between different fault distances and the corresponding PGA is established for large earthquakes above Mw 6.0. Böse et al. [Rjb should be judged based on the EM and PGA thresholds, as in Equation (4):R is the fault distance (km) and GPAtr is the PGA threshold (m/s2). The fault geometry parameters are estimated in real-time by matching the binary image f with a set of near-field templates g in the wave number domain to find the minimum error and calculate the current position, length (L) and strike (θ) of the rupture  are predicted from the observed IM parameters by the seismic networks. For earthquakes with a long duration and large rupture extent, and for multiple simultaneous earthquakes, wavefield methods are well suited to solve problems regarding accuracy, whereas traditional methods for estimating ES parameters have limitations in these aspects. Hoshiba et al.  pointed dictions . Table 2t − Δt and the previous wavefield distribution. The real-time estimation of IMs is achieved by minimizing the difference between the prediction of the current situation and the actual current observation by comparing the observed wavefield at moment t with the wavefield predicted at moment t. Radiative transfer theory predicts future wave fields by simulating wave propagation without source parameters when the spatial and temporal distribution of the waves is known. Compared with the Kirchhoff–Fresnel integral theorem, it is less computationally intensive and more computationally time efficient. However, the method requires a dense observation network, and the accuracy of the wavefield estimation decreases when fewer stations are available. Ogiso et al. [To address the shortcomings of the traditional way of predicting IMs by using the PSA, Hoshiba  proposedo et al.  improvedMw 9.0 earthquake and the 2016 Kumamoto earthquake in Japan showed that the PLUM method delivered alerts faster than conventional methods and could provide early warnings for areas closer to the epicenter. Kodera et al. [Kodera et al.  streamlia et al.  discussea et al. .The PLUM method consists of two parts: GM prediction and event construction. The former predicts the seismic intensity of all target sites by using the PLUM method. The latter handles the release, identification and termination of seismic events depending on the observed real-time seismic intensities. For real-time GM predictions, the seismic intensity at the target site is given by Equation (6):I  represents the seismic intensity, r is the location of the predicted target point, r1 is the location of the observation point, v0 is the velocity at r1 and θ is the propagation direction. F0 denotes the amplification factor. When the distance between the observation point and the prediction point is much smaller than its distance from the ES, the equation can be simplified to Equation (7), shown in i and k represent the location, k, i, CR denotes the area centered at the prediction point with a circle of radius R and F0 is the magnification factor, which indicates the seismic intensity difference. The predicted seismic intensity at the target location is taken as the maximum value of the observed real-time seismic intensity in a circular area of radius R, and the site effect is also taken into account. When the radius R is small, the wave can propagate without attenuation in the region. The early time window (TW) can be defined as Equation (8):Vs is the S wave velocity and ΔSDt is the system delay. To ensure the accuracy of the intensity prediction, the radius R is required to be small. In the ideal case, the system delay is considered to be zero. When the maximum GM propagates to the station, the TW that can be received at the edge station of radius R depends on the size of the S wave velocity and radius R. The maximum TW is about 10 s (R is 30 km and S wave velocity is 3 km/s).Here, R of an earthquake. Therefore, the TW is short regardless of the distance from the source of the earthquake. To extend the TW, the radius (R) must be expanded. However, since waves demonstrate undamped propagation, if all the areas within the radius (R) are predicted to experience the maximum observed intensity, the expansion of the radius will increase the uncertainty of the intensity prediction and may overestimate the intensity. To solve the problem of overestimating the intensities of earthquakes around stations and to improve the accuracy and timeliness of the PLUM method for early warnings, an improved PLUM method was proposed by Kagawa [Mw 6.6 magnitude earthquake that occurred in Tottori Prefecture, Japan, in 2016. This method improved the accuracy of the intensity estimation by introducing an attenuation factor in the wavefield. The maximum value of the intensity estimates at grid points within one kilometer of the earthquake was also considered for the comparison with the observed value.The PLUM method must be employed when centered on a station, and it is used to warn individuals in an area with radius y Kagawa  after anR) of the reference station and find the orientation of the station with the lowest intensity to obtain the wave attenuation direction. Using different types of local distance attenuation models, the most suitable GMM model is selected to predict the intensity within the range (R). The study showed that the PLUM method and ALPHA can complement each other and can provide a longer TW for near- and far-earthquake regions, respectively [Another way to improve the accuracy of the intensity estimation is to construct a suitable attenuation model based on the observations of the actual distance attenuation relationship between the current wavefield and predicted intensity. Kodera  developeectively .Grid definition: each station is connected to its six neighboring stations regardless of spacing. When a station monitors motion above the threshold, it sends its maximum predicted value to the surrounding six stations;Intensity modification: the intensity Dual station triggering algorithm: when both adjacent stations trigger the threshold, the station whose maximum value of ground vibration is triggered satisfies the primary threshold, and the adjacent triggered station meets the secondary threshold to solve the noise-spike false-alarm problem.The advantages of the PLUM method are that the algorithm is simple, can accurately predict the intensity distribution from faults with large rupture scales, has a better reliability than the PSA for forecasting complex earthquake sequences, and does not underestimate the intensity of earthquakes. The PLUM algorithm combined with the conventional algorithm can improve the TW and make up for the insufficient prediction of the PSA. Cochran et al.  were theThe test results showed that the PLUM algorithm solved the problem of no alerts being sent in areas near the ES and could operate independently, which serves as a complement to the ShakeAlert system, and it also performed well when warning individuals of the California earthquake .To solve the issues regarding the applicability of the PLUM method in different regions, further research is needed on the schemes and threshold methods that are suitable for local conditions. Minson et al.  exploredThe goal of an EEWS is to provide information about IMs  at the user’s location to mitigate earthquake damage. With EEWSs, the methods used to obtain IMs can be divided into two categories: one is calculating the source parameters by using the PSA and then obtaining the IMs distribution map by using the GMM, and the other method is based on FF or simulated seismic wave fields to provide the predicted IMs values in a region. Therefore, there are two methods used for the evaluation of IMs, point source assessments and IMs accuracy assessments . TraditiThe key parameters for the general evaluation method of the point source algorithm are magnitude, epicenter location, origin time and alarm time. To test the algorithm, historical earthquakes are retrospectively analyzed, and the predicted source parameters are compared to the true values in the earthquake catalog. The aim is to determine whether the errors regarding the magnitude and epicenter location are acceptable and whether there is sufficient warning time between the start time and when strong motion reaches the target location. For example, ShakeAlert’s point source code evaluation scheme sets five parameters, i.e., the goodness of the magnitude (Mg); goodness of the epicenter (Eg); goodness of the origin time (Og); goodness of the alert time (Tg); and a combination of Mg, Eg, Og and Tg into an alert assessment (Ag) to evaluate the performance of the point source algorithm .Point source assessment methods are not applicable when using the FinDer method to determine the real-time inversion of FF lengths or when using the PLUM method to predict future GMs. However, the ultimate goal of all algorithms is to be able to accurately predict the intensity of the target location. However, an accurate IMs prediction requires not only the location and magnitude of the ES, but it also needs to have source faults, stress drops, path effects and local site characteristics taken into account. The accuracy assessment of IMs predictions can consider the effects of these factors and respond to the IMs prediction performance of the algorithm. By setting an IM parameter threshold, the method compares the IMs field distribution predicted by the algorithm with the real observed IMs field and triggers an alarm if the predicted IMs in the target area exceed this threshold. The “predicted” field is calculated by using the algorithm’s source parameters, and the observed field is derived from all available real observations of IMs. Finally, the alarms are classified into four categories depending on whether the alarm is triggered in the target area or not . Meier  classifiThe relationship between the TW and threshold is also important in the evaluation of the algorithm, where higher thresholds result in shorter TWs and lower thresholds result in more false alarms. Meier  exploredm is the ratio of the number of missed and correct alarms, f represents the ratio of the number of false and correct alarms and r is the degree to which users tolerate false alerts. The CR is the ratio between the cost of damage that could have been prevented by the user receiving the alert and the cost of loss caused by taking action. A higher CR represents a more effective warning system. A CR value of 1 means no loss, and a value less than 0 indicates that the cost of alerting exceeds the cost of possible losses. Kodera et al. [The accuracy assessment of IMs predictions can be considered in terms of the cost of alerts. The risk avoidance cost is incurred when an alert is issued, regardless of whether it is a correct alert or not. The EEWS is only meaningful if the cost of taking action is less than the potential cost of the losses. Additionally, the correctness of an alert is related to the setting of the intensity threshold. When the thresholds are high, the accuracy of the alerts is high, but high thresholds are not a good choice considering the need for the high stability of precision instruments and people’s perception of ground shaking. However, low thresholds can generate more unnecessary alarms and lead to higher alarm costs. To determine the optimal alerting strategy, Minson et al.  presentea et al.  analyzedReal-time IMs predictions are an important part of the EEWS implementation process. They can provide an effective TW for near-source areas as well as different types of intensity distribution maps such as the PGA and PGV. This information will help with the delineation of earthquake disaster areas, seismic hazard analyses and disaster relief guidance for early earthquake hazard mitigation. In the past, the early EEWS deployed only had the PSA and threshold warning methods, which could provide little information on the ES and limited the warning range to a small number of areas or important projects. Now, owing to the increasing construction of seismic networks, the methods and facilities of early warning systems are advancing. In the past decade, the FinDer algorithm and PLUM algorithm for FF have been tested in the United States, Japan and Europe and are considered to be promising, as they have achieved good results. These methods improve the richness of seismic information acquisition and increase the fault tolerance of seismic detection. Therefore, future EEWS construction system should be built with a variety of earthquake monitoring equipment and by using multiple EEWSs, which can collect earthquake information from different perspectives; as a result, the early warning information will be more abundant, timely and reliable. As part of the China Earthquake Early Warning and Intensity Velocity Project, an integrated infrastructure with velocity meters, accelerometers, intensity meters and GNSS stations will be built to complete the integration of different types of earthquake-monitoring networks. Multiple systems such as the integrated PSA, FF algorithm and predictions based on simulated seismic wave fields will be used to enrich the ES information and produce a more reliable early warning result . The earTo cope with seismic monitoring in remote areas and areas not covered by stations, building a low-cost dense station network is a better choice. These MEMS sensors are cost effective, have a low power consumption and are easy to install. Taiwan’s P-Alert can provide a variety of IMs maps, such as maps of the PGA, PGV and spectral acceleration. In addition to monitoring earthquakes, P-Alert sensors are also used for various seismological studies including fault directional effects and structural health monitoring . A studyML will have an important place in the future of EEWSs, whereby they will provide new solutions for signal identification. Seismic signals are susceptible to local impulse noise from natural or man-made sources, and it is not an easy task to distinguish the true seismic signal from all signals quickly and reliably. Based on the powerful pattern recognition and feature extraction capability of ML, it can be used to improve the accuracy and stability of earthquake early warnings by learning and training with large amounts of earthquake data. ML can also extract important parameters from the raw input data and establish relationships between IMs without considering complex geological conditions, site effects and other factors. This allows ML methods to be widely used in EEWSs . NeverthAfter decades of research, EEWSs have undergone significant development. Initially, threshold warning systems were developed, but their warning effect was limited. The PSA was then proposed, which provides a longer TW and formed the basis of current EEWSs in various countries. However, it treats faults as points and has high error rates. To solve this problem, the FF algorithm was introduced, which considers the directionality of faults and predicts the IMs more accurately. Despite these advances, the GMM used by both the PSA and FF algorithm result in large errors due to its empirical nature. To improve the accuracy of the IMs predictions, the physical-based wave-field simulation method was proposed, which incorporates the physical properties of wave propagation. The integrated application of multiple methods can improve seismic information, and the performance of each method needs to be evaluated to determine its accuracy and timeliness. This study found that the PSA can provide the longest TW for distant seismic areas, while the FF algorithm and PLUM method can provide early warnings near the epicenter with more accurate predictions of the IMs’ field distribution. In conclusion, a combination of multiple methods is needed to improve EEWSs, and the evaluation of the performance of each method is crucial to ensure the accuracy and timeliness of IMs predictions. The use of multiple seismic monitoring data and multiple real-time IMs prediction algorithms and models can provide more complete early warning information in different earthquake situations and can help individuals more accurately assess the degree of earthquake hazards and corresponding defensive measures.