Sign language is the main channel for hearing-impaired people to communicate with others. It is a visual language that conveys highly structured components of manual and non-manual parameters such that it needs a lot of effort to master by hearing people. Sign language recognition aims to facilitate this mastering difficulty and bridge the communication gap between hearing-impaired people and others. This study presents an efficient architecture for sign language recognition based on a convolutional graph neural network (GCN). The presented architecture consists of a few separable 3DGCN layers, which are enhanced by a spatial attention mechanism. The limited number of layers in the proposed architecture enables it to avoid the common over-smoothing problem in deep graph neural networks. Furthermore, the attention mechanism enhances the spatial context representation of the gestures. The proposed architecture is evaluated on different datasets and shows outstanding results. Sign language is the main communication form in deaf communities. It conveys the complex semantics of language through highly structured manual and non-manual components. The manual component is encoded in the configuration of hand fingers and the relative movements of the hands to the signer body. The non-manual components on the other hand are encoded in the micro-motions of the facial expressions and head postures .There are two types of sign language gestures: static and dynamic. Static gestures mainly refer to the fingerspelling of an individual alphabet and numerical characters, which involve a fixed fingers’ configuration and fixed hand location. Dynamic gestures on the other hand might be isolated words or continuous sentences . This woThe sign language recognition systems follow one of two approaches: contact-based systems and contactless-based systems. In the contact-based systems, the gesture data are collected as a signal that represents the body movements or a sequence of body poses represented by the skeletal joints’ locations ,4. The cAnother main drawback for the contact-based systems is that the user should be accustomed to some hardware setup to collect the gestures, which makes it inconvenient and uncomfortable. This restriction is avoided in contactless-based systems. The most common form of systems in this category is the vision-based systems, which utilize imaging devices such as RGB and RGB-D cameras to record the gestures .The current advancement in deep learning techniques achieves promising performance not only in vision-based sign language recognition but also in many other vision-based fields such as object detection, image classification, and action recognition ,7. It is also applied in other fields such as human–robot interaction and smart healthcare systems. For instance, an incremental learning based on deep CNN is applied for human-like kinematic mapping on the robot and achieved interesting online prediction . On the Scarcity of large, labeled sign language datasets.The representation learning of signs by conventional CNN is inefficient. It involves a lot of non-relevant patterns in the scene, while the relevant patterns are only in the movement field of the signer’s hands.A few years ago, researchers started utilizing graph neural networks for sign language recognition. They have been encouraged by the success achieved by deep architectures such as Openpose in human pose estimation . These aEffective embedding for sign language gesture features with attention-based context enhancement for the spatial representation.Ability to avoid the representation over smoothing via minimizing the number of messages passing between the graph nodes.Optimum computation cost by reducing the number of layers as well as utilizing an efficient skeletal estimator.A novel graph-based architecture of separable 3DCNN layers is proposed for sign language recognition, with the following characteristics:The performance of the proposed architecture is demonstrated on different datasets with highly varied characteristics.To address the mentioned issues of data scarcity, the high complexity, and the inefficiency of conventional CNN representation, a shallow graph-based 3DCNN architecture with a spatial attention mechanism is proposed in this work for sign language recognition. The main contributions in this work are as follows:The proposed architecture is based on the raw RGB videos without extra preprocessing, a need for colored gloves, or any costly and complex setup for gesture data recording. It only relies on the efficient estimation of input skeletal data, which leads to a robust graph representation. This representation is not only immunized against the bad effects of noisy and cluttered background, light variation, and signer clothing style but also able to avoid all non-relevant patterns in the scene.The rest of this paper is organized as follows. The vision-based sign language recognition addresses the problem in three phases: extracting the visual patterns locally in each frame, temporal dependence modeling, and classification. The oldest works utilized various hand-crafted feature extraction techniques such as SIFT ,13, HOG Like other computer vision applications, most of the recent works exploited the advancement in deep CNN architectures for learning holistic features from each frame. Recurrent neural networks, on the other hand, were utilized for modeling the temporal dependence between consecutive frames. For example, Okan et al. applied inception architecture on both RGB frames and optical flow for sign language recognition . Chen etRecently, Qin et al. presented a system of dual-channel video transformer network (VTN) with bidirectional LSTM for continuous sign language recognition . The twoThe main drawback of conventional CNN-based architectures is that they holistically involve the scene. This might not be a problem in applications such as action recognition, where most of the patterns in the scene are relevant. Unfortunately, this is not the case in sign language recognition, as most of the patterns in the scene are non-relevant. A few hand gesture recognition-based works overcame the challenge of non-relevant patterns indirectly, but these works still have other limitations. In one of the most recent studies, a sentence-based recognition system is presented in . This syIn another recent study, LSTM and multisensory 3D data of operator’s fingers are utilized in developing a human–robot interaction prototype for surgical robot control . The inpOne of the most promising solutions for this challenge is graph neural networks, in which the representative patterns in the scene can be selected easily. For instance, Spatial–Temporal Graph Convolutional Network (ST-GCN) is the basis for many graph-based systems for sign language recognition . It utilThe Skeleton Aware multi-stream sign language recognition framework is one of the most recent graph-based systems for sign language recognition ,33. ThesOther recent work employed a video transformer network (VTN) for sign language recognition . VTN is Most of the graph-based architectures in the literature are so deep that they perform a large number of message passing operations, which involves a large number of feature aggregations and updating. This repetition for nodes’ feature aggregation leads at a certain point to the well-known phenomenon of feature over smoothing . In suchThere are other numerous works on fusion-based approaches ,38. SingThe proposed architecture is evaluated on the KSU-SSL dataset and four other benchmark datasets for sign language recognition. These datasets vary in terms of scene complexity, number of classes, number of samples per class, and the average length of videos.The collection of the KSU-SSL dataset is still in progress. It is collected by the Center of smart robotic research at King Saud University. It consists of 293 classes from the daily life sign language gestures in the Kingdom of Saudi Arabia (KSA). The KSA sign languages have several dialects; however, we choose the one which is recommended by the Saudi Association for Hearing Impairment. It can be noted that some deaf people sometimes do not adhere to a specific dialect. In the database, we do not include such deaf people. Most of the gestures were selected from the medical field for their importance and high demand by deaf people. This dataset is recorded in a well-controlled environment with suitable lighting conditions and uniform background color. The recording studio setup utilizes three frontal imaging devices: a high-speed RGB camera, an IR camera, and a mobile camera. The recorded gestures are performed by deaf, hard-of-hearing, sign language experts, and non-deaf people. The recorded gestures are also verified by sign language experts. Each participant is invited to perform each gesture five times. The last repetition is recorded with colored hands. Until the time of conducting this study, 30 participants have completely or partially recorded the dataset gestures. In this study, only the RGB camera videos without colored hands are involved. Sample frames from the KSU-SSL dataset are illustrated in The other datasets used for evaluation are the AUTSL dataset , the ArgTo build an efficient graph-based sign language recognition system, on the one hand, we proposed a lightweight 3DGCN with a low number of trainable parameters for representation learning. On the other hand, we utilized MediaPipe, which is the most efficient human landmarks estimator to extract the required graph nodes for recognition. Furthermore, other techniques such as multi-head self-attention and frame nodes’ partitioning were also utilized to boost the learning efficiency.MediaPipe is a recent framework presented by Google. It offers cross-platform, machine learning solutions for streaming media. It enables the live perception of human pose, hand tracking, and face landmarks on mobile devices and in real time . Each soTo build the sign graph, we only selected the most relevant 25 landmarks so that we can obtain an excellent recognition rate while maintaining a minimum computation complexity. From the 21 landmarks illustrated in The output of this step is an undirected spatial–temporal graph Moreover, the number of selected frames from each video sample was controlled by a predefined window size. For simplicity, the value of the window size was set based on the average number of frames per video in the dataset.The proposed architecture for representation learning consists of five consecutive layers of separable 3DGCN. As illustrated in A. The self-connections of nodes are also represented by an identity matrix I; hence, the spatial convolution was implemented as in Equation (1).The implementation of the convolution operation on graph data is not straightforward, as it is implemented on image data, which is in the form of a regular grid. Similar to most of the work in the literature, we investigate an implementation such as the one presented in . Within A multi-head self-attention layer was added after each spatial convolution operation to enhance the nodes’ context representation in each frame. To achieve that, the adjacency matrix was scaled by a matrix of normalized attention scores of the same size. Each element in the attention matrix encoded a pair-wise normalized attention score between two neighbors. Accordingly, the new spatial embedding 1.The node features were transformed through a 2D spatial convolution as in Equation (2).2.i and j, i and j on the entire batch.Unnormalized attention scores were computed between each pair of neighboring nodes as in Equation (3).3.The attention scores were then normalized via SoftMax as in Equation (4). The normalized attention matrix was then used to scale the adjacency matrix.4.Finally, the output of the convolution was multiplied by the scaled adjacency matrix to form the new embedding of the nodes as in Equation (5).As in the basic approach, the temporal dependence between consecutive frames was modeled by another 2D convolution operation along the temporal dimension of the output tensors. The output layer utilized the categorical cross-entropy function for loss optimization.Furthermore, we investigated three partitioning strategies suggested by  for moreThe uni-labeling partitioning: All the neighboring nodes of a corresponding root node are considered as a single set, including the root node itself. The representations of all nodes are transformed by a single learnable kernel.The distance partitioning: The nodes are partitioned into two subsets based on their distance from the root node. The first subset includes the root node with a distance d = 0, and the second subset includes the remaining nodes with d = 1. Two different learnable kernels are utilized to transform the nodes’ representation in the two subsets.The spatial partitioning: The neighboring nodes are partitioned into three subsets based on their distance from the root node and the gravity center of the whole skeleton as follows: (1) the root node itself; (2) the centripetal: the set of nodes that are closer to the gravity center of the skeleton than the root node; and (3) the centrifugal: the set of nodes that are closer to the root node than the gravity center of the skeleton. In this work, the nose point was set as a reference point instead of the center of gravity.The proposed architecture in this work was implemented using Pytorch and the training was conducted on NVIDIA RTX 3090 24 GB GPU. The recognition accuracy, which is the percentage of recognition rate, is used as an evaluation metric in our experiments. It is defined as:To demonstrate the performance of the lightweight basic 3DGCN architecture, the evaluation was conducted on the KSU-SSL dataset and the other four datasets. Most of these datasets are publicly available with training and validation splits but not the test split. Hence, the reported results in this part are in terms of recognition accuracy on the validation data. The number of samples in both training and validation splits used in this work is summarized in A fixed configuration was used to conduct the training, where mini-batch gradient descent was utilized with a batch size of 32 samples and an adapted learning rate. The initial value of the learning rate was set to 0.1 with an updating step size of 40 epochs. The learning rate was decayed after each step size according to Equation (7), which enables a smoother fine tuning for the trainable parameters with the advancement in training time.In each experiment, the architecture was trained for 200 epochs. After each epoch, if the architecture achieved better performance, the parameter values were saved so that the final model is the one that achieved the best performance regardless of the number of training epochs. The performance of the architecture on different datasets is illustrated in Even though the basic architecture is very light and has a small number of trainable parameters, This experiment was started by optimizing the most important hyperparameters of the partitioning strategy and the number of self-attention heads on the KSU-SSL dataset. We conducted a grid search to optimize these two hyperparameters. The search space was defined as follows:The partitioning strategy The number of self-attention heads The result of this grid search step in terms of recognition accuracy (%) is illustrated in After that, the optimized architecture was evaluated on the other datasets. The results are illustrated in Furthermore, it is noticed that the performance enhancement of the architecture on the LSA-64 is very slight, which might be attributed to the fact that LSA-64 was recorded under very restricted conditions regarding the background homogeneity, and there was light consistency in addition to using colored gloves. These restricted conditions enabled even conventional CNN models to achieve high accuracies on this dataset, as the relevant patterns are easily distinguished from the non-relevant background. In such a situation, the contribution of the spatial attention layer is minimal. On the other hand, the performance of the enhanced architecture was worse on both versions of the ASLLVD dataset. This bad performance can be attributed to the size of the training data. As shown in The proposed architecture was compared with the state-of-the-art graph-based architecture on both AUTSL and ASLLVD datasets. In Similarly, Even though the proposed architecture achieved encouraging performance, it still has some limitations. The most important limitation of this architecture is the presumption that only the sign performer should be in the scene. The performance degrades dramatically when multiple persons appear in the scene. This limitation is common in all human skeletal-based systems. For that reason, the performance of such systems in action recognition is very limited on the Kinetics dataset as in . The assA lightweight 3DGCN architecture is proposed in this study for sign language recognition. The proposed architecture utilizes a few 3DGCN layers to avoid the common over-smoothing effect in deep GCN architectures, which results from the high repetitions of messages passing between the graph nodes. This shallow architecture is utilized to construct a graph representation from the most relevant MediaPipe landmarks of the signer body. This embedding step transfers the recognition problem to graph classification. Reducing the depth of the architecture might intuitively lead to less efficient representation, but we substitute that by enhancing the spatial representation with less computation cost. To achieve that, a spatial attention mechanism is added to the proposed architecture to enhance the modeling of the spatial patterns of gestures. The 3DGCN layers are decoupled into a spatial and temporal convolution, which are separated by a spatial multi-head self-attention layer. This added layer enhances the local representation in each frame rather than modeling the global dependence of the frames. The proposed architecture is evaluated on various datasets of different characteristics and compared to state-of-the-art architectures. Our experiments demonstrate the effectiveness of the spatial attention layer to enhance the performance of such a lightweight 3DGCN architecture. Moreover, the proposed architecture shows outstanding performance and generalizes well on various datasets. For future work, the attention mechanism will be utilized in different ways to enhance the temporal molding of the gestures toward a greater generalization of the architecture. The proposed architecture will also be prototyped and evaluated in a real-time sign language recognition scenario.


Stress resistance resources, such as social support and resilience, have been found to be important in promoting psychological well-being during the COVID-19 pandemic. However, most prior research studies have conceptualized stress resistance resource variables as having a mediating or moderating role. Cooper (2018) proposed a model of the relationship between stress and health which posits that coping resources are always present and not only invoked in the face of adversity. Thus, we hypothesize that coping resources are causally antecedent to stressors and influence well-being indirectly via the stressor. We focused specifically on school teachers due to them being at the frontlines of service provision during the pandemic. Teaching was already identified as a highly stressful profession prior to COVID-19 and disease containment measures placed additional strain on teachers who had to adapt to emergency remote teaching.The current study tests this hypothesis by examining the indirect effects of resilience on indices of psychological health via role stress.N = 355) were teachers who completed the Connor-Davidson Resilience Scale-10, the Role Stress Scale, the Satisfaction with Life Scale, the State-Trait Anxiety Inventory-Trait Scale, and the Center for Epidemiological Studies Depression Scale. An electronic version of the questionnaires was distributed to teachers via Facebook and to officials from the Department of Education, who assisted with distribution of the electronic link to the survey. Participants were mostly women (76.6%) and mean number of years in the teaching profession was 15.7.Participants (Structural equation modelling results demonstrated significant direct effects of resilience on life satisfaction, anxiety, and depression, which indicates that resilience is beneficial for psychological health even in the absence of stress. Resilience also had a significant indirect effect on indices of psychological well-being via role ambiguity but not role conflict.These findings have theoretical implications for the understanding of the role of resilience in promoting psychological health among educators. Practical implications include an empirical contribution to education policy and information that can inform interventions aimed to promote resilience among educators. In South Africa, as in the rest of the world, the COVID-19 pandemic has severely impacted schooling, leading to severe consequences for teachers and pupils. When the pandemic first hit South Africa in March 2020, the government swiftly introduced a 5-level lockdown system, in which alert level 5 is the most restrictive and alert level 1 the least restrictive. Level 5 features drastic lockdown measures intended to contain the spread of the virus, including complete school closures ) and depression  and negatively associated with life satisfaction . The coefficients can be regarded as medium effect sizes. This finding indicates that high levels of role ambiguity were associated with high levels of anxiety and depression and low levels of life satisfaction. Role conflict was positively associated with anxiety  and depression  but not associated with life satisfaction. In all instances the coefficients can be regarded as medium effect sizes. These findings indicate that high levels of role conflict were associated with high levels of depression and anxiety. Resilience was positively associated with life satisfaction  and negatively associated with anxiety  and depression . With the exception of the correlation between resilience and anxiety, the coefficients can be regarded as medium effect sizes. In the case of the association between resilience and anxiety, the coefficient can be regarded as a large effect size. Thus, high levels of resilience were associated with high levels of life satisfaction and low levels of anxiety and depression.Role ambiguity was positively associated with anxiety  and negatively associated with role ambiguity , anxiety , and depression . All of the regression coefficients in respect of the direct effects can be regarded as medium effects. These findings partially support Hypotheses 1–2 regarding the direct effects of resilience, but they do not support the part of the hypotheses related to the association between resilience and role conflict.The indices associated with the direct and indirect effects resulting from the model in Fig. β = 0.11, p = 0.001, 95% CI ), anxiety , and depression . These findings support Hypothesis 4.Regarding the indirect effects of resilience, Table The aim of this study was to explore the indirect effects of resilience on indices of psychological well-being via role stress. The majority of research on resilience during the pandemic has investigated its mediating role in the association between stress and burnout, stress and quality of life and social isolation and wellbeing –66. FurtConsistent with the Cooper model of the stress and health relationship, resilience indirectly impacted psychological well-being via role ambiguity. Thus, resilience was found to be antecedent to role ambiguity. As explained by Cooper and Baglioni , this fiA further finding was that resilience did not impact psychological well-being via role conflict. It is probable that teachers in the study were able to rationalize that the COVID-19 pandemic placed additional demands on all frontline workers and this meant greater responsibilities. This finding is similar to that of a study of a U.S. disaster response workforce, in which role ambiguity was found to be a stronger predictor of psychological well-being  than role conflict . New andDe Clercq  has suggThe study has certain limitations. First, the use of an electronic survey may have led to response bias in that respondents who were more familiar with information communication technology may have been more inclined to respond than others. Second, the use of a cross-sectional design limits the conclusions that can be made about causal relationships. A longitudinal analysis is necessary to examine the causal processes that underlie the relationship between resilience, role stress, and indices of psychological distress. Third, the sample was mostly comprised of female participants from one geographical location, and this lack of heterogeneity may limit the generalizability of the results. Finally, the data was collected during the COVID-19 outbreak and it is probable that the results do not reflect the interactions between these variables in non-pandemic conditions.The current study examined the indirect effects of resilience on indices of psychological health via role stress among a sample of school teachers. Even when work-related stressors are absent resilience was found to be a protective factor that had a significant indirect effect on indices of psychological well-being via role ambiguity but not role conflict. Accordingly, policy makers and management teams should be proactive in reducing sources of role ambiguity for teachers. Management teams could foster resilience through knowledge sharing, provision of written documentation detailing roles, and promotion of awareness of the inter-relationships between job tasks.


Automatic detection of epileptic seizures is important in epilepsy control and treatment, and specific feature extraction assists in accurate detection. We developed a feature extraction method for seizure detection based on multi-site synchronous changes and an edge detection algorithm. We investigated five chronic temporal lobe epilepsy rats with 8- and 12-channel detection sites in the hippocampus and limbic system. Multi-site synchronous changes were selected as a specific feature and implemented as a seizure detection method. For preprocessing, we used magnitude-squared coherence maps and Canny edge detection algorithm to find the frequency band with the most significant change in synchronization and the important channel pairs. In detection, we used the maximal cross-correlation coefficient as an indicator of synchronization and the correlation coefficient curves’ average value and standard deviation as two detection features. The method achieved high performance, with an average 96.60% detection rate, 2.63/h false alarm rate, and 1.25 s detection delay. The experimental results show that synchronization is an appropriate feature for seizure detection. The magnitude-squared coherence map can assist in selecting a specific frequency band and channel pairs to enhance the detection result. We found that individuals have a specific frequency band that reflects the most significant synchronization changes, and our method can individually adjust parameters and has good detection performance. Epilepsy seizures are brain disorders predominantly characterized by recurrent and unpredictable interruptions in normal brain function , which aIn seizure detection algorithms, proper signal feature selection is the key factor in improving the efficiency of the detection result. Many features of single-channel seizure detection methods have been widely used, such as line length , amplituAnother method to observe the synchronization change between signals in the frequency domain is called magnitude-squared coherence (MSC): a normalized cross-spectral density function that measures the strength of association and relative linearity between two stationary stochastic processes on a scale from zero to one . In freqIn this study, based on the existing MSC methods, we extended the MSC method to the time-frequency domain by drawing an MSC map to detect the synchronization change of two local field potential (LFP) signal channels. We used Canny edge detection algorithm to obtain information regarding the frequency band and when synchronization increases. We subsequently selected the frequency band where the synchronization change was the most apparent and calculated the maximal cross-correlation coefficient curve of the selected channel pairs after filtering. We used the average value (AVG) of correlation coefficient curves as the feature to detect seizure onset and added standard deviation (STD) as a parameter to reduce false alarms. This study focused on synchronization changes in the limbic system in a chronic spontaneous temporal lobe epilepsy (TLE) animal model. We found that individuals have a specific frequency band that reflects the most significant synchronization changes. Compared with other seizure detection features, the results showed that our features can individually adjust parameters and has good detection performance.All surgical and experimental procedures were approved by the Animal Care and Use Committee of Zhejiang University. Pilocarpine-induced chronic TLE rat model was adopted in this study since this model closely resembles TLE in human. The procedure of inducing a chronic spontaneous TLE model is similar to our previous work . Six aduOne week after drug induction, the rats in which spontaneous seizures were observed were selected for electrode implantation. All the rats were implanted with electrodes in the bilateral subiculum  The original image is smoothed. The Gaussian filter is usually used, and in our study, the (2) Calculate the magnitude and direction of the gradient using the finite-difference of the first-order partial derivative. The x direction first-order partial derivative:y direction first-order partial derivative:The The magnitude of the gradient:The direction of the gradient:(3) Non-maximum suppression: each pixel, (4) A dual-threshold algorithm is used to detect and connect the edges. We set the high threshold value to 0.90 to remove most of the noise. The low threshold value is set to 0.4 times the high threshold value to add a connection between the edges.As shown in Under a uniform threshold, not all edges can be detected by the Canny edge detection algorithm in MSC maps, such as channels 11 and 12 in The calculations of MSC and Canny edge detection algorithm were complicated, and detection cannot be performed promptly. Therefore, the maximal cross-correlation coefficient was further applied in the detection step as a synchronization indicator to reduce computing resources and improve detection efficiency.m is the number of each function value appearing in the sequence, and n is the time lags between N is the length of signal sequence. The selected signals were passed with zero-phase digital filters, and the frequency range was decided upon in the preprocessing step. As in the construction of MSC map, we split the signals with a 400 ms window and an overlap of 200 ms. We subsequently calculated the maximal cross-correlation coefficient in each window and arranged the results in the time series to construct the cross-correlation curves. The maximal cross-correlation coefficient was calculated for every channel pair as follows:According to the Wiener–Khinchin theorem, the cross-correlation calculation results in the time domain can be transformed into the product form in the frequency domain:x and y. Therefore, the cross-correlation results can represent the MSC results.After calculating the maximal cross-correlation coefficient curves of selected channel pairs, we extracted the AVG of these maximal cross-correlation coefficient curves as a detection feature.The AVG value represents the overall enhancement of correlation, but is susceptible to common-mode noise. Therefore, it is necessary to add ten channel pairs with insignificant correlation enhancement, and calculate STD with the previously selected channel pairs. It is an important parameter to reduce false alarms.u is the average value of the AVG and STD curves, and This study aimed to find a stable way to identify the starting time point of seizure activity. Two parameters, AVG and STD, were introduced to characterize the threshold of the seizure start time point. We used the initial five seizure activities of each rat to calculate the initial threshold using the following formula:etection .x and y, defined as .  The x and y:x.One of the multi-channel features, mean phase coherence (MPC)  was usedfined as :(11)R feature ,41 withoThe data and statistical analyses were performed using MATLAB R2020b software . Continuous variables with normal distribution are presented as mean ± standard deviation; non-normal variables are reported as medians. All recorded seizure activities were reviewed and labeled by experienced neurologists. The seizure detection efficiency is evaluated by detection rate, detection delay, and false alarm rate. Detection rate is calculated as the rate of automatic detection number and total seizure number. Detection delay is calculated as the difference between automatically detected time and manually marked time. False alarm rate is calculated as the number of false positives per hour.The number of intercepted seizure segments of the five rats was shown in Despite the same drug-induced seizure procedure in all rats, the results of MSC maps showed that each rat had its seizure activity mode. Two typical results are shown in t and f represent the two cumulative dimensions of time and frequency in the MSC map, the time dimension starts from point For a single rat, not all channels showed an increased synchronization. We defined the quantified cross-correlation strength (QCCS) as the sum of all pixel values within 5 s before and after the edge detected in the MSC map:After the channel selection, we further determined the frequency band by intersecting the edge results of the selected channels. We calculated the frequency band every six days to ensure stability. The final frequency band selection is shown in For the selected channels, the stability is also evaluated every six days. The QCCS matrix is used for the evaluation, and the results are shown in A typical example of the detection results from Rat-1 is shown in The frequency band selection was sensitive to seizure detection. As shown in We used this method to detect all seizures in this study, and the results are shown in It should be noted that every feature has the possibility of optimization. This comparison only reflects this dataset’s results, which may be limited and incomprehensive.We also compared our method with other previous works, as is shown in Our study developed a method to find a suitable feature for seizure detection in TLE model rats. We used the MSC map and Canny edge detection to verify the synchronization change phenomenon in the seizure rats and find the strongest change frequency band. We used the maximal cross-correlation coefficient to indicate the synchronization change in a computationally efficient and appropriate manner. We extracted two features of the correlation coefficient curves and iterated the thresholds to obtain the available feature thresholds. The feature extraction method for seizure detection proposed in this paper has the following advantages: high detection rate, low detection delay, low false alarm rate. In the preprocessing step, the feature extraction method has individually adaptive but has a certain computational complexity. Therefore, in the detection step, the feature extraction results can be transformed into filtering and cross correlation calculation to reduce the computational complexity.Because it is difficult to build a Pilocarpine-induced chronic TLE rat model, we have limited data in this study. However, we can still get some interesting phenomena. One of them is the selection of the specific frequency band and channels. Although many studies have found synchronization changes in different frequency bands , some stFurthermore, detecting lesion sites and key nodes is also an important task in epilepsy research, especially in clinical treatment, where doctors use CT, MRI, PET , and othBased on the limitations of the current experiment, future research into the following is warranted:(1) Experimental verification of different species of organisms. The epilepsy characteristics of different species have similarities and differences. The synchronization of epilepsy in different individuals requires further investigation. Ultimately, the seizure detection method must to serve as an early warning and treatment for epilepsy. Our future studies will utilize the MSC map method to detect seizure activities in the epileptic brain of humans, confirming whether the method is effective.(2) Identification of the seizure sites. Our study focused on finding the onset of seizure activity, and some important channel pairs can be selected. However, our study also had a limitation. The ten important channel pairs are changed during the progression of epilepsy, but we only used the initially selected channel pairs to detect the seizure to reduce computational complexity. The results of this detection are not superior. The location of lesions has also not been studied in detail. We can determine when the seizure begins by which sites have a strong correlation, which may be related to the location of the lesion, but further study is warranted.(3) Dynamic analysis process. The MSC map can show the synchronization changes over the entire period, including the preictal, ictal, postictal, and interictal. It can help us to analyze the dynamic changes during seizures. Especially in recent years, the emergence of technicals such as sub-scalp EEG has made it possible to monitor patients’ seizures for a very long time . The metThis study developed a feature extraction method for seizure detection based on multi-site synchronous changes in TLE. We investigated five chronic TLE rats with 8- and 12-channel detection sites in the hippocampus and limbic system. We used the MSC map and Canny edge detection algorithm to determine the synchronization changes in the specific frequency band and channel. We developed this phenomenon into a detection method and tested the seizure data of five rats. The method achieved high performance, with an average 96.60% detection rate, 2.63/h false alarm rate, and 1.25 s detection delay. The experimental results show that synchronization is an appropriate feature for seizure detection. The MSC map and edge detection algorithm can assist in selecting a specific frequency band to enhance the detection result.


Endovascular interventional radiology (IR) is a minimally invasive procedure for the treatment of vascular diseases. This procedure requires physicians to be highly skilled at manipulating interventional devices under the guidance of two-dimensional X-ray imaging. By offering a non-error-sensitive and radiation-free environment, a virtual reality-based simulator provides a promising alternative for surgical skills training and surgery planning. Building a realistic and interactive simulator is a challenging task. To achieve better realism, this paper proposes a novel method of simulating the heartbeat for both standard and patient-specific anatomical data. A time-dependent offset field approach is proposed to efficiently and stably simulate the interactive behavior between the dynamic heart mesh and surgical devices. For medical imaging simulation, we propose a GPU-based linear depth subtraction method to approximate fluoroscopic images based on the attenuation of the X-ray. On this basis, a topology-based flow map method is proposed to simulate the propagation of the contrast medium in angiography. Experimental results show that the proposed algorithm can simulate heartbeat stably for meshes with varying geometrical shapes and complexities. In efficiency, the dynamic heart mesh can interact with surgical devices stably at 60 frames/s. Under the simulated fluoroscopic imaging effect, the injected contrast medium can realistically visualize both dynamic and static vessels. In a face validity by medical students and clinicians, the category of effectiveness score 8.35 out of 10 on average, demonstrating that our simulator is useful in surgical skills training and surgery planning. In recent decades, endovascular procedures have gained increasing popularity because of their advantages over traditional open surgeries, such as minimum invasiveness, short hospital stays, and fewer complications. However, these procedures require physicians to be highly skilled at manipulating surgical devices under the guidance of two-dimensional X-ray imaging. In addition to achieving a complex understanding of three-dimensional anatomy from two-dimensional displays, physicians must spend years of practice acquiring good hand-eye coordination . TraditiInteractive virtual reality-based simulators provide a promising solution that addresses all the above difficulties by offering a non-error-sensitive and radiation-free environment in which physicians can practice surgical skills repeatedly –5. ReseaBuilding a realistic interventional simulator for surgical skills training and preoperative rehearsal is a challenging task. Realism, real-time interaction, and the capability of processing patient-specific data are the essential features of such a simulator.Heartbeat is the vital sign of a live person. Most simulators  proposedThe collision detection and response between the dynamic heart mesh and surgical devices is another challenging task . Unlike The realism of medical imaging simulation is another criterion used to evaluate an IR simulator. Existing methods , 16 use In this paper, a simplified heartbeat model is proposed to simulate real-time heartbeat for patient-specific data. The interactive behavior of surgical devices and dynamic heart meshes is simulated stably by the proposed time-dependent offset field approach. A linear depth subtraction algorithm is proposed and implemented in the GPU for the real-time simulation of X-ray imaging. Based on the vascular meshes’ topological adjacency information, a blood flow map is generated to simulate the propagation of the contrast medium under virtual fluoroscopy. Experimental results demonstrate that the heartbeat can be simulated realistically and efficiently for heart models reconstructed from patient-specific data and that the simulated surgical devices interact with the dynamic heart meshes stably in real-time. Face validity by medical students and physicians indicates the feasibility of the proposed simulator in surgical skills training and procedure rehearsal.Our contributions can be summarized as follows:1)We propose an efficient and robust heartbeat simulation method for standard and patient-specific anatomical models to achieve higher fidelity in the IR simulator.2)We propose a time-dependent offset field to simulate the interactive behavior of the dynamic heart mesh and surgical devices.3)We propose a linear depth-based approach to simulate fluoroscopy in the GPU and a topology-based flow map method to simulate the propagation of contrast medium in DSA.Much work on the development of virtual reality-based simulators has been done since Anderson et al.  first prIn addition to simulators proposed by research institutes and universities, there are some commercial interventional simulators . MenticeMost existing simulators use static polygonal meshes or hierarchical centerlines to represent the vasculature, limiting the realism of the simulation. Although some works , 11 use For medical imaging simulation, the volume rendering-based method proposed by Muniyandi et al.  approximIn contrast to Dawson et al.  and TalbAs illustrated in A collection of clinical cases specialized for training certain types of interventional surgical skills are built. The case files include surgical devices, anatomical models, and optionally patient CT and/or magnetic resonance image (MRI). The simulator imports the digital resources of the selected training case to initiate the simulation. Note that, the CT/MRI images help trainees understand the anatomy’s morphological features for educational purposes. These images are not necessary for the surgery simulation itself.Instead of being imported directly from prebuilt training case files, the anatomical models used for patient-specific procedure rehearsal must be reconstructed from the patient’s CT/MRI image data. The binary masks for the anatomy of interest are first segmented from the input raw images. Then, the segmentation results are used to reconstruct the anatomical triangular meshes. Before being imported for procedure simulation, the reconstructed meshes must be optimized to gain better performance. After selecting the proper surgical devices, physicians can start the patient-specific procedure simulation to rehearse the upcoming surgery.In this section, we first describe the acquisition of anatomical models for patient-specific data . Then, the methods for heartbeat simulation (see section “Heartbeat simulation”), surgical device simulation , and fluoroscopic simulation (see Section “Fluoroscopy simulation”) are described in sequence.The triangular meshes used in our simulator are generated from patient CT images. The process uses two consecutive steps: image segmentation and 3D reconstruction. As a complex task in medical analysis, medical segmentation takes original CT/MRI images and outputs volumetric binary images for the anatomy of interest. Many traditional and deep learning-based methods of medical image segmentation have been proposed over the last several decades . Based o3D reconstruction takes the volumetric binary images generated in the previous step and creates triangular meshes. Research on 3D reconstruction from medical images has been a popular subject for decades , 33. BasMeshes reconstructed from volumetric binary images cannot be directly used for rendering and physics simulation in our simulator. Optimizations including smoothing, simplification, non-manifold edge fixing, and hole filling must be performed before the simulation starts. First, mesh smoothing methods are used to remove the stair artifacts induced by reconstruction algorithms such as the marching cube. Second, the original reconstructed mesh usually consists of an excessive number of facets, which leads to extra computational cost with little improvement in rendering realism and simulation accuracy. Simplification methods are used to reduce the number of facets while preserving the meshes’ intrinsic geometrical details. Third, our DSA simulation algorithm (see Section “Fluoroscopy simulation”) requires the mesh geometry to be a non-manifold to achieve fast and accurate flow map generation. Therefore, all non-manifold edges introduced at the reconstruction stage need to be fixed. Last, our X-ray imaging simulation method assumes that all rendered meshes are closed. Meshes with holes will cause rendering artifacts because their linear depth (thickness) cannot be calculated correctly.As depicted in First, the reconstructed circulatory system is represented by one triangular mesh in which the heart and its biologically connected vessels  are also linked topologically. Compared with the magnitude of the heartbeat, the movement of most arterial vessel walls (pulse) is too small to be visually perceived. Therefore, only the heart and its nearest vessel meshes are used for heartbeat simulation, and the rest of the vasculature is considered static in this paper. With this approximation, the mesh for heartbeat simulation is disconnected from the remaining vascular mesh and used as input for the subsequent algorithm, as shown in The human heart consists of four chambers: two atria and two ventricles. The heart beats because of the contraction and relaxation of the heart muscle. The contraction makes the volume smaller, while the relaxation does the opposite. To approximate this phenomenon, an imaginary point is placed at the geometrical center of each heart chamber. A periodic force in the direction of this point is then exerted on the chamber mesh’s vertices to simulate the volume change of this chamber.Based on the work of Wang et al. , we use For the ventricle, the function is defined as:The force exerted on the vertex of the chamber with index i is defined as:ix is the vertex position, cx is the geometric center of the chamber, and k is the elastic coefficient. To obtain a smooth surface during deformation, we add two stretching constraints: the distance between adjacent vertices and the distance from a vertex’s current position to its initial position  demonstrate that the heartbeat can be simulated in real time for meshes within a limited number of vertices. Nevertheless, the simulation is computationally intensive, and its efficiency drops as the number of vertices increases. As another key component of a realistic IR simulator, the simulation of surgical devices also requires numerous computations. The finite CPU resources must be distributed properly to ensure the real-time efficiency of the whole simulation. The deformation of the heart repeats itself periodically. If the parameters of the simulation remain the same, it is not necessary to repeatedly calculate the shapes of the heart except in the first cardiac cycle. To achieve higher efficiency and reserve enough computational resources for the surgical device simulation, a time-dependent offset field is created to cache the simulation results of a complete cardiac cycle for rendering and subsequent simulation.S1, S2, …, SM) is sampled from the simulation results for the first complete cardiac cycle. The first sample S1 is selected as the base mesh, whose vertex position is treated as the baseline for offset. For the ith vertex jth sample mesh, the positional offset As illustrated in i and a normalized time parameter t ∈ . By adding this offset vector to the position for every vertex of the base mesh, we can calculate the shape of the deformed heart at any instant of a cardiac cycle at a low computational cost. Moreover, the calculation can be further optimized by uploading the offset field to the GPU and adding offset vectors in parallel for each vertex.As depicted in As mentioned above, the offset field remains invariant unless the parameters of the heartbeat simulation change. Hence, for a cardiac cycle, the heart meshes are calculated once, and the results are cached by creating a time-dependent offset field. The subsequent heartbeat simulation and visualization run completely in the GPU, reserving sufficient computational resources in the CPU for surgical device simulation.With this method, the heartbeat for any patient-specific data can be simulated realistically and efficiently. Note that, the magnitude and rate of the heartbeat can be changed by tuning the parameters to simulate the human body’s reaction to stimulation in surgical procedures.0p, 1p, 2p, …, np) that carry the discretized material properties .One of the most important tasks in building an IR simulator is simulating the interactive behavior between the circulatory system and surgical devices. As explained in section “Anatomical mesh acquisition,” we represent the circulatory system with triangular meshes that are suitable for rendering and collision detection. For different types of interventional devices, which in most cases are long, thin, and hollow, we model them as rigid rods linked by points (operties . Based oO(n3) time complexity such as ours, we propose a deviation-feedback approach to update the external forces exerted on the device centerline points as in  are usually manipulated in turn to reach certain places in the circulatory system. To represent nested interventional devices, we propose a shared-centerline model in which the centerline points are shared while their material properties are blended. With this model, the interaction between nested interventional devices is simulated by dynamically blending the material properties of the shared centerline points.With the incorporation of the heartbeat, the circulatory system now comprises static vascular meshes and a dynamic heart mesh. When they are advanced into the heart area, the simulated interventional devices must interact with the dynamic mesh realistically and stably in real time. The shape of an interventional device is maintained when it collides with vessel walls. It is observed that the devices tend to move along with periodically moving vessel walls instead of bouncing back and forth inside the vascular tunnels of the beating heart mesh. Utilizing the time-dependent offset field, we propose a visual-deviation method to simulate the synchronous movements of surgical devices and the beating heart. The method is explained in ip of the centerline. Thus far, the calculated device centerline is static. To make the devices move synchronously along with the beating heart, a time-dependent offset vector is added to every point of the centerline based on the progress of the heartbeat, as shown in When the simulation starts, the baseline mesh in the heartbeat simulation is used to perform collision detection and calculate the stable shape of the device centerline. Meanwhile, we can obtain the neighboring vertices of the heart mesh for each point ix is the stable position of point ip on the centerline. There are n vertices of the mesh S1 in the spatial neighborhood of ip, and their indices are Ci = {c1, c2cn}. The instantaneous offset vector jσ(t) for the heart mesh vertex with index j ∈ Ci at time t can be retrieved directly from the time-dependent offset field generated in the heartbeat simulation. The average offset vector for the vertices in iC is calculated and then added to ix to visually deviate the centerline to synchronize the movement of the devices and the heartbeat.For example, a and δb, as shown in c for the closest non-floating point is used directly, as shown in In the simulation, there are centerline points that are not directly in collision with vessel walls. Thus, no heart mesh vertices can be found in their close neighborhoods, and their offset vectors cannot be directly calculated from the offset field. For brevity, these points are referred to as ‘floating’ points. Floating points are usually located on a sharp curve at vessel bifurcations and at the tip of the device; they are circled with dotted lines in In summary, the baseline mesh for heartbeat simulation is used for collision detection and surgical device centerline calculation. Utilizing the time-dependent offset field, the surgical device centerline is visually deviated to synchronously move with the beating heart, resulting in realistic, efficient, and stable simulation results for the interactive behavior between surgical devices and the dynamic heart mesh.As an important part of IR procedures, fluoroscopy is a continuous X-ray imaging technique that shows the movement of a body part  or the course that a medical instrument or dye (contrast medium) takes as it travels through the body . In thissd, where s and d are the weight and linear thickness of the anatomy, respectively.Based on the physics of X-ray imaging , an X-raThe proposed X-ray imaging simulation method takes two render passes: one for creating the weighted thickness texture for all anatomical models and the other for creating fluoroscopic images.d of the rendered anatomy model is first calculated for each pixel by subtracting its front faces’ linear depth from that of its back faces. The formula is defined as d = zback−zfront, where frontz and backz are the z components of the pixel’s camera space position for the front and back faces, respectively. Then, the thickness is multiplied by the anatomy-specific weight value s. The weighted thicknesses for all rendered anatomies are summed, and the results are written to a rendered texture (target) in float format. In this way, the sum of the weighted thickness for any pixel can be read directly from the texture.In this pass, the linear thickness g for each pixel is calculated based on the attenuation formula, defined asIn this pass, the grayscale a is the uniform attenuation factor and ∑μ is the summed weighted thickness obtained from the texture created in the first render pass. By selecting proper values of the weight s for different anatomies, X-ray imaging can be simulated realistically at a very low cost.where Note that, the depth value in the camera space is used for the thickness calculation because the normalized depth value retrieved directly from the Z-buffer of the rendering pipeline is non-linear and suffers from precision problems. Because both the front and back faces are needed to calculate the thickness, all triangular meshes must be closed. Otherwise, there will be artifacts in the holes of the unclosed meshes.Unlike existing image-based methods, the proposed algorithm for X-ray imaging simulation takes only triangular meshes as input and runs in parallel at the pixel stage of the rendering pipeline, ensuring real-time efficiency. Moreover, the vertices of the heart mesh are offset for heartbeat rendering at the vertex stage, the output of which can be used directly as input for the pixel stage to simulate X-ray imaging. The coherency of the pipeline means that our method works for both static and dynamic meshes.Digital subtraction angiography is a fluoroscopic technique used extensively in IR for visualizing blood vessels. Contrast medium is injected into the vasculature through catheters, and the affected blood vessels are visually enhanced in dark gray. DSA is one of the most commonly used techniques in IR, and it is necessary to realistically simulate this procedure for building a simulator with high fidelity.The contrast medium propagates downstream with the blood flow from the tip of the angiography catheter. Only the vasculature that the contrast medium flows through can be visualized. In this paper, a gradient-based region growing method is proposed to simulate this process with triangular meshes. The idea is to find the submesh that is affected by the contrast medium and render it progressively under the fluoroscope to approximate the propagation of the contrast medium. This method can be divided into three steps.To create the downstream vascular mesh visualized by the contrast medium, the direction of the blood flow must be calculated everywhere in the circulatory system. In this paper, an attribute called a ‘sequence’ is introduced for the triangular facets. The gradient of the sequence approximates the direction of the blood flow nearby. After the sequences for all facets are calculated, a blood flow map can be built for the triangular mesh.d points in the same direction as the blood flow. By performing collision detection between P and the vascular mesh, we can obtain a looped triangle strip that intersects with the plane. Let us define this strip as S1 = {f0, f1, …, fr}, where if is the triangle’s index in the vascular mesh. In the direction of d, we can obtain S2, defined as {fr + 1, fr + 2, …, fs}. Based on the breadth-first search (BFS) algorithm, we can obtain a list of strips {S1, S2SN}, as shown in As shown in bx as the first point of the catheter centerline’s body part . The closest facet of the vascular mesh to the point bx is bf. The direction of the sequence gradient of facet bf is defined as bn. Starting from the seed plane defined by bx and bn, a list of triangle strips is collected by the region growing in the direction of the sequence gradient. By merging all these strips, we obtain an ordered collection of facets that defines the contrast-medium-affected submesh of the vascular model. The order of the facets approximates the propagation of the contrast medium.The contrast medium is injected into the vasculature and propagates with the blood flow to visualize the vessels downstream under fluoroscopy. After building the flow map of the blood in the first step, a sequence value is assigned to every facet of the vascular mesh. Let us define At the rendering stage, the sorted triangles are uploaded to the GPU and rendered incrementally over time. The number of rendered facets over time n(t) is defined as follows:t0 is defined as the moment when the contrast medium is injected and ν is the number of facets rendered per second. To simulate the contrast medium’s influence on X-rays, we use a higher value sangio to replace the vessel’s initial weight vs for thickness calculation when the submesh is being rendered. As shown in vs to sangio at t0. The weight value decreases over time to simulate the dilution of contrast medium in the blood.where The experiments described in this section were conducted in our homemade IR simulator which are composed of haptic feedback devices and a consumer-level laptop .The vascular models are reconstructed from real patients’ CT images. The interventional devices are the most commonly used types in daily interventional procedures. The intrinsic material properties of the devices are selected by trial and error offline. One unit for the vascular model in the coordinate system of the virtual world represents one millimeter in the real world.In this experiment, six different heart meshes are used as input to run the proposed heartbeat simulation algorithm. Meshes 1–4 are reconstructed from four different patients, while meshes 5 and 6 originate from the same patient but with different degrees of simplification . As showWith regard to efficiency, our algorithm can simulate the heartbeat at an interactive rate for meshes 1–4. The frame rate drops dramatically as the number of vertices increases from mesh 4 to 6. This is because the increase in the vertex number greatly slows the proposed algorithm by introducing additional constraints and the resulting computation. However, once the time-dependent offset field is built in the first cycle, the frame rate jumps to 60 frames per second (fps) for meshes 1–5. Even for mesh 6, the simulation achieves interactive rates. Because the heart mesh vertices are offset in parallel in the GPU, this method is less sensitive to a change in the number of vertices within a certain range. With this method, many of the computational resources of the CPU are saved for simulating the behavior of surgical devices, ensuring that the surgical procedure can be simulated in real time.In summary, our method can simulate the heartbeat stably and realistically. By tuning the parameters, we can change the heart rate and the magnitude of the heartbeat. The incorporation of the simulated heartbeat builds a dynamic circulatory system, achieving higher realism for our simulator with little CPU cost.The femoral artery is one of the most commonly used access points for physicians to start endovascular procedures. In this experiment, we navigate a catheter and guidewire from the femoral artery to three clinically typical target vessels to validate our device simulation algorithm. As shown in As shown in When navigated into the cardiac region, the simulated devices move along with the beating heart in synchronization. Three samples are taken at different moments of a cardiac cycle, as shown in In this experiment, a cardiac training case is imported into our simulator. The imported meshes contain a skeleton, vessel, and heart. The devices used in this procedure are a catheter and guidewire. The anatomy rendered in the Phong lighting model is displayed in s as explained in section “Fluoroscopy Simulation.”The skeleton, vessels, and heart mesh can be clearly seen in In this experiment, we simulate the coronary angiography procedure. First, we create the flow map for a complex vascular system that includes the aorta, the carotid artery, the renal artery, and the femoral artery. By placing the seed plane at the beginning of the aorta, we calculate the value of the sequence for all triangles that compose the arterial vessels.As shown in The catheter is then navigated to the entrance of the left coronary artery, and the contrast medium is injected through the catheter to visualize the coronary vessels. The screenshots shown in sangio, as explained in section “Fluoroscopy simulation.” The propagation of the contrast medium is progressive. It runs from the injection point to its downstream vessel branch in the direction of blood flow and dilutes in the blood over time. After the angiography catheter reaches the entrance of the left coronary artery, the contrast medium is injected through the catheter, as shown in In this experiment, we perform coronary angioplasty and stent placement for the heart. There are two blockages in the patient’s left coronary artery. In this procedure, we navigate the catheter and guidewire into the left coronary artery and implant two stents to open the narrowed vessels. Screenshots of key moments of the procedure are shown in In To validate the effectiveness of our simulator, 24 participants with varying degrees of expertise in IR are recruited to evaluate our simulator. A questionnaire is designed for participants to grade our simulator in six respects: the virtual anatomical environment, device behavior, visualization, efficiency, robustness, and effectiveness. The criteria for the scores are as follows: 0–2 means strongly disagree; 3–5 means disagree; 6–8 means agree, and 9–10 means strongly agree. The average score in each category is listed in the second column of In the first category, approximately 80% of the participants agree that the simulated heartbeat is realistic and its incorporation greatly increases the realism of our simulator. The other 20% of participants, who have more than 10 years of clinical experience, suggest that the heartbeat should not only be visually plausible but also be numerically accurate for further application in hydrodynamic analysis. Almost all participants give a high rating to the accuracy of our reconstructed vessels and skeletons after a thorough cross-reference with the original CT images.The device behavior in our simulator obtains a high rating, especially in the interaction with vessel walls, bifurcation, and narrow branches. The average score of the device behavior inside the beating heart is not as high as the other scores because experienced participants expect to see how the devices interact with the blood flow pumped out of the heart. However, our simulator does not incorporate real-time blood flow simulation, which is another very challenging task. Nevertheless, good ratings are received in the device behavior category.In the category of fluoroscopy simulation, more than 90% of the participants agreed that the results are realistic enough for training and surgery planning purposes. Approximately 5% of the participants suggest that more anatomical parts should be involved in visualization to enrich the simulated fluoroscopic images even if they might not be visually evident as skeletons and vessels affected by the contrast medium.All participants agree that our simulator runs quickly and that no operational lag is detected in the whole procedure. In addition, no vital errors that can corrupt the simulation are found. The numerical metrics in In summary, participants with less clinical experience give our simulator high rankings on average. They believe that our simulator is a very useful tool for training interventional skills because it is realistic, reusable, and free of radiation and ethical risks. In comparison, participants with more experience are more interested in the surgery planning capability of our simulator and set a much higher standard for us. Nevertheless, they agree that our simulator’s ability to process and import patient-specific data for simulation can offer much help in making surgery plans.In this paper, we propose a novel approach to simulating the heartbeat for building a realistic and interactive IR simulator. A simplified heartbeat model is used to formulate the driving force of the heartbeat. To achieve better efficiency, a time-dependent offset field is built to cache and reproduce the simulation results. The interaction between surgical devices and the beating heart is simulated realistically and stably with the proposed visual-deviation method. For X-ray imaging, a linear depth subtraction method is proposed to approximate the fluoroscopic images with an anatomy-specific weight value. For angiography simulation, a blood flow map and a gradient-based region growing method are used to generate the contrast-enhanced vascular meshes. A progressive rendering technique with a time-varying weight value is used to render these meshes for simulating the propagation of the contrast medium. To summarize, the proposed methods solve three challenging simulation problems which are the real-time heartbeat simulation for patient-specific data, the stable interaction between dynamical meshes and surgical devices, and the realistic medical visualization and angiography. Experimental results demonstrate that our approach is capable of realistically simulating the heartbeat and its interaction with surgical devices in real time. The simulation results of the angioplasty for the left coronary artery validate the realism and efficiency of the proposed methods for X-ray imaging and angiography simulation.Despite the increase in the realism of our simulator, there are still some problems to solve for achieving better fidelity and functionality. Blood flow simulation is one of the most essential but challenging tasks to be accomplished. The real-time simulation of the blood flow and its interaction with surgical devices will not only achieve higher fidelity for the simulator, but also provide a promising way to perform hydrodynamics analysis for procedures like aneurysm embolization. Computational modelling of the blood vessels , 43 can The original contributions presented in this study are included in the article/Ethical review and approval was not required for this study in accordance with the local legislation and institutional requirements.HW designed and conducted the experiments, analyzed the results, and wrote the manuscript. JW raised critical comments and edited the manuscript. Both authors contributed to the article approved the submitted version.


Severe tricuspid regurgitation (TR) is an undertreated common pathology associated with significant morbidity and mortality. Classically, surgical repair or valve replacement were the only therapeutic options and are associated with up to 10% postprocedural mortality. Transcatheter tricuspid valve interventions are a novel and effective therapeutic option for the treatment of significant TR. Several devices have been developed with different mechanisms of action. They are classified as annuloplasty devices, replacement devices, caval valve implantation and coaptation devices. In this review, we provide a step-by-step description of the procedural steps and techniques of every device along with video support. Significant tricuspid regurgitation (TR) has been associated with poor long-term outcomes that include but are not limited to right heart failure, end-organ damage such as liver failure, and mortality.6–11Significant TR presents with signs and symptoms of right-side heart failure. These include ascites, fatigue, peripheral oedema, painful hepatosplenomegaly and bloating.1The disease currently has limited treatment options that are challenging, themselves. The management of TR includes medical, surgical and percutaneous interventional treatments. Medical therapy for the management of severe TR is often limited and mainly targets symptom relief. Furthermore, surgical replacement or repair is associated with a high rate of complications and an in-hospital mortality reaching up to 10% especially in patients with previous left-sided heart valve surgery or previous tricuspid repair.13–16As such, a novel modality for the treatment of isolated TR was developed in the form of transcatheter tricuspid valve intervention (TTVI). TTVI is classified according to the mechanism of action as such: annuloplasty devices, replacement devices, caval valve implantation (CAVI) and coaptation devices.2. The tricuspid annulus (TA), on the other hand, is an asymmetric and dynamic saddle-shaped ellipsoid, which allows it to change with varying loading conditions. Its circumference is 12±1 cm and its area is 11±2 cm2.27A normal TV complex includes a fibrous annulus, usually three leaflets, papillary muscles, chordae tendinae, right ventricular myocardium and a right atrial myocardium.A detailed study of the TV anatomy, especially the location and number of supernumerary leaflets or scallops, is primordial for procedural planning and may affect intervention outcome.Patient selection for TTVI should be done by a multidisciplinary heart team to assess the chances of procedural success and clinical benefit. After maximising medical therapy, especially diuresis, the cause and mechanism of TR in addition to right ventricular function and size must be evaluated.It was demonstrated that TTVI is a safe and effective treatment option for patients with symptomatic or severe TR who are not suitable for surgery.32TTVIs might pose an increased risk of thrombosis since the right side of the heart has a low-pressure circulation. However, the optimal antithrombotic therapy following transcatheter TV interventions remains controversial.28A systematic and comprehensive literature search was performed electronically using PubMed, Web of Science, Embase, OVID and Cochrane Library. Inclusion criteria were set to retrieve English studies that involved adult patients who underwent transcatheter intervention for TR. Studies were included if they detailed the main procedural steps of the percutaneous intervention being performed. The electronic search captured all articles till the date of 15 January 2021. Abstract-only papers were excluded. The search identified studies based on the following search terms: (Tricuspid regurgitation) OR (TR) OR  OR  OR (tricuspid incompetence) OR  OR (tricuspid insufficiency) AND (Percutaneous intervention) OR (Transcatheter intervention) OR (Coaptation device) OR (Annuloplasty device) OR  OR (CAVI) OR  OR  OR  OR (Forma repair system) OR (TriClip) OR (MitraClip) OR  OR (TriCinch) OR (Cardioband) OR  OR (MIA) OR (PASCAL system) AND  NOT (Aortic) NOT (Bicuspid) NOT (Pulmonary).The electronic search of the databases and the hand search of the bibliographies yielded 803 articles. The two authors AM and VZ performed the title/abstract, full-text screening and data extraction. The discrepancies regarding the eligibility of the articles were resolved by a discussion among authors until a consensus was reached. After deduplication, 612 articles remained. Overall, 129 articles underwent full-text screening. The final count of articles included in the study was 17 studies.Data extraction was previously shared among authors to ensure consistency in pooling the data, captured information of the author, year of publication, the type of the study, the procedure being performed and the details of the procedural steps.The TriClip System is an edge-to-edge repair technique that was developed and based on the MitraClip device. The technique has been used in the treatment of mitral regurgitation. This, as a result, has gained the technique a lot of experience in the field.The TriClip System configuration consists of two parts: The Clip Delivery System (CDS) and the Steerable Guide Catheter (SGC).33Two methods using the TriClip System have been described: The Triple Orifice Technique, where the clips are placed centrally between the septal leaflet, anterior tricuspid leaflet, and the posterior tricuspid leaflet, and the bicuspidisation technique, where the clips are placed most of the time between the septal and anterior tricuspid leaflets. The bicuspidisation technique has been more feasible and done more frequently.33The TriClip procedure is performed under general anaesthesia and with transesophageal echography (TEE) and fluoroscopy guidance.34First, a right femoral venous access is obtained with a 24F sheath insertion. A double ProGlide preclosure is performed. Unfractionated heparin is administered to achieve an activated clotting time of 250–350 s34The next step is done under TEE bicaval view guidance and left anterior oblique (LAO) on Fluoroscopy. Unsheathe the device in the RA with the ± knob returned to the neutral position and the wire retracted into the device, such that a parallel orientation of the system with the septum is achieved. Next, flex the F/E knob to steer the tip of the device towards the TV. It is of utmost importance to have it orthogonal to the annulus of the valve.35Success of the procedure is assessed by fluoroscopy and TEE. It is defined by a reduction in the regurgitation by one or more grades without creating a valvular stenosis, in addition to the safe implantation of the clip without partial leaflet detachment or device migration.The Edwards Pascal transcatheter mitral valve repair system is an edge-to-edge repair technique adopted from mitral regurgitation (MR) repair for the treatment of TR.3817 33The PASCAL System is made up of an implant that has a 10 mm central nitinol woven spacer. It behaves like a filler in the regurgitant orifice of the atrioventricular (AV) valve, along with two paddles and clasps that attach to the valve leaflets.39The procedure is done either in a catheterisation laboratory or a hybrid operating room under general anaesthesia with fluoroscopic and TEE guidance.33The procedure starts by obtaining right femoral venous access and inserting a 22F guide sheath. The guide sheath is advanced all the way into the RA. After that, the introducer and wire are removed, along with air removal from the system. Then, the steerable and implant catheters are inserted into the guide sheath. They are advanced to the tip of the guide sheath until the implant is exposed in the RA. The implant clasps are closed, and the steerable guide flexed down by turning the knob so that the implant lies just above the TV in the RA. It is very important to ensure that the implant is perpendicular to the tricuspid plane. Then, the system is opened and advanced towards the ventricle to have one of the leaflets grasped by the first clasp on breath hold. When this is ensured, a gentle pull back of the implant is done to optimise the position of the first leaflet. The clasp is then closed and secured. The system is manipulated by the knobs along back-and-forth manoeuvres to swing the system to ensure a proper grasp over the second leaflet. Occasionally, both leaflets can be grasped simultaneously.33Finally, when both leaflets are closed, the residual TR grade and the transvalvular gradient are evaluated before the device is deployed.33 40The Edwards FORMA System is a combination of a spacer and a rail. The spacer is placed in the regurgitant orifice to reduce TR and to provide a surface for leaflet coaptation.4217The procedure is done under general anaesthesia with fluoroscopic and transesophageal echocardiography (TEE) guidance in a Cath lab or hybrid operating room. Venous access to the left subclavian is achieved by surgical cut-down or percutaneously from the axillary vein. A 24F sheath is inserted and advanced to the level of the left innominate vein/SVC junction to support the delivery catheter. Then, a right ventriculography is done to identify a target location that allows the device to be perpendicular to the valve plane as well as to allow all leaflets to coapt with the device. After that, vascular closure is obtained with ProGlide (Abbott vascular) sutures or surgically.41Through the sheath, a steerable delivery catheter is advanced and manipulated under TEE guidance, serves to station the anchor and rail in their correct position. The anchor target is on the lateral wall of the RV close to the interventricular septum and perpendicular to the annular plane to ensure optimal coaptation of the TV leaflets.41 43The rail is anchored to the RV with six curved prongs designed in a manner that prevents it from exiting into the pericardial space while still grasping the RV myocardium.41Finally, the length of the rail is trimmed, and its proximal end is sutured to the subcutaneous tissue of the deltopectoral groove to fixate the rail and the spacer. Venous closure is then achieved either percutaneously or surgically.41The Trialign device is a valve annuloplasty apparatus that is introduced transjugularly to reduce the tricuspid annular diameter via tissue plication.4445The goal of this procedure is to perform a transcatheter bicuspidisation of the TV. Two 14 French sheaths are placed in the ventral and lateral portions of the right internal jugular veins. In addition to that, a guide catheter via femoral access is placed in the right coronary artery.46A deflectable tricuspid guide catheter is advanced via jugular access to the RV. The latter is designed to aid in directing procedural devices across the TV orifice and towards the TA. A tricuspid wire delivery catheter is then advanced to access the ventricular side of the TA at the site of pledget delivery and then, with the aid of fluoroscopy and TEE, it is advanced to the septoposterior location. Then a crossing wire is introduced into the delivery catheter and advanced to penetrate through the annulus from the ventricular side to the atrial side by that providing a path for pledget delivery. The crossing wire is mounted with a radiofrequency energy source at the distal tip that is activated as the wire is moved through the tricuspid annular tissue. Echography is then used to confirm the position of the crossing wire.46Once the crossing wire is properly positioned in the septoposterior of the TA, an endovascular snare system is used to trap the distal tip of the crossing wise, which is retracted through the jugular access site. After snaring of the crossing wire, a tricuspid pledget deliver catheter is positioned in the RA facing the deflected tricuspid guide catheter positioned under the TA in the RV.46The pledget delivery catheter is moved through the TA into the tricuspid wire delivery catheter until it seats the distal portion of the pledget on the ventricle side of the TA. Next, the pledget delivery catheter is retracted through the annulus to the atrial side where it deploys the proximal portion of the pledged. The steps are then repeated on the opposite anatomic site of the anteroposterior commissure to position the second pledget.46A dedicated plication lock delivery catheter is advanced towards the annulus over the pledget sutures, which results in the plication of the annulus and the leaflet in between the two pledgets. A lock is then deployed onto the sutures. Finally, a suture cutter catheter is tracked over the sutures to lock the implant and cut the sutures above the lock, via a blade contained in the tip of the catheter, to allow the removal of the proximal suture parts.46The percutaneous TriCinch System is an annuloplasty device designed for the management of functional TR by means of a transfemoral fixation of a corkscrew into the anteroposterior TV annulus.47–49The system consists of a Dacron band connecting both the self-expanding stent, and a corkscrew anchor.49The procedure is planned based on cardiac CT to identify the optimal implantation site and is performed under general anaesthesia with the guidance of fluoroscopy and echocardiography  or intracardiac echocardiography).48 49Frist, a coronary wire may be placed as a marker in the right coronary artery.49Based on the baseline cardiac CT along with intraprocedural fluoroscopy, the implantation location is identified, and the stainless-steel corkscrew is fixed. The fixation is localised in the anteroposterior TV annulus.48The Cardioband Tricuspid System targets the management of patients with TR with annular dilation by annular reduction using an adjustable implant.50 51The Cardioband System consists of four main accessories which are: (1) the implant (a polyester sleeve with radiopaque markers that has a contraction wire running in the sleeve and connected to an adjustment spool which adjusts the length of the implanted device), (2) the TF Delivery System (TDS) , (3) the implantable mental anchors and anchor delivery shafts and finally (4) the size adjustment tool (SAT).50The procedure is similar to that of mitral valve treatment, with a major difference with the band clockwise implantation from the anteroseptal commissure to the septal annulus first part, after the coronary sinus.52First, transfemoral artery access is established so that right coronary artery angiography is performed, and a wire is placed to mark it and prevent any compression or kinking.48Then, transfemoral venous access is established and a 24F sheath is inserted. Through which a highly manoeuvrable TDS is advanced into the RA under TEE guidance and directed towards the TA. The implant is deployed and fixed by inserting up to 17 anchors on the atrial surface of the annulus.50After deploying the anchors, the implant catheter is exchanged for the SAT, which is advanced over the contraction wire to reshape of the annulus. Then, the band is clinched, by that reducing the septolateral TA and anteroposterior TA diameters.52Transcatheter CAVI has been used to treat severe TR.54 55Patients undergo screening for anatomic suitability via a computed tomographic angiogram. This studies the anatomy and diameter of the device landing zones in the SVC and IVC at the cavoatrial junction. Right heart catheterisation is done to confirm severe TR and caval backflow. Patients with severe pulmonary hypertension and TAPSE less than 10 are excluded. The procedure is performed under general anaesthesia along with fluoroscopic, transesophageal echo and haemodynamic guidance.There is a growing experience in the off-label usage of the Edwards Sapien XT and Sapien three valves for treating severe TR. Both were previously commercially used for the treatment of aortic stenosis.49First, the right femoral vein is punctured and a 6F sheath is placed in it. A stiff wire is advanced to the SVC, and then a 16F or 20F sheath is advanced to reach a level just below the diaphragm.53The TricValve System constitutes two self-expandable valves designed specifically for SVC and IVC implantation in the low-pressure circulation. The SVC valve is a belly-shaped tapered device designed to anchor in a dilated, tapered SVC configuration.53All procedures are fluoroscopy and transesophageal echocardiography guided.53To implant the valve, access is obtained via the right femoral, left femoral vein and left basilic veins. Then, an Amplatz wire with a long soft J-tip is inserted and advanced through the right femoral vein to the RA and into the right internal jugular vein. To note that both devices are loaded into 27F catheters and undergo sheathless implantation.53Then, the SVC valve is advanced and deployed with the landing zone of the enlarged midportion of the stent  above the marked right pulmonary artery. The upper edge of the valve should avoid the occluding the brachiocephalic vein, which can be marked by a multipurpose wire. The lower edge of the valve should be at the atrium–SVC junction with a small protrusion into the RA tolerated. Then, deploy the IVC valve. Hepatic vein angiography and Lunderuist guidewire can be used to identify the hepatic vasculature and the RA, respectively. With the upper, skirt-lined segment of the stent protruding into the RA, and the device fully anchored in the IVC, the IVC valve is deployed as well.The transatrial intrapericardial tricuspids annuloplasty TRAIPTA is a concept where after establishing venous access, pericardial access is obtained by puncturing the right atrial appendage. This technique allows the delivery of an implant along the AV groove within the pericardial space that exerts compressive forces over the annulus. Interactive adjustment of the implant controls its tension and helps modifying and reducing the TR. The right atrial puncture is sealed at the end of the procedure with a nitinol device. While the procedure has been performed in pigs, human testing with a new device is under development. Limitations of such a procedure include the need for a free pericardial space, while the risk of coronary artery compression remains a risk.56–58The Millipede System, a complete repositionable and retrievable ring, can be implanted both surgically and via transcatheter, on the right atrial side, to restore the shape of the TV. While this technique has been used for the management of mitral valve pathologies, the use of the Millipede System for TR is promising.58Transcatheter TV replacement whether after tricuspid annuloplasty  or a degenerated surgical bioprosthesis  is also under study. The NaviGate self-expanding tricuspid AV valved stent has been studied in a preclinical pig model and used in humans under the title of compassionate use. The device comes in different sizes and constitutes a ninitol tapered stent with three xerogeneric pericardial leaflets and annular winglets that allow for annular and leaflet fixation without protrusion of the stent into the surrounding chambers.58Finally, the percutaneous pledget-assisted suture tricuspid annuloplasty targets the reduction of the size of the TV orifice by approximating the septal and lateral tricuspid annuli. This is done via pledgeted sutures between the middle of the anterior and the posterior lead. This technique has proven feasible, and results in a reduction in the annular area along with TR in swine models. PASTA looks promising for patients with TR without other good treatment options.59TR is a highly prevalent disease that is associated with high mortality and morbidity when left untreated. Historically, surgical repair or replacement were the only therapeutic options; however, due to their high complication rate they were often deferred. TTVIs that have been showed to be feasible, associated with low mortality rates, and effective in reducing TR associated with alleviation of symptoms.