Tongue cracks refer to fissures with different depth and shapes on the tongue’s surface, which can characterize the pathological characteristics of spleen and stomach. Tongue cracks are of great significance to the objective study of tongue diagnosis. However, tongue cracks are small and complex, existing methods are difficult to extract them effectively. In order to achieve more accurate extraction and identification of tongue crack, this paper proposes to apply a deep learning network based on image segmentation  to extract and identify tongue crack. In addition, we have studied the quantitative description of tongue crack features. Firstly, the pre-processed tongue crack samples were amplified by using adding salt and pepper noise, changing the contrast and horizontal mirroring; secondly, the annotation tool Crack-Tongue was used to label tongue crack; thirdly, the tongue crack extraction model was trained by using SBDL; fourthly, the cracks on the tongue surface were detected and located by the segmentation network, and then the output and features of the segmentation network were put into the decision network for the classification of crack tongue images; finally, the tongue crack segmentation and identification results were quantitatively evaluated. The experimental results showed that the tongue crack extraction and recognition results based on SBDL were better than Mask Region-based Convolutional Neural Network (Mask R-CNN), DeeplabV3+, U-Net, UNet++ and Semantic Segmentation with Adversarial Learning (SegAN). This method effectively solved the inaccurate tongue crack extraction caused by the tongue crack’s color being close to the surrounding tongue coating’s color. This method can achieve better tongue crack extraction and recognition results on a small tongue crack data set and provides a new idea for tongue crack recognition, which is of practical value for tongue diagnosis objectification. Tongue cracks can effectively reflect some diseases and can be further diagnosed in combination with other tongue features3. As shown in Fig. 4, so it is difficult to extract them effectively.In recent years, machine vision has been developed with the development of computer software and hardware, and the combination of medical diagnosis and computer vision has gradually become a hot topic. Researchers combined tongue diagnosis with image processing and machine learning in order to achieve the objectification and modernization of tongue diagnosis. This method reduced the interference of environmental factors on the tongue image and avoid doctors’ inaccurate judgment due to subjective factors, thereby achieving the effect of objective tongue diagnosis and improving the accuracy of judgment. At present, researchers have made considerable progress, but there are still some problems that are difficult to overcome. The research on tongue image mainly focuses on the tongue coating and tongue color5 used U-Net and an adaptive threshold technique to extract tongue cracks within post-processing. Liu et al.6 attempted to extract tongue cracks by using a method based on the wide line detector which extracts the whole of the line by employing an isotropic nonlinear filter. Li et al.7 proposed a new method using statistic feature extracted by wide line, such as Max-distance, to train a binary SVM as a classifier for cracked tongue. And now more deep learning methods are used for crack defection based on classification, pixel segmentation and object detection10. Weng et al.11 proposed to train the tooth-mark and crack detection model by using tongue images annotated bounding-box. It was a weakly supervised method that added several classification branches to recognize the tooth-marked tongue and cracked tongue according to the YOLO object detection model. Peng et al.12 proposed a P-type neural network architecture based on a lightweight encoder-decoder structure which could get the detailed extraction result at pixel level. Xue et al.13 proposed to use cracked and non-cracked regions to train Alexnet to extract deep features of cracked regions. This method focuses on localized cracked regions and trains a multi-instance support vector machine (SVM) to make the final decision. However, the effectiveness and generalization of these methods still need to be improved.There are several studies on segmentation and recognition of tongue cracks, and early studies almost concentrated on the methods of threshold segmentation and line detection. The threshold method uses the grayscale difference or color difference between the tongue crack and other parts of the tongue surface to establish a threshold for segmentation, while the line detection method calculates the speed of change in the color or grayscale brightness of the edge of the tongue crack, so as to obtain the contour of the crack. Rhee14.The threshold segmentation method only considers the characteristics of a certain pixel, which doesn’t consider the deeper semantic features, so tongue cracks cannot be extracted well by traditional methods. Only obtaining the tongue crack pixel features on the tongue surface cannot describe the tongue crack well, it is necessary to dig deeper semantic features of tongue images. Therefore, accurate segmentation of tongue crack can be achieved by combining the shallow features with the deep features. The convolutional neural network can gradually extract the semantic features of pictures from shallow to deep and use these features as an important basis for classifying cracks and surrounding tongue. It has achieved good results in medical image processing and other fields. However, the color of the crack in the tongue image is similar as the surrounding area, and its main feature is that its shape is a zigzag strip of a certain width. In addition, not only are the crack widths of the same tongue image different, but also the width of the same crack may change as the crack end extend. So it is difficult for semantic segmentation algorithms to deal with the segmentation of the tongue crack. SBDL is a two-level special network with a segmentation network and a decision network, which achieved accurate results for the segmentation of small objects with fewer samplesTherefore, this paper applies a deep learning network SBDL to extract and identify tongue cracks. The rest of this article consists of five parts. Section "The flow chart of tongue crack extraction based on SBDL is shown in Fig. The tongue images used in this paper were provided by the Four Diagnostic Information Comprehensive Laboratory of Shanghai University of Traditional Chinese Medicine, including 176 images with tongue crack and 140 images without tongue crack. These tongue images are assigned to training samples and test samples in a ratio of 8:2, which means that there are 141 images with tongue crack and 112 images without tongue crack in training samples, and there are 35 images with tongue crack and 28 images without tongue crack in test samples. Since the tongue image dataset is too small, overfitting is easy to occur during training, which makes the training results more specific to a certain type of images and lacks sufficient generalization ability. Therefore, it is necessary to expand the number of tongue crack samples.15. Visually speaking, images subjected to salt and pepper noise will randomly produce small white or black dots.Adding salt and pepper noise. Salt and pepper noise, also known as impulse noise, is a common noise. After the image is subjected to this kind of noise, the gray value of the noise point is very different from the surrounding pixels. Usually, even if the image suffers very little salt and pepper noise, its details will be greatly destroyedChanging the contrast. There are many ways to change the contrast, which can be mainly divided into three types: image sharpening, smooth denoising and grayscale adjustment according to the processing purpose. This paper mainly uses grayscale adjustment to change the contrast.Horizontal mirroring. Horizontal mirroring refers to mirror swapping the left and right parts of the image around the vertical center axis of the image.As shown in Fig. After the above three image processing methods, the crack tongue data in training samples has been expanded from the original 141 to the existing 564. Therefore, there are 564 positive and 112 negative samples used as training samples; and there are 35 positive samples and 28 negative samples are used as test samples.The model is trained by the SBDL network, the tongue crack image and the ground truth of tongue crack, so the tongue cracks in tongue crack image need to be labeled with the relevant labeling tools.The self-developed Crack-Tongue labeling tool is used to label tongue crack. The tool uses the shortest path between the labeling points to label based on the magnetic lock sleeve, and obtains a polygon that can fit the contour of the tongue crack. The labeling result is shown in Fig. 14. It uses the public crack detection to train and test, which achieves very good results. It uses a small data set to train a crack detection network model with small amount of calculation and high precision. Considering the surface crack detection problem as a binary image segmentation problem, pixel-level semantic segmentation can firstly be achieved through a semantic segmentation network, and then the segmentation results can be used as the input of the next decision stage.Deep learning networks have two important factors that determine the speed of computation: the amount of training data and the computation amount of network parameters and floating-point numbers. Usually, in order to improve the accuracy of the training model, deep learning needs to use a large number of data samples to perform deeper feature learning under the adjustment of multiple network parameters. In order to improve the training speed and reduce the amount of calculation, the SBDL network divides the network into two stages: semantic segmentation stage and decision stageThe first is the segmentation network, which is mainly used to detect the tongue surface for cracks, generate a mask, and locate the location of the tongue crack. The segmentation network consists of 11 convolutional layers and 3 max-pooling layers. Each max pooling layer reduces the size of the output feature map by half. The convolutional layer consists of a BN layer and a nonlinear ReLU layer. The BN layer adjusts the output to the range of the standard normal distribution. The ReLU layer is the activation function layer used to adjust the output value. Both BN and ReLU layers have the effect of increasing the convergence speed. The method of dropout is not used in the network structure, because the weight-sharing convolutional layers have already provided sufficient regularization in the case of small training set. In order to capture small cracks in large-resolution tongue images, the network uses a 15 × 15 convolutional layer, which greatly increases the receptive field and uses a pooling layer instead of a convolutional layer to reduce the size of the feature map, thereby preserving feature information as much as possible.Then is the decision network which uses the output of segmentation network for the classification of cracked tongue images. Before entering decision network, the output of segmentation network will be superimposed with the last feature of the output layer, and both of them will be used as the input of decision network at the same time. This part of the network uses three 5 × 5 convolutional layers and three pooling layers as the method of convolution and downsampling, so that the network can capture the local shape of the image and observe a large range of shapes. After the last 5 × 5 convolution, a 32-channel feature is formed. Decision network also has a part of additional network, which performs the maximum and average global pooling of the output of the segmentation network. And it is attached to the final fully connected layer to prevent overfitting when the number of parameters is large. Finally, in order to simplify the network, different shortcut paths are added to the global maximum pooling layer and the convolutional layer, which is also one of the innovations of the SBDL network. The network architecture of SBDL is shown in Fig. The tongue crack extraction model was trained by using the SBDL network in tensorflow 1.13 with CUDA10.0 and cuDNN7.5.0. The whole training process adopted stochastic gradient descent (SGD) to achieve a faster convergence speed. For the binary classification problem, the segmentation network only classified a single pixel in the image, used Mean Square Error (MSE) loss regression to perform bounding box regression, and used the cross-entropy loss function as a single pixel binary classification training strategy. The two networks were trained separately. Firstly, the segmentation network was trained, and then the decision network was trained while the weights of the segmentation network remained unchanged. The decision network only fine-tuned the weights during training, which largely avoid the risk of overfitting. Both stages were trained using SGD with a learning rate and cross-entropy loss rate of 0.005 and 0.1. Due to the image size and GPU memory constrained, the batch size was set to 2. Although the selection of training samples was random, in order to balance the number of cracked and non-cracked tongue images, the network detected cracked tongue image when the iteration number was even and non-cracked tongue image when the iteration number was odd. This approach ensured that the network detected tongue images at a constant rate. Otherwise, the imbalance of feature learning would bring poor generalization ability, overfitting and low learning speed to the model. The experiments are conducted on a 64-bit windows server with one Intel Core i7-10700 CPU, a 16G memory and one Nvidia RTX 3060 GPU.The extraction of tongue cracks belongs to semantic segmentation. Semantic segmentation is pixel-level classification. The indicators used to quantitatively evaluate the effect of semantic segmentation are Mean Pixel Accuracy (MPA), Mean Intersection over Union (MIoU) and Frequency Weighted Intersection over Union (FWIoU). And Class Pixel Accuracy (CPA) and Intersection over Union (IoU) are specially added as the evaluate indicators of tongue crack segmentation. The formulas are as follows:n is the number of all image categories except the image background;i is misclassified into class j, j is misclassified into class i,i is correctly classified into class i.In the formula, In addition to MIoU, general image processing algorithm performance evaluation adopts other three indicators: Sensitivity (SE), Specificity (SP) and Accuracy (ACC), which are defined as follows:In the formula, TP means the positive samples are correctly predicted, and TN means the negative samples are correctly predicted; FP and FN mean the samples are mispredicted, the former means the positive samples are wrongly predicted as negative samples, and the latter means the negative samples are wrongly predicted as positive sample. SE refers to the number of positive samples correctly identified, and specificity (SP) refers to the number of negative samples correctly identified. Accuracy (ACC) refers to the total number of samples correctly identified.At the same time, considering the application of tongue crack recognition in actual situations, this paper also adopts the professional evaluation for tongue cracks in traditional Chinese medicine (TCM). The experts of TCM usually use three terms: single crack, double cracks and multiple cracks to describe the number of tongue cracks, and use shallow cracks and deep cracks to describe the depth of tongue cracks. Therefore, in the objective study of tongue diagnosis in modern Chinese medicine, the visible index is commonly used to indicate the number of cracks, and the depth index indicates the depth of the crack. The specific formulas of the visible index and the depth index are as follows:In the formula, Since there are various types of tongue cracks which have different shapes, the ability of the model to identify cracks is particularly important. Therefore, the ability of the SBDL model to extract multiple tongue cracks must be considered. The trained SBDL network model is used to extract and identify the tongue cracks in the test set, and finally the test results are further processed through morphological processing such as corrosion and refinement to obtain tongue cracks.In this paper, the extracted tongue crack image is corroded and refined with a rectangular template of size 3 × 3, and the skeleton pixels of the tongue crack are extracted to obtain a more accurate tongue crack area.Generally speaking, when the color of the tongue crack area is not close to the surrounding tongue or tongue coating and the tongue crack edge is obvious, the crack extraction is easy to complete, and the effect is satisfactory. When the color difference between the two is large, even if the edge of the crack is not obvious, it can easily obtain better crack extraction results. However, when the tongue crack area is similar to the color of the surrounding tongue or tongue coating and the edge of the crack is not obvious, it is difficult to extract the tongue crack. But the model based on SBDL network can handle this situation well. In order to more intuitively illustrate the advantages of the SBDL network compared with the traditional tongue crack identification methods and further reflect the better robustness of the model, we compared it with the local grayscale threshold method, as shown in Fig. In Fig. In order to more accurately measure the results of the crack extraction method based on SBDL network, the tongue crack extraction results are calculated according to formula –5) and  and 5) aIt can be seen from Table It can be seen from Table When the color of the tongue crack is slightly different from the surrounding tongue and the tongue crack isn’t single crack or double cracks, although the local gray threshold algorithm can also be used to obtain the approximate shape of the tongue crack, the complete crack area cannot be obtained, most of the crack area is missed. Not only that, tongue contours and tongue pricks are often misidentified as cracks and appear in the crack identification and extraction results. Therefore, the crack extraction results obtained by local grayscale threshold method not only miss some parts but also have fake cracks, the crack extraction effect is not good. However, the tongue crack extraction based on the SBDL network can obtain a more accurate tongue crack region. Furthermore, tongue pricks are not included in the crack results due to their different color and texture characteristics from the surrounding tongue tissue. On the contrary, the tongue contours and pricks can be better identified as non-cracks according to the color, texture and shape characteristics, so the crack extraction effect based on SBDL is satisfactory.In addition, when identifying the non-crack tongue, the local grayscale threshold algorithm cannot exclude the color change of the tongue body well, so the area with the changing tongue color is misidentified as crack area, which misidentificates non-cracked tongue as cracked tongue. However, the SBDL model can better distinguish the difference between the tongue tissue with changing color and the tongue crack, so as to make a correct judgment, no cracks are mistakenly extracted from non-cracked tongue.To sum up, the SBDL-based crack extraction method can better utilize the color, texture and shape features of the tongue to distinguish between the tongue crack and the tongue contours, prick, tongue body with changing color. Therefore, it can effectively solve the problem of inaccurate tongue crack extraction area caused by the closeness of the tongue crack to the surrounding tongue and tongue coating color and obtain better crack extraction results.Combined with the results of the Mask R-CNN, DeeplabV3+, U-Net, UNet++ and SegAN models shown in Fig. It can be seen from Table In a word, the model based on SBDL can not only accurately identify the location of the tongue crack, but also can better extract the details of the tongue cracks, and the segmentation results of tongue crack are relatively complete, which are better than the other five models. And this also indicates that SBDL retains superior and stable performance when small number of training samples are available.In order to further illustrate the ability of the SBDL model to judge the classification of tongue cracks, the classification ability of the SBDL model is evaluated by using formulas –8), and, and8), As can be seen from Table The crack extraction model based on SBDL has a special network structure, which uses a combination of two different types of networks: a segmentation network using the pixel-level segmentation principle is responsible for generating a segmentation mask and determining the specific regional location of the tongue crack, and the decision network for binary classification is mainly responsible for judging whether the current tongue image has cracks. By using multiple convolutional and downsampling layers to ensure the extraction performance of complex shapes, the network is able to capture local shapes that span the entire image. The decision network takes full advantage of the output of the segment network and obtained features, reducing the use of a large number of feature maps. In the extraction and identification of tongue cracks, the decision to whether having cracks and the localization of cracks are equally important, and SBDL has satisfactory performance in both aspects. Because of the advantages that the structure of SBDL is combined by a segmentation network and a decision network, SBDL can achieve a good effect on tongue cracks dataset, and the performance improvement can also be generalized to other public datasets.The crack extraction model based on SBDL can obtain excellent results. In order to better apply it to the clinic, the four tongue crack extraction results in Fig. For the four tongue images of single crack, double-crack, multi-crack and none- crack in Fig. In this paper, a tongue crack extraction and recognition method based on SBDL network are presented. There are five stages of the proposed method. Firstly, crack tongue and non-crack tongue images were assigned to training samples and test samples in a ratio of 8:2. Secondly, the labeling tool Crack-Tongue was used to label tongue cracks to obtain the ground truth of tongue crack, which were put into the corresponding training set and test set. Thirdly, the training set was used to train tongue crack extraction and recognition model. Fourthly, we input test set into the model to obtain test results. Finally, test results were processed by corrosion and refinement operations to obtain final crack extraction results. And the results of SBDL, Mask R-CNN, DeeplabV3+, U-Net, UNet++ and SegAN were compared by qualitative and quantitative evaluation methods, which verified the effectiveness of SBDL model. Although tongue crack images are different, the crack extraction model based on SBDL can achieve more accurate extraction and recognition for different tongue cracks.The experimental protocol was established, according to the ethical guidelines of the Helsinki Declaration and was approved by the Human Ethics Committee of Shanghai Hospital of Traditional Chinese Medicine. Written informed consent was obtained from individual or guardian participants.Written informed consent for publication of this paper was obtained from the East China University of Science and Technology and Shanghai University of Traditional Chinese Medicine and all authors.


Moving object detection and tracking are technologies applied to wide research fields including traffic monitoring and recognition of workers in surrounding heavy equipment environments. However, the conventional moving object detection methods have faced many problems such as much computing time, image noises, and disappearance of targets due to obstacles. In this paper, we introduce a new moving object detection and tracking algorithm based on the sparse optical flow for reducing computing time, removing noises and estimating the target efficiently. The developed algorithm maintains a variety of corner features with refreshed corner features, and the moving window detector is proposed to determine the feature points for tracking, based on the location history of the points. The performance of detecting moving objects is greatly improved through the moving window detector and the continuous target estimation. The memory-based estimator provides the capability to recall the location of corner features for a period of time, and it has an effect of tracking targets obscured by obstacles. The suggested approach was applied to real environments including various illumination (indoor and outdoor) conditions, a number of moving objects and obstacles, and the performance was evaluated on an embedded board (Raspberry pi4). The experimental results show that the proposed method maintains a high FPS (frame per seconds) and improves the accuracy performance, compared with the conventional optical flow methods and vision approaches such as Haar-like and Hog methods. Technologies to detect and track moving objects are of significance in many applications such as unmanned vehicles and surveillance cameras to detect and recognize pedestrians and track workers in factory environments to ensure personnel safety. Conventional algorithms to detect and track moving objects include frame difference algorithms, background subtraction algorithms, optical flow-based algorithms, and static learning algorithms. In addition, various deep-learning-based moving-object detection algorithms based on deep convolution networks such as YOLOv5s and R-CNNs have been developed. However, these methods are highly sensitive to changes in the background brightness, which increases the probability of erroneous detection. Furthermore, deep-learning-based algorithms are computationally complex, require sufficient training samples, and are not suitable for real-time processing on a board without graphical processing units (GPUs). In addition, conventional moving-object detection and tracking algorithms cannot effectively detect and track targets obscured by obstacles within the images or when an image is distorted by camera vibrations. These limitations are particularly concerning, especially in applications aimed at ensuring the safety of workers operating near heavy equipment. The Korea Occupational Safety and Health Agency (KOSHA) reports that among the many causes of safety-related accidents, the most notable cause of collisions with objects around heavy equipment in workplaces is the lack of visibility of equipment workers in the environment.To address this problem, this paper proposes a moving-object detection and tracking algorithm that can be applied to various safety-related applications. The proposed algorithm can efficiently eliminate the camera vibration related noise in the image frames and perform continuous tracking for moving objects obscured by obstacles. Specifically, the conventional sparse optical flow algorithm  is enhanced to detect and track multiple moving objects at a low computational cost. Moreover, the corner extraction algorithm (Shi–Tomasi) is used to track feature points to detect and track moving targets. A moving window detector and memorized estimator are used to enhance the detection performance, ensure the robustness of the algorithm to noise, and improve the worker safety. In particular, the moving window detector uses the window memory at each feature point as the window size and detects and tracks the moving target by evaluating whether the feature point is noise or a moving object. The location history of the detected points is memorized, and a halted or invisible target is identified from the location history of the feature points. Subsequently, the estimator decides whether the target state is maintained.The proposed moving-object detection algorithm is computationally effective, and the performance of the sparse optical flow algorithm based on the LK method is enhanced using the Shi–Tomasi corner extraction algorithm. Moreover, a novel moving window detector and memorized estimator function are used.The performance is enhanced by operating on a non-GPU platform and suing a low computational power embedded system such as Raspberry Pi.The moving window detector helps enhance the robustness of the proposed approach against vibration related noise in surveillance system environments.The memorized estimator function can prevent accidents of workers in fields with hazards and obstacles.In recent decades, methods to detect and track moving objects have been widely applied. Before the development of learning-based algorithms, methods based on the optical flow, frame difference, and background subtraction were typically used. In these algorithms, the difference between frames was used to determine the movement of objects, and thus, the objects could be accurately detected. However, these algorithms were computationally intensive and complex.Certain researchers attempted to perform object recognition using the optical flow based on a camera attached to a moving vehicle . SeveralMoreover, the object detection performance for a complex background can be enhanced using the optical flow. Object detection may not be effective when the frame difference technique is integrated with the optical flow technique. Certain researchers developed algorithms to perform background modelling tasks, using edge detection to solve problems . MoreoveMachine learning techniques have also been used to detect and recognize objects. The motion of Earth movement equipment was detected based on the vision at ground level . Images Recently, many researchers developed approaches to detect moving objects by using optical flow and deep learning. With the widespread application of unmanned aerial vehicles (UAVs), moving objects have been attempted to be detected and tracked using cameras within the UAV . In thisIn addition, moving objects for UAVs were detected by obtaining images in real time. A dense optical flow technique was used; however, the background was assumed to be fixed. By obtaining top-view images in the aviation domain, the map for moving objects can be extracted using background removal and mean shift segmentation techniques. Notably, dense optical flow techniques are time intensive, and nearly 3.5 s are required per frame, which limits the application of such techniques to heavy equipment such as microcontroller units (MCUs) . To addrCertain researchers developed a deep-learning-based framework for tracking UAVs. In this approach, moving objects (UAVs) were accurately detected at a high speed by modifying and improving CNN models based on YOLOv3-tiny in a real-time measured video stream . The algCertain researchers attempted to exploit consistent video frame information by directly applying image object detection technology to videos . NotablyMoving-object detection techniques can also be applied for safety evaluations. Certain researchers used computer vision technologies to measure the vibration of buildings . This apObject tracking algorithms have also been applied for ensuring personnel safety during the operation of heavy equipment such as unmanned excavators ,22,23. MThe LK method uses feature points to track the optical flow, which may render object detection over large distances challenging. In contrast to dense optical flow techniques, which evaluate all the pixels on the frame and neighbouring pixels, LK optical flow calculates feature points such as corners to facilitate tracking. Notably, if the moving objects are at a large distance, they are difficult to distinguish from noise. In detecting the motion information for each frame, a noise filtering function is introduced to alleviate the camera vibrations and light spread phenomena. To detect and track the moving objects obtained through feature extraction and LK optical flow, we introduce a memorized estimator to estimate the position of moving objects by memorizing information regarding the last missing position. This approach is expected to be effective in situations in which the object stops moving or is obscured by certain obstacles in the video input.Notably, the proposed algorithm is based on the pyramid LK method but incorporates corner reset, noise filtering, and moving-object estimation  from time To enhance the detection performance, especially when capturing a distant moving object, the moving window method changes the distances of the first and last locations in the window memory, to determine whether the feature is a moving object. Instead of the sum of moving distances within a period, the threshold measurement is based on the varied distances of the points because the vibration related noise moves continuously in a certain period. In contrast to the summed value, the change in location in the period does not cumulate the moving distances, and only the changes in the initial and final point locations are determined. Thus, we define the change in point locations in a constant time interval, Equation .(3)Li,tIn Equation , Li,t isWe propose a function to estimate the location of the tracking object, even if it is temporarily stopped or hidden behind obstacles, by estimating the feature points pertaining to the target in i at time t and is reset to i has a detection state of In Equation , τ is thEquation . If the The non-zero estimator , and number of real-generated features. To select the optimal parameters for each algorithm, we investigate the semi-metric comparison results, apply the selected parameters to the algorithms and evaluate the performance. The follow subsections describe the semi-metric parameters of each algorithm and performance evaluation.We conduct an experiment by applying the proposed algorithm to two large datasets and perform a comparative analysis with other existing algorithms, as shown in The first dataset focuses on the recognition and evaluation performance of moving objects instead of pedestrian shapes, and the second dataset is aimed at comparatively analysing the proposed algorithm with algorithms that can estimate shapes and moving objects. The two datasets are significantly different: The first dataset contains slowly walking people, whereas the second dataset contains rapidly walking people. In the first dataset, the moving targets are more difficult to detect because of the presence of fewer people and people who are walking slowly. Because the two datasets have different resolutions, we consider the resolutions of 384 × 288 and 768 × 576 as small scale and large scale, respectively, to ensure a fair comparison. In the comparison of the optical flow methods, video 4 is resized to the small scale  is presented owing to the limited space. The recall, precision, and number of missing boxes, and number of generated features are presented in the following figures.The following metrics are typically used in object detection experiments: True positive (TP) means successful detection of the ground truth labels, false positive (FP) means detection failure, and false negative (FN) indicates the number of non-detected labels. The precision, recall, and F-score are determined as Moreover, we evaluate the grid located point results. The randomly located point method exhibits an unstable performance, likely because of the stochastically generated point locations. Thus, well-distributed tracking point must be used when implementing a limited number of generation corner number. Distance between corner refers the parameter using in the corner extraction algorithm, which determines the distance between extracted corners. In the case shown in The proposed method differs from other moving-object detection algorithms owing to the implementation of the moving window and estimator method on the LK optical flow algorithm, which enhances the object detection performance and helps overcome the limitations of sparse optical flow techniques. The estimator operates synergistically with the moving window method by preventing the failure of determining the sparse optical flow. Specifically, when the moving window tracking for the change in the location of the feature points fails, the estimator is activated. The estimator remembers the last location of the tracking object and predicts the presence of the disappeared object on the spot. Therefore, we conduct experiments to examine the detection performance with changes in the parameters of the window and estimator: window sizes, distance thresholds, and estimation times.We test four datasets: Browse_WhileWaiting1.mpg, Browse1.mpg, and Walk1.mpg from the CAVIAR dataset for comparing moving-object detection algorithms using typical sparse and dense optical flow; and PETS09-S2L1.webm from the ETS2009 dataset for comparing pedestrian detection algorithms with Hog and Haar-like SVM detection models ,34. The t, and thus, each window memory has a specific size. When the window memory is full, the last location memory is eliminated, and the recent location memory is pushed to the end of the list. Therefore, a constant window memory size is maintained. The window tracks the change in the location of each point and decides whether it is a moving object or noise by considering the moving distance threshold. The influence of the change in the window size is reflected in terms of the true number of detection boxes, recall, and precision. The total number of alarms is the total number of predictions obtained using the proposed detection method, and this value is compared with the true number of detection boxes.The window size is a key parameter of the moving window function in the proposed method. The function contains the locations of all tracking points  in the moving window memory from time In the case of a small window size, the history of the tracking point is limited, and the change in the points’ locations is observed. The probability of the point being identified as a noise instead of a moving object is higher. In contrast, for a large window size, the history of the tracking point is adequate, and the point location can be tracked to examine if it is moving object. However, the detection of true target boxes may be missed because considerable time is required for the evaluation, and the window may fail to track the point when the moving object disappears. For example, if the window size is 40 and the object is moving in 50 frames, the detection box has less than 10 frames. Therefore, the window size must be properly selected. We conduct experiments with different window sizes and examine their influence on the number of detected boxes and accuracy.As shown in With the change in the window size, there occurs a crossing-over point at which the recall and precision curves intersect. This point is likely an optimal value to ensure a stable performance between the recall and precision as well as the F-score (F1). The total true detection number steadily increases and adversely influences the recall performance. Thus, longer tracking of the change in the point location leads to the detection of more true moving objects because the moving window detector obtains the interpretation based on a longer history at each point. The precision decreases as the recall increases because the optic flow tracking points that remain at and depart from moving object are considered to be in the moving state by the moving window memory. We validate this analysis by investigating the output detection labelled video. The red labelled box is the ground truth box, the green labelled box is the detection result of the moving-object detector algorithm, and the blue labelled box pertains to false detections. The findings indicate that the optimal window size is 10 frames.The memorized estimator remembers the last position at which the calculation was stopped for a certain period (frames) and continues prediction even when the moving object stops or disappears from the video. To evaluate and select optimal parameters for this function, we determine the number of detections according to the estimation time and existence of the estimation function. The performance indicators of recall and precision are determined.The memorized estimator influences the detection accuracy. When the object is not sensitively detected, the detector misses the moving target and the estimator supplements the insufficient information of the moving target by memorizing the target’s last location information such as the moving window memory. As shown in the last row in The proposed algorithm has three key parameters. Based on the experimental results, we determine the optimal parameters for the corner features , memorized moving window (window size and distance threshold), and memorized estimator (estimation time).In The white bar corresponds to a low recall and high precision accuracy, which is not suitable for detection and may increase the possibilities of accidents pertaining to missing moving objects such as workers in the industrial field. The proposed moving window detector and memorized estimator can help enhance the safety and detection performance. Both methods exhibit higher recall and precision. Moreover, the estimator enhances the true detection rate, and the estimator influences the false detection rate for disappeared or stationary tracking objects. As shown in We halve or double the resolutions of the input videos according to the object detection algorithm. Machine learning detection algorithms such as Hog and Haar-like algorithms typically learn objects with resolutions of 64 × 128. The resolution of videos 1, 2, and 3 is 384 × 288 and that of video 4 is 768 × 576. The resolution of videos 1, 2, and 3 is converted to 768 × 576 and compared with video 4 in terms of machine learning object detection algorithms; the videos are named videos The LK optical flow method using in the comparison experiment is a typical moving object detection algorithm based on sparse optical flow, and a simple noise filtering function of the moving distance threshold is applied between the locations of the previous and present pixel. The Farneback method is also a typical moving object detection algorithm based on dense optical flow and simple noise filtering function of the optical flow magnitude threshold for pixels. The Hog and Haar-like object detection algorithms are famous machine learning classification methods based on the learning weights of pedestrian. All the algorithm codes are sourced from the official OpenCV community site.Our method consists of the moving window system and the target estimator. Although the computational speed of the proposed algorithm is similar to that of typical sparse optical flow in The Hog and Haar-like methods exhibit inferior detection performance and low robustness in various environments because of the fixed pretrained filter weight, distortions, and various sizes of the moving targets in the frame. We compare the LK and Farneback methods . The parameters of the compared methods is optimized to ensure a fair comparison. For the LK method, the corner quality parameter ranges from 0.01 to 0.1 to avoid a large number of corners being generated on the noise. For the Farneback method, we set the mean N parameter from 3 to 30 to normalize the baseline of decisions between noises and moving targets, which smoothens the generated optical flows. The precision, recall, F-score, and FPS results are compared.The results presented in The proposed method memorizes the location history of each feature point in the moving window and tracks the target in the window. The proposed method can effectively distinguish the additional noise and moving targets because each feature point has its own window memory. The proposed method outperforms the LK method and achieves a higher FPS than the Farneback method. Therefore, In this paper, a new approach to moving object detection and tracking is proposed. One of the major attributes is to improve the accuracy performance and reduce the computation time of responding to moving objects or moving pedestrians. The proposed method is based on the sparse optical flow approach, that is, a coarse-grained optical flow, but it includes the corner feature reset with a moving window. A sequence of images effectively finds the flow of moving objects and thus the moving window of image frames easily captures moving targets without wasting much time.The moving window detector improves the noise filtering and the detection rate, by looking at a history of optical flows. In a hazardous environment, such as the construction sector, there may be the risk of meeting many obstacles including walls and trees, and various optical flow patterns are often observed, when pedestrians should be detected. The memory-based target estimator plays the role of monitoring the targets or pedestrians without missing when the targets move around or stagger at some positions. Even if a moving target is initially recognized, the target may move continuously with occasional pause. With this estimator, the last position of a moving targets is estimated and this improves the performance of detecting moving objects in a row.We adapt this detection algorithm in the embedded board system, Raspberry Pi4, for real application. The experimental results demonstrate that the suggested approach is effective for preserving the detection performance even with a low computing power of the embedded device. According to the experimental results, our proposed method shows similar or higher accuracy performance, compared to the conventional algorithms for moving object detection using optical flows or vision processing algorithms: Lukas–Kanade’s method and Farneback’s method in addition to Hog and Haar-like methods. It also provides a more efficient computing time than dense optical flows and vision processing algorithms. The approach works well even for distorted views from a top-viewed camera and also for blurred images or noisy image frames, and thus it can be robustly applied to various environments. For future work, we consider the optical flow approach with the identification of targets. Deep learning approaches have been popular for identifying pedestrians or objects, but they need much computing time. The suggested approach helps to detect moving objects easily and then the objects could be identified with a small size of neural network to reduce the computing time.


Predicting the apparent age has been quite difficult for machines and humans. However, researchers have focused on machine estimation of “age as perceived” to a high level of accuracy. To further improve the performance of apparent age estimation from the facial image, researchers continue to examine different methods to enhance its results further. This paper presents a critical review of the modern approaches and techniques for the apparent age estimation task. We also present a comparative analysis of the performance of some of those approaches on the apparent facial aging benchmark. The study also highlights the strengths and weaknesses of each approach used for apparent age estimation to guide in choosing the appropriate algorithms for future work in the field. The work focuses on the most popular algorithms and those that appear to have been the most successful for apparent age estimation to improve on the existing state-of-the-art results. We based our evaluations on three facial aging datasets, including looking at people (LAP)-2015, LAP-2016, and APPA-REAL, the most popular and publicly available datasets benchmark for apparent age estimation.Apparent age estimation  Age estimation is a very prolific area of research within the computer vision community  a better method for the estimation task. Many researchers have studied several of these CNN methods, and these methods have improved the results and performances of apparent age estimation tasks. However, due to the challenging nature of apparent age estimation, further attempts to enhance the accuracy of the age estimation are still very much in progress. Researchers continue to examine different CNN and modern methods to enhance the results further. Hence, this paper critically reviews the modern approaches and techniques employed for apparent age estimation. We also present a comparative analysis of the performance of some of those approaches on standard apparent age datasets. The study also highlights the strengths and weaknesses of each approach to apparent age estimation to guide in choosing the appropriate approach to further improve the existing state-of-the-art results in the field. To ensure fairness in evaluating the performance of these approaches, we employed the popular apparent aging datasets and standard evaluation metrics that are widely used in literature in age estimation. The contributions of this paper are highlighted as follows:We outlined different state-of-the-art algorithms and techniques for apparent age estimation.We described the performance evaluation analysis of different state-of-the-art models in apparent age estimation.We presented three facial aging datasets widely employed in the research of apparent age estimation.We also highlighted the standard performance evaluation metrics common in most literature for the apparent age estimation.Apparent age estimation has many notable real-world applications. Different intelligent application scenarios can benefit from computer-based systems that predict the apparent age of people among such application areas, including the following:Computer-based age prediction, as perceived by people, helps determine whether factors like the environment, depression, sickness, fatigue, and stress affect the premature aging of a person. This automatic age prediction will assist in obtaining the required information needed on a decision to improve the person's aging system  result and improves the performance of the estimation accuracy. However, MR generates an unsteady training mode, causing a significant error term affecting accuracy performance. Some of the typical regression methods include Gaussian Process -error metric was employed for the purpose of the Chalearn LAP competition in 2015. It calculates the error between an estimated value and the average labeled age. ϵ-error is calculated as follows:where x: the estimated age.σ: apparent age label provided for a given face image.μ: standard deviation of all estimated ages for the given face image.ϵ-error not only calculates the error between the estimated value x and the averaging labeled age σ but also considers the standard deviation of μ. The final ϵ-error result is the average overall prediction, and obviously, the lower the ϵ-error, the better the estimator's performance.In recent times, a study in apparent age estimation has progressed steadily in terms of performance. From the literature presented above, it was clear that no concise conclusion can be made on the best type of algorithm for the estimation task. In As presented in Consequently, significant observations from earlier research work in the estimation help make some justifiable conclusions. Hence, we highlighted those observations stating some of the points which we deem very important:It was observed that the image processing method employed for face detection, facial landmark, and face alignment has an impact on the performance of an apparent age estimator.It was also important to assert that the performance of learning algorithms was determined by many factors, among which are the size and label distribution of the employed dataset, the degree of image variability, etc. The deep learning algorithms perform differently on different datasets, most likely due to the peculiarities of each dataset.Data augmentation improves the performance of age estimation models, especially on unevenly distributed not-too large datasets.We also observed from the literature that models pre-trained on large-scale datasets before fine-tuning on the original dataset performed better than training the model on just the original dataset.From this review, we observed that MC algorithms had been the most popularly used individual algorithm for apparent age estimation on the mentioned datasets.We also observed that ranking and DLDL are the most suitable algorithms for the estimation when the label distribution of the dataset is uneven.For apparent age estimation on LAP-2015, a hybrid algorithm (LDL and expectation regression) showed the best performance on MAE and ϵ-error metrics , 6. HoweThis work presents a comprehensive review and the suitability of modern algorithms for apparent age estimation focusing on those algorithms that are most popular and those that appear to have been the most successful. In this review, we based our evaluations on LAP- 2015, LAP-2016, and APPA-REAL, the most popularly-used publicly available facial aging datasets for apparent age estimation. Apparent age estimation, estimate “how old the person looks like”? A hybridized algorithm showed the best performance on LAP-2015 and LAP-2016 datasets. From this study, we could deduce that MC was the most popularly used individual algorithm. We also asserted that the performance of these age estimation algorithms is mostly influenced, not only by choice of the approach but also by some other factors, among which are the method of image pre-processing applied, size and the distributions of the employed datasets, etc.However, many encouraging future directions in apparent age estimations may improve performance. Huge datasets with apparent age label annotation rather than real age will help improve the research accuracy of apparent age estimation. Also, it is necessary to investigate further study that focuses more on predicting apparent age (how old does the person look?) rather than a person's biological age to enhance the practical and real-world applications of this research.All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Ion channels remain fascinating molecular machines implicated in virtually every cellular function. Their activity can be studied in deep detail using biophysical techniques down to the single-molecule level. However, as large hydrophobic proteins embedded in a lipidic environment, their structure has traditionally been very difficult to study. Cryo-EM approaches have boosted our knowledge in the last few years, expanding the collection of resolved structures almost on a weekly basis. Yet, there are still open questions regarding the structure-function of the channels that are now starting to find answers.Ion channels react rapidly to a wide range of stimuli, opening a pathway for the flow of ions across the membrane. The coupling of the stimulus to the opening of the gate can be studied in ligand-gated channels by comparing the structures of the ligand-bound and unbound channels. Still, such a comparison is more difficult to achieve when the channel responds to physical rather than chemical stimuli, as is the case of voltage-gated channels. The molecular principles of voltage-dependent gating of ion channels have been known for four decades. The mechanism consists, in essence, of the movement of some parts of the protein (the voltage-sensing domains) relative to others. The displacement results in a conformational change that produces the opening of the gate, but the intimate molecular mechanisms linking both events remain only partly known in many cases. Although the problem might appear like an academic discussion for experts at first glance, it has many practical implications. On the one hand—mainly in nonexcitable cells—it is often the movement of the voltage sensor, and not the flow of ions through the channel that determines the cells biological functions. The interaction of Kv11.1 (HERG) with β-integrin is a prominent example of this situationAlthough there is still room for improvement because the structure resolved is not that of the permeating channel but of a complex with calmodulin, which acts as a channel inhibitor, this kind of experiment will not only advance our understanding of the intramolecular interactions during voltage-dependent gating. They also provide information on the rearrangements of the membrane lipids induced by the displacement of the voltage-sensing domain, which are important modulators of channel function, and on the changes of interactions with partner proteins. The functional channels are integrated into larger supramolecular complexes composed of different proteins, often containing more than one channel type. Because the function of the channels can be measured very accurately, they are a proxy for studying the operation of the whole complex. Refinement of the cryo-EM technology itself and the use of elegant reconstitution approaches has allowed resolving structures of such large supramolecular complexes of channels and accessory/modulatory proteins (egIn summary, the ion channel field is living a rebirth driven by structural information progressively closer to the native conformation of the complexes where channels operate. Together with rapidly advancing molecular dynamics simulations, improvements in biophysical techniques, and sophisticated models, our understanding of integrative ion channel physiology will certainly flourish in the coming years.


Many surgical complications can be prevented by careful operation planning and preoperative evaluation of the anatomical features. Virtual dental implant planning in three-dimensional stereoscopic virtual reality environment has advantages over three-dimensional projections on two-dimensional screens. In the virtual environment, the anatomical areas of the body can be assessed and interacted with in six degrees-of-freedom. Our aim was to make a preliminary evaluation of how professional users perceive the use of the virtual environment on their field.We prepared a novel implementation of a virtual dental implant planning system and conducted a small-scale user study with four dentomaxillofacial radiologists to evaluate the usability of direct and indirect interaction in a planning task.We found that all four participants ranked direct interaction, planning the implant placement without handles, to be better than the indirect condition where the implant model had handles.The radiologists valued the three-dimensional environment for three-dimensional object manipulation even if usability issues of the handles affected the feel of use and the evaluation results. Direct interaction was seen as easy, accurate, and natural. In implant surgery, missing teeth are replaced with dental implants and prosthesis, like an implant-supported denture or fixed supra-structures. The workflow of computer-aided design (CAD) includes several steps , 2, of wImaging systems with potential to capture medically and diagnostically relevant images have improved, but devices for visualization and interpretation of the acquired images have lagged in development. The visualization tools used in the medical field are quite conservative compared to tools in the entertainment like 3D movies, augmented reality games, and virtual reality (VR). Radiologists and surgeons usually evaluate 3D images on 2D screens from three standard directions  . Seeing VR has significant advantages over 3D visualizations on 2D screens. It allows interactivity and immersion within the virtual 3D environment. The interaction in VR is usually done directly with hand controllers. The image can be translated, rotated, and scaled from arbitrary angles .Zorzal et al.  studied The implants in the VR system are moved to their planned positions by picking them up using the VR controller and doing translations and rotations. While that is straightforward, fine-tuning the position and orientation may be challenging when movements will affect all 6 degrees-of-freedom  simultaneously. When correcting the position, a small tilt of the controller will also change the orientation.To facilitate the problem of unwanted movements of linked axes, Mendes et al.  used a wWe made a preliminary evaluation of the usability of the implant handles when the participants were asked to position an implant into a suitable location. The task was executed in two conditions, with and without the implant handles. The widget handles will enable more careful moving and turning of the implants but will also use more time as complicated movements would require several operations.The accurate placement of dental implants is important to achieve the functional and aesthetic demands. The anatomical features of the jaws limit the placement. The biggest concerns are in the molar and premolar regions. In the upper jaw, the maxilla, maxillary, and nasal cavities, and in the lower jaw, the inferior alveolaris nerve and foramen mentale set the boundaries for the implantation. In the anterior areas, the shape and the amount of available bone of the alveolar ridge may be a challenge.Proper planning of implantation and placement of the implant are essential for successful implantation. Complications associated with implant surgery include for example bleeding, in some cases even life-threatening hemorrhages, temporary or permanent nerve injuries, malposition or displacement of implants, injury to adjacent teeth and fracture of the mandible –18.VR environment can engage the viewer in a 3D space and enable evaluation of the anatomical structures from a new perspective . VirtualTranslation, rotation, scaling, and selection are common actions in virtual environments , 20. DirWith virtual handles, the interaction is no longer direct . Hand inMendes et al.  discusseHands and controllers are common interaction methods in VR, and they provide the same actions –26. ContWithout handles, the implant was manipulated by grabbing the body of the implant. In another condition, Handles, there were eight handles attached to the implant that enabled more careful, restricted manipulations of the position, and orientation. An implant without and with handles is shown in Fig. We conducted a controlled small-scale user experiment to evaluate the usability of the virtual handles. The participants performed three dental implant planning tasks with two different conditions. In one condition, For the experiment, we prepared three different skull models. Skull visualizations were generated in real-time from CBCT DICOM  volumetric data. Skull models were pseudonymized and may be used with permission from individuals in product development.Eight handles were located around the implant to translate and rotate the implant. The handles had a shape of an arrow. A straight arrow was for transferring the implant and a curved arrow was for rotating it. There were three straight arrows to transfer the implant along the basic axes x, y, and z. By grabbing a handle, the implant could be transferred on both directions.The curved handles would rotate the implant around the x, y, and z axes, while the pivot point location would vary. One curved handle was perpendicular to the implant main axis to rotate the implant around that. The four curved handles that would tilt the implant were paired (one up and one down) so that grabbing the upper handle would use the apex of the implant as a pivot point and the lower handle would use the collar of the implant as a pivot point. The user could fix, for example, the apex of an implant and carefully tilt the orientation without compromising the apex location.Efficiency: Task completion timeEffectiveness: Marking consistencySatisfaction: Subjective evaluationsTo evaluate the usability of the interaction methods in accordance with the ISO definition , three dWe measured the task completion times from the moment that the implant was picked up to the moment that the participant released the implant for the last time.For the marking consistency, we measured the positions of each implant and compared the positions in the analysis phase. We also counted the number of implant pick ups and releases.We asked the participants to evaluate several subjective questions and a statement  had some experience of using VR devices, but only a few months.Upon arrival, the participant was introduced to the equipment and the system functions and controls. S/he was asked to read and sign a consent form and fill in a background information form. The participant practiced the system to be able to do the tasks. When ready, the participant made three implant plans for the first condition.Handles condition and then Without handles. The two others did it the other way around.For all participants, the skull models were presented in the same order. In the first skull model (model 1), the implant was positioned to the left side (patient view) of mandible bone. In the second model (model 2), the implant was positioned to right side of mandible and in the last model (model 3) to the front of the maxilla bone. The order of conditions was balanced for the experiment. Two participants did the implant planning first with After three models, the participant was asked to fill a questionnaire to do a subjective evaluation of the condition. After that s/he would repeat the procedure for the second condition. After both conditions, the participant was asked to rank the conditions. The experiment, including practice and the questionnaires, took around 45 minutes per participant.There was no difference in efficiency between conditions. The task completion times varied between participants and the conditions. The completion time minimums were around 40 seconds, and the maximums around 500 seconds, with no trends. The mean time was 189.3 seconds with standard deviation of 132.7 seconds.We did not measure any difference in efficiency between the conditions. We recorded all the final positions of all 24 implants that were set , one participant evaluated Handles to be easier, while the rest of the participants evaluated Without handles easier. The last statement was if the participant needed to learn a lot of things before working with the system, and with Handles more things had to be learnt.In the satisfaction, the subjective evaluation results show that the interaction method without handles was seen more positively. The results of the subjective evaluations are shown in Fig. Without handles got better evaluation than Handles as indicated by mean and median values . The evaluation values were consistently better  for Without handles than for Handles for the five questions. For the last statement, the grade 1 means no need for learning and Without handles got lower values.On all measured scales, Without handles to be better than Handles.In the final ranking, all the four participants ranked Our experiment was a preliminary study to evaluate the use of virtual environment by professional users on their field and for that we recruited four dentomaxillofacial radiologists who did six dental implant planning tasks each. The small number of participants and tasks limits the generalizability of the results.The task completion times varied a lot. We encouraged the participants to think aloud to get comments. As the size of a handle was smaller than an implant, grabbing a handle might have taken longer (based on Fitts’ law ) than grAs different radiologists have slightly different views of the constraints set by the anatomical features, we do not have an unambiguous target location for the implant and some variation in placement would be expected. As shown in Fig. Without handles was easy, accurate, and natural. In subjective results, almost all statements got better grades for Without handles. Each handle took space so that the closest handles blocked the farthest handles. In addition, grabbing a specific handle was sometimes difficult. Based on Fitts’ law [In the evaluations, the participants indicated that it is better to do the implant placement without any handles. Multiple participants commented that tts’ law  and Freetts’ law , the sizThe two modes compared here could be seen as complementing each other. While free hand is convenient in the beginning of planning, restricted movement may be required when multiple implants have been placed and their relative positions should be maintained.The participants commented that the handles were fine and subjective results show that handles did not get poor evaluation. Two participants commented that the handles were good for fine-tuning, but large movements were difficult. When working without handles, one participant said “My hands are shaking so fine-tuning tool would be nice.” Another participant commented that s/he was surprised how accurate the implant placement was by direct interaction and that no shaking could be noticed. Participants discussed alternative fine-tuning methods for handles, which indicates that there is a need for implant fine-tuning.An HMD presents 3D view naturally based on the position and orientation of the user’s head . VR provFour dentomaxillofacial radiologists performed implant planning tasks in virtual reality environment using direct and indirect object manipulation methods. The task was executed in two conditions, either the implants had virtual handles or there were no virtual handles. All the four participants ranked the direct interaction method where the implants had no virtual handles to be better than the indirect condition where implants had virtual handles. The participants noticed the advantages of the handles for fine-tuning the placement but the handle usability issues were disturbing. The direct interaction condition was commented as easy, accurate, and natural, while more learning would be required to use the virtual handles. Overall, both object manipulation methods were evaluated to be good, and the 3D virtual environment was valued for the implant planning task. The implant placements made by the radiologists were practically similar. This finding gives important information of the accuracy and potential to use this kind of VR methods in clinical work. Future research for technical and clinical validity is needed.


With the progress of social production, the competition for talents among enterprises is fierce, and the market often lacks capable leaders, which leads to the lack of management of enterprise employees and cannot bring more economic benefits to enterprises. Traditional leaders make subordinate employees work actively and achieve the common goal of the enterprise by exerting their own leadership characteristics and observing their subordinates, but they cannot take care of the psychological state of each employee, resulting in the employee's work efficiency is not very high. In recent years, charismatic leadership has become an important economic leader in the new era, and the theoretical spirit of charismatic leadership can well guide employees to work actively. Artificial intelligence affective computing can well identify the psychological state of the subject, and the combination of artificial intelligence affective computing and charismatic leadership can achieve effective management of employees through the predictive analysis of employees' psychological state. This paper compares the psychological state prediction analysis of employees' work attitudes between charismatic leaders based on artificial intelligence affective computing and traditional leaders through experiments. The results show that: charismatic leaders based on artificial intelligence affective computing predictive analytics can improve sensitivity to employee needs, resulting in an 8.2% increase in employee trust in leadership, a 4.4% increase in employee commitment to achieving organizational goals, and a 19.3% increase in employee job satisfaction. The psychological state prediction analysis of charismatic leaders based on artificial intelligence affective computing on employees' work attitudes can improve the work efficiency of employees and the economic benefits of enterprises. Since the middle of the 20th century, the global economy has developed rapidly, the competition among enterprises is fierce, lack of competent management personnel, and there is a shortage of capable management talents. Organizations or enterprises are in urgent need of capable management talents to cope with the changing market. Therefore, charismatic leadership emerges as the times require, and the charismatic management concept is to encourage and guide employees to make positive work responses through the manager's own charisma. Charismatic leadership mainly analyzes the psychological state of subordinate employees' work attitude and makes corresponding measures according to different employee characteristics. With the development of technologies such as artificial intelligence and sentiment analysis, the results of predictive analysis of people's psychological states are becoming more and more accurate. Combining charismatic leaders with artificial intelligence emotional computing can accurately predict the work attitude of subordinate employees, and make corresponding strategies. Through the psychological state prediction analysis of the charismatic leadership of artificial intelligence affective computing on employees' work attitudes, the employees' work attitudes are more positive and the group performance of the enterprise is improved. Therefore, this paper has research significance.The psychological state prediction analysis of charismatic leadership on employees' work attitude can make corresponding strategies for employees' psychology to improve employees' work attitude, and relevant workers have conducted in-depth research on it. Among them, Lee showed through research that the analysis of employees' psychological state by charismatic leadership can improve employees' work enthusiasm and organizational performance  Combining artificial intelligence emotional computing technology with charismatic leadership; (2) Comparing with traditional leadership models, highlighting the advantages of charismatic leadership based on artificial intelligence emotional computing in managing subordinates.The combination of charismatic leadership and artificial intelligence emotional computing can improve employees' job satisfaction and improve the economic benefits of enterprises through tracking and monitoring of employees' work attitudes, emotional prediction, and emotional analysis. The underlying basic algorithms of intelligent emotional computing include speech emotion recognition algorithm, artificial neural network, convolutional neural network, etc. , it represents the pre-emphasis parameter, and the value range is .Speech emotion signal is a continuous time function, and it fluctuates greatly, so the processing method of stationary signal is obviously not suitable for the analysis of speech emotion signal. In order to better analyze the speech signal, the speech signal is divided into a small segment of speech. When the segmentation is short enough, the speech signal in a short period of time can be regarded as a stable value. Dividing each small part is called a frame, and the length of a frame is 15 ms in this paper.However, since the language spoken by people in reality is not discontinuous, that is to say, there will be a connection between two adjacent frames, that is, the problem of overlapping of the two frames will occur, so the speech frame needs to be divided into frames. However, the framing process will cause discontinuity at the front and rear ends of the speech frame, and the more framing times, the greater the speech error. Therefore, the speech frame after framing should be processed by windowing , k represents a window coefficient.In addition to the specific provisions of linguistics, the reason why people can understand the meaning of the other person's voice, there is also the unique information of the voice, which is called the feature quantity of the voice. To accurately analyze speech emotion, it is necessary to clearly understand the relationship between the characteristic parameters of speech emotion signal and emotion state. Analysis of the data in In addition to the speech feature parameters in The Mel cepstral coefficient can perceive the characteristics of speech emotional signals at different frequencies. The Mel coefficient is an important indicator in the Mel cepstrum. The Mel coefficient has a nonlinear relationship with the frequency of the speech emotional signal. The Mel function is expressed as:In formula (4), v represents the frequency of the speech emotion signal, 1 n means take the logarithm.In the process of extracting emotional features by Mel cepstrum, the process of extracting Mel coefficients by using the Mel function is as follows: the speech signal is preprocessed and then Fourier transform is performed to make the time-frequency speech signal into a frequency-domain speech signal; using a first-order filter Filtering, the filtering range is the range of human audible speech; the speech signal is processed logarithmically, and finally the speech signal is represented by trigonometric function transformation, and the change process is expressed as:Short-term energy is the energy attached to language emotional signals in a very short period of time. The short-term energy is related to the vibration of the sound. Usually, the more intense the emotion, the greater the short-term energy. For example, the short-term energy of fear or surprise is generally than the short-term energy of grief. The short-term energy pass is used to distinguish between unvoiced and voiced speech  represents the speech emotion signal.In formula (6), The short-term jitter energy of the voice signal is:Ek represents the short-term jitter energy of the speech signal.In formula (7), The linear regression equation of short-term energy is expressed as:Eg represents a linear regression coefficient of short-term energy.In formula (8), The short-term zero-crossing rate is a way to detect the presence or absence of sound at both ends of the voice signal, and the short-term zero-crossing rate is the number of times the voice signal frame crosses the time axis. The short-term zero-crossing rate is expressed as:sx is the speech emotion signal frame, and Qx is the short-term zero-crossing rate, sgn stands for symbolic function.In formula (9), sgn [x] is a mathematical symbolic function, which is specifically expressed as:The artificial neural network is a process that imitates the human brain to process information. The artificial neural network is composed of many neuron nodes. It has high memory and learning ability and can accurately predict the psychological state of the employee's work attitude Alanis, .Artificial neuron is the most basic processing unit of artificial neural network. The structure of artificial neuron is shown in The function of the artificial neuron is expressed as:In Equation (11), net represents artificial neuron processing.When ϕ is a threshold function, the output is expressed as:k represents the threshold of the kth neuron.In Equation (13), θAn artificial neural network is composed of a large number of neuron nodes connected by a specific connection method. The connections between neurons are different, and the resulting artificial neural network structures are also different. The structure of artificial neural network is mainly divided into: forward network and feedback network , λ represents the learning rate, and ΔConvolutional neural network is a network mechanism based on artificial neural network. Convolutional neural network has the characteristics of deep learning and is widely used in analysis of emotion recognition, image recognition, speech recognition and so on. The network topology of the convolutional neural network is shown in As can be seen from The convolutional data is subjected to the bias operation and then processed by the activation function to obtain the output data. The convolutional layer formula is expressed as:In formula (15), The subsampling layer is to sample and separate the input data and reduce the data dimension. The sampling dimension reduction process of the subsampling layer is expressed as:d represents the sampling dimension reduction function, and In formula (16), There is also a process of backpropagation in the learning process of convolutional neural networks. Take the reverse derivative of the fully connected layer:In formula (17) and formula (18), t represents the current convolutional layer, and t-1 represents the previous convolutional layer.During backpropagation, the transfer process from the convolutional layer t-1 to the t layer is:19)Nt−1 represents the number of features obtained in the t-1 layer.In formula (19), C represents the convolution calculation process, and Experiment and analysis on the psychological state prediction analysis of charismatic leaders based on artificial intelligence affective computing on employees' work attitudes.In order to better analyze the psychological state prediction analysis of charismatic leaders and traditional leaders on employees' work attitudes, this paper analyzes charismatic leaders with artificial intelligence emotional computing, while traditional leaders only rely on daily data and experience such as personal attendance. The selection of experimental samples is very important. Poor selection of experimental samples can easily lead to invalid experimental comparisons. The sample data needs to cover the entire sample as much as possible.This experiment will select 20 companies and divide the employees and leaders of the companies into two groups, namely the charismatic leadership group and the traditional leadership group. An extrinsic indicator of psychological state predictive analysis that affects leaders' attitudes toward employees' work. From the analysis of the data in When selecting the psychological state prediction and analysis indicators for evaluating leaders' attitude toward employees' work, it is necessary to carry out a correlation analysis between the above-mentioned influencing indicators and the psychological state prediction analysis of leaders' attitude toward employees' work , 50 people with salaries between ,  between 50 people. The experimental period was set to 6 months, and employees' trust in leaders was surveyed every month. Data reflects: Predictive analysis of the psychological state of charismatic leaders based on artificial intelligence emotional computing on employees' work attitudes will improve employees' trust in leaders, and the higher the employee's salary, the more obvious the increase in trust, and the greater the employee's confidence in charismatic leaders. The highest level of trust in leadership is 85%.The sense of mission of employees to achieve organizational goals is an important indicator to measure the psychological state predictive analysis of leaders' attitudes toward employees. Since male and female employees may interfere with the achievement, the experimental employees are separated by gender. The data in The employee's job satisfaction is the dominant result of the leader's psychological state prediction analysis of the employee's work attitude. A 6-month job satisfaction survey was conducted on three tiers of salary employees. The data in Predictive analysis of leaders' psychological state of employees' work attitudes can help leaders better understand employees' needs. The experiment will investigate leaders' sensitivity to needs of employees of different genders. The experiment will be conducted every 2 weeks for a period of 12 weeks. From the data analysis in Through a multi-faceted comparison of the psychological state prediction analysis of the two leaders' attitudes toward employees' work, the experimental results show that the charismatic leadership based on artificial intelligence affective computing has an important effect on employees' trust in leaders, employees' sense of mission to achieve organizational goals, and employees' commitment to work. Satisfaction, leadership sensitivity to employee needs are better than traditional leadership. Traditional leaders cannot accurately predict and analyze the psychological prediction of their employees, which leads to a series of problems of low work efficiency of employees such as low enthusiasm for work and insufficient trust in leaders, which leads to low economic benefits for enterprises. Charismatic leaders can use artificial intelligence affective computing technology to predict and analyze the psychological state of employees' work attitudes and improve the economic benefits of enterprises.Through the comparative analysis of 4 aspects of the psychological state of employees' work attitudes between charismatic leaders and traditional leaders based on artificial intelligence affective computing, the experiment shows that: (1) charismatic leaders and traditional leaders based on artificial intelligence affective computing The average job satisfaction is: 74.6 and 55.3%, respectively; the sense of mission of employees under charismatic and traditional leadership to achieve organizational goals is: 73.3 and 59.4%, respectively. (2) Charismatic leaders can use artificial intelligence emotional computing to analyze the psychological state of employees' work attitudes, which can make leaders more sensitive to employees' needs, and make employees trust leaders 15.2% more than traditional leaders. The psychological state prediction analysis of charismatic leaders and traditional leaders based on artificial intelligence affective computing on employees' work attitudes can make subordinate employees trust leaders more, improve work efficiency, and increase group performance. However, at present, the accuracy of artificial intelligence emotional computing's prediction of the psychological state of employees' work attitudes cannot reach a very accurate level. Therefore, improving the accuracy of psychological state prediction of employees' work attitude will be the direction of future research.The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.YL: writing-original draft preparation. JS: editing data curation and supervision. Both authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.