Friction is an inherent nonlinear disturbance that can lead to creeping, jitter, and decreased tracking precision in an electro-hydraulic servo system. In this paper, the LuGre friction model is used to describe the dynamic and static characteristics of the friction force of a servo system comprehensively. Accurate identification of model parameters is key to implementing friction compensation. However, traditional genetic identification algorithms have the shortcomings of a premature solution, slow convergence, and poor accuracy. To address these shortcomings, this paper proposes an improved adaptive genetic identification algorithm. The proposed algorithm selects evolutionary processes adaptively according to the population concentration in the initial stage of population evolution. Moreover, it adjusts the crossover probability and the mutation probability to identify a local optimum accurately and converge to the global optimum rapidly. During the late stage of population evolution, the accuracy of the global optimal solution can be improved by reducing the search range of identification parameters. The simulation results show that the relative error of the model parameter values identified by the proposed algorithm is reduced to less than 1% and the convergence speed is faster. Compared with the existing traditional genetic algorithm and adaptive genetic algorithm, the overall performance of the proposed method is better. This study provides a feasible and highly accurate identification method for parameter identification of friction models used in electro-hydraulic servo systems. The electro-hydraulic servo system has the advantages of high-power density, fast response speed, and high control precision, and is widely used in aviation, aerospace, and ships ,2,3. TheIn terms of reducing the impact of friction, it is more practical and cheaper to compensate nonlinear friction than pure mechanical methods, such as improving the contact surface material and lubrication conditions . For comThe LuGre friction model is composed of a set of nonlinear equations, including six parameters, namely, two dynamic parameters which are the bristle stiffness coefficient and damping coefficient, and four static parameters which are coulomb friction, static friction, viscous friction coefficient, and Stribeck velocity. These parameters are based on the experimental data of observable state variables and calculated by the identification algorithm. The acquisition of model parameters is the premise of the implementation of these friction compensation measures, and the more accurate the parameters are, the more effective the compensation is. However, since the coupling between the six different parameters as well as the internal state variables involved in the model are difficult to measure in the actual system, it is very difficult to determine the parameters of the LuGre friction model, especially the two dynamic parameters. In addition, due to the large number of parameters to be identified, the calculation cost also increases significantly, which also brings great difficulties to the parameter identification of the LuGre friction model. For this reason, many scholars have carried out relevant research. Xu studied a new identification method for determining the parameters of dynamic systems from step responses based on the Newton algorithm. The simulation results displayed that the obtained models can effectively capture the dynamics of the systems . CanudasIn addition to the above methods, among the current model identification algorithms, the intelligent algorithms represented by the simulated annealing algorithm (SAA) , the neuAlthough the genetic identification algorithm is not sensitive to initial value selection and has global search ability, the traditional genetic identification algorithm adopts fixed control parameters, resulting in it possessing a poor global search ability and easily causing a ‘premature’ phenomenon to occur, falling into local optimum ,26. In oBased on the above, a new approach for the parameter identification of the LuGre friction model based on the improved adaptive genetic recognition algorithm is proposed. The main contributions of this paper can be epitomized as follows:1. On the basis of the traditional genetic algorithm, combining it with the adaptive evolution module and the optimal solution accuracy optimization module, a new parameter recognition algorithm is proposed.2. Based on the simulation experiments, it is verified that the proposed algorithm can identify the six parameters of the LuGre friction model effectively. In order to verify the advantages of the proposed method, it is compared with the existing methods. The results show that the proposed method can significantly improve the accuracy of parameter identification on the basis of guaranteeing fast convergence speed.The rest of this paper is arranged as follows: σ0 is the bristle stiffness coefficient; σ1 denotes the microscopic damping coefficient; σ2 represents the viscous friction coefficient; cF denotes the Coulomb friction force; sF is the maximum static friction force; sv is the Stribeck velocity; and z denotes the average bristle deformation. Accurate determination of the six parameters σ0, σ1, σ2, cF, sF, and sv is the key to the study of friction compensation using the LuGre friction model.The LuGre friction model is a synthesis of the Dahl model and the Bristle model, which simulates the contact surface of two objects with countless tiny elastic bristles. The restoring force generated by the elastic deformation of the bristles is regarded as a friction force . This moz of actual systems is difficult to measure. Therefore, it is regarded as an intermediate parameter in the process of friction parameter identification. After discretization, Equation (1) can be converted to the following form:t0 is the sampling period and k is the number of sampling points.The average bristle deformation m, is taken as the research object. The object is placed on a fixed horizontal plane under the action of external load nF, which drives the hydraulic cylinder to produce displacement x. The friction force of the electro-hydraulic servo systems is fF.In order to study the dynamic characteristics of the bristle in the LuGre model at the adhesion stage without losing generality, the piston rod, and its connectors with mass According to Newton’s second law, the dynamic equation of the actuator of an electro-hydraulic servo system is given by:x is defined by The state parameter A(k), input matrix B(k), and output matrix C(k) are, respectively, expressed as follows:Combining Equations (2) and (3), the discrete state equation of the actuator of an electro-hydraulic servo system is given by:v(k) and driving force nF(k) of the observable measured hydraulic cylinder into Equations (2)–(7).The objective function in the optimization process is defined as follows:The fitness function is defined by:v(k) and driving force nF(k)  are substituted into the parameter identification model. An appropriate identification algorithm is used to calculate the friction parameters of LuGre friction model.Equations (2)–(9) constitute the parameter identification model of the LuGre friction model of an electro-hydraulic servo system. The collected observable data of velocity The GA is a global search algorithm that simulates the law of biological evolution. It selects individuals that can adapt to a defined environment for replication, crossover, and mutation according to the principle of survival of the strongest individuals to produce a new generation of individuals that are more adaptable to the environment. This process is repeated until the result finally converges to a single individual that is the most adaptable to the environment among all individuals, and this solution represents the global optimal solution to the considered problem.However, the traditional genetic identification algorithm adopts fixed control parameters, resulting in it possessing a poor global search ability and easily causing a ‘premature’ phenomenon to occur, falling into local optimum. To solve these problems, based on the traditional genetic algorithm, this section introduces an improved adaptive genetic identification algorithm consisting of two modules, the adaptive evolution module and the optimal solution accuracy optimization module. The proposed algorithm aims to solve many problems in the traditional genetic identification algorithms, such as premature solutions, slow convergence speed, and poor accuracy. The flowchart of the proposed algorithm is shown in Pm to make the algorithm able to avoid local optimums and obtain the global optimum. Then, the crossover is performed to reduce the crossover probability Pc. When the population concentration is lower than 0.5, differences within the population are large. Crossover can increase the crossover probability Pc to ensure the full evolution of individuals to generate better individuals. The follow-up mutation can reduce the mutation probability Pm, i.e., the probability of destroying high-quality individuals.The main function of the adaptive evolution module is to avoid premature solutions and accelerate convergence. Its specific calculation steps are as follows. When the population concentration is higher than or equal to 0.5, differences within the population are small, so the crossover is not effective in generating better individuals. Moreover, much computing time is consumed to execute the algorithm. Therefore, mutation should be first conducted to increase the mutation probability if is the fitness value of individuals in the population.The population concentration is defined as follows:k1 is the crossover coefficient.The crossover probability is calculated by:k2 is the mutation coefficient.The mutation probability is calculated by:j,if is the fitness value of the ith individual in the jth evolution; k0 is the number of evolutions of the population; m denotes the total number of evolutions; n is the total number of individuals in the population; δ, γ, and α are the starting coefficients of the optimal solution accuracy optimization module.When the solution obtained by the adaptive evolution module converges to the vicinity of an optimal solution, the optimal solution accuracy optimization module starts. The starting conditions are defined as follows:βn (0 < β < α) individuals with the highest fitness values in the population are the same, the genes at the highest positions of these βn individuals are released (emptied) to store in a decoding formula, and the genes of the rest positions are shifted to the left by one position, i.e., the q-th position is changed to (q + 1)th position, where q = 1, 2, …, (l− 1), and l denotes the individual length. The emptied-first (lowest) position is used to contain a new gene 0. Then, randomly generated n individuals constitute a new initial population. The adaptive evolution module is then used to calculate and generate optimal individuals in this generation.The optimal solution accuracy optimization module is based on the idea of zoom . The calr iterations of accuracy optimization of the optimal solution for the population, the decoding formula of the individuals is used as follows with coding a is the lower limit of the encoded parameter, b denotes the upper limit of the encoded parameter, and iu the gene position released at the ith time.After b − a) to Equation (14) shows that the optimal solution accuracy optimization module can narrow the parameter search range from  and hydraulic cylinder driving force nF(k). The parameter settings of the LuGre friction model are provided in A dynamic simulation model of an electro-hydraulic servo system was constructed using the AMESim software to simulate collected discrete data of hydraulic cylinder velocity To verify the validity of the proposed algorithm, the traditional genetic identification algorithm, the adaptive genetic identification algorithm, and the proposed (improved) genetic identification algorithm were used to identify the parameters of the LuGre friction model. The parameter identification results corresponding to different methods are shown in Based on the identification results, the following conclusions can be drawn:(1) Although the traditional genetic identification algorithm converges faster, it causes a ‘premature’ phenomenon and falls into a local optimum, causing inaccurate parameter identification results of the friction model and large errors compared with the actual (true) values;(2) Compared with the traditional genetic identification algorithm, the adaptive genetic identification algorithm effectively alleviates the ‘premature’ phenomenon, but the convergence speed is significantly slow. In addition, the identification results are very unstable in the iterative process;(3) Compared with the two existing methods above, the improved adaptive genetic algorithm proposed in this paper not only avoids the ‘premature’ phenomenon of the traditional genetic identification algorithm, so as to have more accurate identification results, but also enjoys faster convergence speed than the adaptive genetic identification algorithm. Therefore, the recognition performance of the proposed algorithm is better.The identification accuracies of the three different algorithms are compared in Based on the optimal parameters obtained by different optimization algorithms, the relation curve between hydraulic cylinder speed and friction force is obtained, as shown in In order to accurately and quickly realize the effective identification of the LuGre friction model parameters, based on the adaptive evolution module design and the optimal solution accuracy optimization module design, this article proposes an improved adaptive genetic identification algorithm to address the shortcomings of the existing parameter identification algorithms. Compared to the traditional genetic algorithms and adaptive genetic algorithms, the proposed identification algorithm can effectively avoid premature solutions falling into a local optimum and improve identification accuracy significantly. The simulation results verify that the proposed algorithm is effective and feasible and has an identification error of less than 1%. Based on the LuGre friction model and the improved adaptive genetic identification algorithm, the research results of this paper are expected to contribute to reducing the impact of nonlinear friction on the electro-hydraulic servo system. However, due to the influence of various factors, the performance of the proposed method is only preliminarily verified in the simulation model. In future research, our work will mainly focus on the following aspects:(1) We will try to apply the algorithm proposed in this paper to the actual electro-hydraulic servo system so as to verify its performance in practical engineering applications, and make further adaptive improvements to the algorithm for possible problems.(2) Although the convergence speed of the algorithm proposed in this paper is faster than the adaptive genetic algorithm, it is still significantly slower than the traditional genetic algorithm. The further optimization of the proposed algorithm so as to improve the convergence speed and make the algorithm more suitable for engineering applications is also the focus of future research.


Most robots are programmed to carry out specific tasks routinely with minor variations. However, more and more applications from SMEs require robots work alongside their counterpart human workers. To smooth the collaboration task flow and improve the collaboration efficiency, a better way is to formulate the robot to surmise what kind of assistance a human coworker needs and naturally take the right action at the right time. This paper proposes a prediction-based human-robot collaboration model for assembly scenarios. An embedded learning from demonstration technique enables the robot to understand various task descriptions and customized working preferences. A state-enhanced convolutional long short-term memory (ConvLSTM)-based framework is formulated for extracting the high-level spatiotemporal features from the shared workspace and predicting the future actions to facilitate the fluent task transition. This model allows the robot to adapt itself to predicted human actions and enables proactive assistance during collaboration. We applied our model to the seats assembly experiment for a scale model vehicle and it can obtain a human worker’s intentions, predict a coworker’s future actions, and provide assembly parts correspondingly. It has been verified that the proposed framework yields higher smoothness and shorter idle times, and meets more working styles, compared to the state-of-the-art methods without prediction awareness. Innovations in design and control technology are allowing various robots to complete complicated tasks in wide segments of our life. Increasingly sophisticated robots have been crafted to populate different areas. With the booming of computer science and artificial intelligence, social robots  utilize The traditional assembly process  can be eAs production trends are rapidly changing, the main trigger points in intelligent manufacturing  are starTo deal with these issues, the deployed robots must be able to change assignments easily and be able to work alongside  humans. Recently, many developers have improved HRC technologies  to meet Early research on the prediction of human motions was generally based on heavy handcrafted models to heuristically arrange the predefined processes. Mainprice et al.  proposedThere were various studies adapted from the statistics-related method in the human action prediction area. Hawkins et al.  designedSome works rely on the recognition of human gestures to classify and make reasonable predictions about the actions performed by a human. Luo et al.  used a tSome studies try to estimate human intention through various sensors other than vision sensors. Muthugala et al.  proposedDespite the different methods in various intention identification applications, the data source is also miscellaneous in human-robot collaboration. In general, wearable sensors provide more condensed information than vision sensors. Remote status monitoring provided by wearable technology has become an important tool in the industrial collaboration scenario. Furthermore, wearable devices can provide real-time feedback regarding collaboration conditions. However, wearable devices may introduce inconvenience during the assembly procedure, while vision-based methods are non-intrusive. The vision-based methods can offer natural, contact-free interactions. In addition, vision sensors have the advantage of deployment simplicity and low cost compared to wearable devices. Furthermore, vision-based methods can provide sufficient high-dimension data for the system and sustain flexibility with extensibility for future applications. Since the collaborative actions may vary spatiotemporally with assorted implications, the vision-based methods can enable human-robot collaboration that is difficult or impossible to fulfill with other modalities. In the meantime, the vision-based method can be enhanced through wearable feedback in multimodal settings.A comparison of the representative methods in human action prediction is presented in The main contributions of this work can be summarized as follows: (1) We propose prediction-based learning from a demonstration framework for robots to understand assignment descriptions and human customized working preferences toward improved human-robot collaboration. (2) We extend the ConvLSTM concept and develop a novel state-enhanced ConvLSTM model for robots to learn spatiotemporal dependency and predict long-term action dynamics efficiently in collaborative works. (3) We establish a tracking result guided CNN model to leverage the prediction knowledge to proactively assist humans according to their preferences. The remainder of this paper is organized as follows. After the discussion of the related work, we describe the proposed framework and necessary theory in The purpose of this paper is to predict a human worker’s intentions early and fulfill the human-robot collaboration properly for the assembly scenarios. The gist of the proposed framework is shown in The framework only shows the main data flow and information process pipeline, while omitting the ROS connection system  and roboThe proposed framework uses an end-to-end recurrent neural network to implement human demonstration learning and human intention prediction. Specifically, the ConvLSTM  was appltX is the input sequence at time t and equals the output from the encoder. W** and b** serve as the weight matrices and bias terms. σ indicates a sigmoid function, and tanh denotes the hyperbolic tangent function. ti, tf, to represent the input gate, forget gate, and output gate, respectively. *C and *H refer to the cell unit and hidden state. ‘*’ represents the convolution operator and ‘o’ represents the Hadamard product operator.The ConvLSTM layer can be explained by the equations below:H state and input X needed to be designed to keep more useful information in the challenging models. Compared to the fully connected LSTM , ConvLSTHt−1 and tX richer to keep persistent knowledge between network layers and reduce information loss as the time sequence expands. Similar to the filter process, we take advantage of the convolutional approach to build a feature map and constrain the network hidden states with this map. In this way, more crucial spatiotemporal information will be sustained in the model during the sequence processing.We add some extra linkages in the model to extract the essential relations for the long-time forecasting performance. The linkages make the coupling of Ht−1 is the hidden state in the previous equation, and all the following Ht−1 parts are replaced with the state-enhanced xhW and xhb serve as the corresponding weight and bias terms. ‘o’ represents the Hadamard product operator as before. Other terms remain the same as in Equation (1).Specifically, the linkage is described in the following equations:As the data flow through the ConvLSTM network, more and more details fade away due to the information decay. We add the additional operations to learn more global and local patterns in the input data which have temporal properties. Then, the input-related feature map treated by the Hadamard product with the hidden state also helps to improve prediction performance since the operations can be considered as the regularization in the sequence processing. This can extract spatiotemporal details more effectively using combinations of input and state data flow. The state enhancement procedure can be deployed multiple times like the convolutional layers. In order to improve the efficiency of the proposed model, a convolutional encoder  is used Xt1,…, represents the feature maps extracted from the convolutional encoder. I1, I2, …, tI represents the input sequence. nH and nC represent the hidden and the cell states of ConvLSTMs with state enhancement. Yt+1, Yt+2, …, t+TY stands for the predicted frames from the convolutional decoder. This model can be tuned by applying different layers of encoder/decoder and ConvLSTMs in accordance with various situations. The complete learning and prediction framework can be explained by Equation (3):An extended Kalman filter (EKF)-based tracking algorithm is proposed here to track the human intention part. The Kalman filter  has beenk − 1 to the current time step k. uk−1 is the driving function, and f is the non-linear function. kQ is the process noise covariance. kA and kW are the process Jacobian matrix at step k. The following three equations show the correction process for state and covariance estimates with the measurement update. kP is the estimation error covariance. kH and kV are the measurement Jacobians at step k, and kR is the measurement noise covariance at step k. h stands for the non-linear measurement-related function. kz represents the measurement, and kK represents the Kalman gain.The EKF equations are shown below:Besides the non-linear estimation problem, there are several challenges for the intended object tracking in the collaboration scenario. For instance, the position of the intended object may be measured with error, the intended object may be occluded partly by a human hand, and a new intended object may appear in the tracking area. To address these challenges, the EKF was utilized in our tracking system. After initialization and parameter tuning, the system will make a state prediction based on previous parameters and the model. Then, the predicted position and the measured position will be combined to update the system. Meanwhile, the EKF will change the weight ratio depending on the uncertainty. This process repeats as the tracking evolves. The EKF-based tracking system outputs the tracked position of the former prediction robustly. Moreover, the intended parts can be cropped out according to the tracking position. Once these images of intended parts are available, the next step is to recognize the specific part and prepare the robot to collaborate with the human in the assembly scenario. During the tracking process, the type and the shape of the intended part could be difficult to handle for the traditional methods . A convoThe model accepts tracking input images to abstract the feature map with the convolutional operation. With the process of several convolution layers and pooling layers, features of different levels can be captured for further computation. These features are fed into the fully connected layers for classification. The probability of each class is calculated by the softmax function. Finally, the model uses the cross-entropy loss function to evaluate the consistency of probability and the objective function.X stands for the input of each convolutional layer, W and b stand for the weight and bias terms to be trained for each layer. Y′ represents the feature map output from the convolutional layers and pooling layers. q represents the probability of the output class. p shows the objective one-hot vector for the cross-entropy loss function. As for the activation function, we use a rectified linear unit (ReLU) in the model. To improve the efficiency and performance of the model, the batch normalization approach [The process can be explained in the following abbreviated equations:approach  is addedGenerally, moving parts recognition can be particularly tricky while facing complex assembly situations. The learning CNN model can ease this dilemma with its decent generalization performance. The final collaboration decision can be settled conveniently for different task arrangements and various assembly preferences.In the human-robot collaborative assembly process, robots are required to assist humans dynamically. Therefore, in the context of the assembly, robots controlled by rigid codes are undesirable due to the incapacity to adapt to changing situations and the need for expert knowledge. We proposed a learning from demonstration based model to handle this problem. The robot can be programmed by human workers without expert knowledge. The collaborative task arrangement can be transferred to the robot in a natural and expressive way. The diagram of this model is shown in In the demonstration phase, the human worker acts like an experienced teacher for the robot. During the human-robot interaction process, the complex tasks and customized assembly styles may be difficult to be defined as a known function, but they can be demonstrated directly. The robot collects the assembly information from the sensors while the human performs the assembly assignments. With the human demonstration, the robot will learn what is going on in the shared workspace. As in the learning process, the robot learns the task details implicitly and analyzes the optimal actions naturally without increasing the programming burden. This process will eliminate the need for expert knowledge-oriented and time-consuming methods. The robot accomplishes a complex mapping to the human demonstration considering the constraints from the shared workspace and the requirements from the assembly actions. After the robot obtains the fundamental experience, various applications can be realized in different scenarios. Here, we will focus on the assembly collaboration circumstance, i.e., how to make early plans to assist humans by delivering parts at the right time. In this project, a state-enhanced recurrent neural network based model is introduced to learn the task description and arrange proper executions to cooperate with the human worker while improving the fluency and safety of the human–robot collaboration.In this work, the signal was collected from the collaborative research platform, as shown in Humans can gather assembly information through their eyes and efficiently figure out the proper manner to cooperate with other coworkers. Hence, the natural way to collect assembly information for the robot is through cameras. The main purpose of this study is to build a prediction-based human–robot collaboration model for assembly manufacturing. During the assembly process, humans will make corresponding actions according to the assembly goal in the shared workspace. The robot should focus on the ongoing actions within the workspace and the motion of human hands  is a sigIn this project, the human and the robot are working together to assemble a vehicle’s seats in a real industrial context. The participants are required to cooperate smoothly by conducting a series of tasks. The robot participant carries out the part hand-over tasks and the human worker performs the assembly job. Sub-task 1: install the left seat holder;Sub-task 2: install the right seat holder;Sub-task 3: install the back seat holder;Sub-task 4: install the left seat;Sub-task 5: install the right seat;Sub-task 6: install the back seat.The task employed in this experiment derives from real industry assembly cases. The assembly task contains several sub-tasks; the detailed instructions are shown in Sub-tasks 1–3 are independent of each other, which means Sub-tasks 1–3 can be carried out regardless of order. Sub-tasks 4–6 are independent of each other likewise. However, Sub-tasks 1–3 and Sub-tasks 4–6 are dependent so Sub-tasks 1–3 need to be finished before Sub-tasks 4–6 are carried out. After the completion of Sub-tasks 1–3, Sub-tasks 4–6 can be accomplished based on the human worker’s preference. In this project, the human and the robot cooperate by installing and delivering particular parts to accomplish the seat assembly. The human worker conducted and repeated the assembly procedures to demonstrate assembly preferences. The robot observed and learned the ongoing action intentions at the beginning phase. Subsequently, the robot constructed the collaboration knowledge and inferred the predicted actions. Afterward, the human worker sought the next part needing to be installed from the robot implicitly. The human worker reached out for the part when it was delivered by the robot, and proceeded with the following task. All the parts requirements and the delivery timing were conveyed indirectly, and the robot fulfilled its job without explicit commands. The assembly variances and noises were diminished by the prediction system. In other words, the robot needed to figure out the human worker’s collaboration hints and correctly respond in line with the human worker’s future intentions. The framework of this project is expandable and not bounded by the types of these tasks. Step 1: The human worker starts to install the left seat holder on the chassis;Step 2: The robot picks up the right seat holder and delivers it to the worker;Step 3: The human worker finishes the installation of the right seat holder;Step 4: The robot picks up the back seat holder and hands it to the collaborator;Step 5: The human worker concludes the installation of the back seat holder;Step 6: The robot grabs the left seat and passes it to the human worker;Step 7: The human worker finishes the installation of the left seat;Step 8: The robot picks up the right seat and hands it to the human worker;Step 9: The human worker completes the installation of the right seat;Step 10: The robot grasps up the back seat and provides it to the human worker;Step 11: The human worker conducts the installation of the back seat.In general, the task allocation in the collaboration depends on many factors, such as the attributes of the parts and the capabilities of the participants. In this case, the assembly process is adapted to establish robot-assisted collaboration. That is, the human worker handles the assembly maneuver in a guiding role and the robot compliantly handles the delivery of the parts. The installation of seat holders and seats is fulfilled by human workers, while the corresponding components are reached for and delivered by the robot. For instance, a possible collaboration sequence could be listed as: It should be noted that this experiment setting is designed for robot-assisted collaborative assembly. In this assembly assignment, the human worker governs the tasks in general while the robot mainly follows the human’s lead. This specific setting is established based on the current SME assembly scenario and customized human comfort considerations. Within this setting, the robot makes the subtle action prediction, provides essential assistance, and improves assembly efficiency. Our experiment setting takes the part prediction and delivery as the key collaborative action. The human worker needs 6 parts to accomplish the seat assembly task, which means 6 collaborative action predictions in one assembly instance. Each assembly instance takes 3–4 min to complete, with some individual variations. In addition, the human worker can focus on the assembly task and save the cost of robot action prediction. This setting also benefits the collaboration safety in some way.The structure of the control system is shown in We designed the robot control system based on the MoveIt! package which is integrated with the ROS. The control system allows the robot to construct the workspace environment setting, generate motion plans, and carry out the motion plans according to the change in the workspace. In the human–robot collaborative assembly assignments, the path planning performance directly affects the collaboration quality. The Open Motion Planning Library (OMPL)  in the RWe conducted several delivery tests to compare the performance of different planners on the robot. Given the same task in the sharing workspace, the robot runs different planners multiple times to verify the planner’s performance. Finally, considering the planning path length, computation time, and execution time, we choose the fast marching tree (FMT)  planner During the assembly procedure, the robot drives its arm from the initial position to the inventory position, then picks up the desired part and delivers it to the human at the appropriate position, and eventually returns to the initial position. The grasp orientation in the part picking up position is set to downward, while the initial position remains the same during the assembly process. The robot’s motion paths are generated by the planner FMT. Safety for humans in our collaborative assembly procedure is ensured by several strategies and methods. First, we impose speed limits on the robot and ensure that it stays out of the human safety zone. When moving close to the human, the robot is allowed to move partially at a speed that is slow enough to raise no risk to the human worker. This collision avoidance strategy fits our project well enough and provides greater flexibility than the global constrained safety methods. In addition, collision avoidance is not always possible in the assembly scenario. Safety is a built-in feature of the robot we use. The soft padded arms and floating-plastic-covered skeleton combined with force-sensing control ensure the safety of human coworkers in the shared workspace. The material wrapped in the robot absorbs the unexpected impact, and the collision sensing system maintains the power and force below injury thresholds. Once the collisions were detected, the control commands to stop the robot were executed immediately. Furthermore, to deal with unpredictable events, emergency stops are equipped within easy reach in the shared workspace. If the emergency stop is triggered, the entire system will be overridden and shut down. The safety of human-robot collaboration is the top priority and all the strategies are just a small part of the safety-related system developed. In this part, we describe several experiments to evaluate results for the proposed model in the learning from demonstration, the predicted actions generation, and the following human-robot collaboration in the unlabeled assembly manufacture process. Then, we compare the performance of the proposed model with the speech commanded method to demonstrate the efficiency and smoothness of our strategy in the industrial human-robot collaborative assembly scenario.The model uses visual data as the input features to learn spatial and temporal information. The model was trained with our human demonstration dataset. During the demonstrations, human workers perform the parts assembly process for the robot without explicit definition. The dataset is separated into the training part and the testing part. The testing dataset was used to test the accuracy and generalization of the proposed model. The performance was evaluated by predicting actions of different lengths from input video clips. With the proposed model, the robot can monitor the human worker’s actions, learn the human worker’s intentions from the movements, and understand the ongoing assembly tasks.In the model training experiments, the training results of different configurations are compared. The loss of the training process is shown in In order to validate the learning and prediction performance of the proposed model, we applied another unknown demonstration to examine and generate the predictions. It can be seen that the first predicted actions are the back seat holder being installed after the right seat holder assembly in the long-term prediction, as shown in Likewise, the second actions are the right seat being assembled following the left seat installation. As can be seen from The clear prediction results prove that the model can make accurate predictions during the assembly process without predefined tasks.After obtaining the prediction output based on the neural network model, the potential human intention is also embedded in the predicted action sequences. The human intention has to be converted into the equivalent robot control command for the collaboration. Hence, for the future human–robot collaborative assembly, the specific part needs to be tracked and recognized at the correct time step.Since the human intentions are nested inside the assembly motion of parts, we applied the EKF to track the intended part in the prediction motion sequence. For instance, the left seat is intended to be installed following the installation of the back seat holder. The EKF method can make stable estimations of the tracked parts by weighting the measurement and estimation. It uses the measurements to readdress the estimation states for the filtered tracking, and output the estimation based on early states if no measurements occurred. The tracking in our experiment is based on the part motion within the shared workspace, thus the misplacement of the parts is under control and the tracking errors are relatively small. In addition, more complex motion models and attributions of other parts can be used to reduce the tracking errors if needed.Since we already concluded the future action sequence and captured the intended assembly part, obviously this precise part should be recognized in the next step and fed into the robot control system for preparation of assembly assisting executions. Before that, we need to build a large dataset for the following neural network’s training procedures. When we want to apply machine learning models in real industry scenarios, the desired data resource plays a crucial role in these data-driven  approachFor this specific assembly scenario, the potential data-driven model shares the same challenges. First and foremost, enough data need to be collected effectively. Here, we take advantage of the previous tracking model to alleviate the pain of data collection. During the tracking process, we tracked a lot of images for desired parts in the prediction video sequences, these images contain the independent parts in an organized way. In addition, the travel behavior of these parts is slightly different, showing different angles and positions in the tracked images. These various images can be cropped out during the intermediate tracking process, as shown in After gathering the raw data from the shared workspace, the samples already covered all use cases and vital information. Generally, there is plenty of work that needs to be done in the data labeling  process.During the prediction tracking process, the images of each tracked part are mapped to a specific label that corresponds to the correct part name for the following assembly action. As in the prediction tracking process, the images of each tracked part in the demonstration tracking process are assigned to the corresponding part name as well. Here, we treat the demonstration input as the prediction output cautiously. Each prediction yields a series of tracked images, and these images may not follow the normal assembly sequence, but each part and its labels do not change at each step of the assembly process. Therefore, we only need to focus on the beginning and end of each image series. If the beginning and ending parts are the same, then the whole image series can be labeled without hesitation. If the beginning and ending parts are different, then it indicates that a new part was assembled after the previous part was assembled during the assembly process, and both parts should be labeled individually. After the tracking and label process is finished, the dataset can be used to train a CNN to recognize the predicted part. The label represents the human intention in the next assembly step. When the human operator is assembling the product, the predicted images are sampled and all the images will be converted to labels, and these labels will be transferred to the robot to carry out a real collaboration task in the subsequent process.The robot needs to learn through enough data to obtain the correct part based on human intentions. The model structure and corresponding parameters are displayed in In the experiment, the model was trained by different dataset configurations, one with the demonstration dataset and one with the combined dataset. We use 70% data to train, 20% data for validation, and 10% data to tune the CNN model in the same way. The samples shown in We compare the performance of the model trained only from the demonstrations dataset with the model from the combined dataset. The model was trained for 350 epochs, and the validation was carried out every 1000 iterations. With each saved best model, the performance test procedure can be fulfilled. The recognition accuracy of the selected model is higher than 99% during the following test scenarios. This model can handle human intention parts recognition successfully for the prediction actions in our assembly collaboration task. The specific intention part will be fed into the robot control system during the following collaboration process.After the completion of learning from the human demonstration process, predicting human future action, and obtaining the correct required part, the human-robot collaborative assembly routine can be established. While launching the human-robot collaborative assembly procedure, the proposed model is deployed upon ROS to assist the human worker to install vehicle seats. After learning the customized assembly information from the human demonstration, the robot infers the human worker’s future intention and provides the corresponding required part. The intended part’s information, such as location and destination, is transferred to the robot’s controller. Based on that, the robot finds the part in the inventory, plans the proper motion trajectory, grasps the correct part, and delivers it to the shared workspace. The collaborative vehicle seat assembly procedure is shown in From In a real assembly scenario, the same product can be assembled in a different style according to the human worker’s preference. We also test the model on a different assembly option in our experiment. The result is shown in As demonstrated in To verify the performance of the early prediction-based approach, we have implemented a more straightforward method in the same assembly scenario. In a human-robot collaborative assembly context, the human worker can express his intention directly through natural language. The robot can interpret the human worker’s intentions through language and then deliver the corresponding part to the human. For this setting, the natural language commands are the bridge between robot and human worker. This is similar to the human workers’ cooperation procedure. A human gives the speech instruction to the collaboration partner, the partner understands the instruction and arranges the corresponding action, then the partner executes the action. In this situation, the human worker serves as the leader and delegates the desired task to his partner, the assisting robot. We adopted the Google Cloud speech recognition platform  to proceHowever, after receiving the human’s instruction and planning the potential action, the robot locates and delivers the required parts within a relatively longer time span than the prediction-based method. The actual intention will be fulfilled after the whole commands are interpreted by the robot and the trajectory optimized by the robot planner. Furthermore, the human worker needs to pay extra attention to things other than his own tasks. This additional work introduces a tedious burden in the collaboration, which is against the safety and comfort expectations in the long run. We invited another six individuals  within an age range of 21–35 as participants in the experiments. The participants included students and researchers with non-directly related backgrounds. Each of them collaborated with the robot to conduct the assembly tasks twice using the proposed prediction-based HRC approach and speech commanded approach. Data from these collaborations were collected and exploited for further model performance evaluation. To evaluate the fluency of the human–robot collaboration, we compare the human idle time and robot idle time  between Collectively, it appears that the robot idle time is much lower than the human idle time, suggesting effective use of the robots. The higher human idle time indicates that relevant issues such as human safety and comfort are considered in task allocation and can be reduced in future studies. Moreover, compared with the speech commanded approach, the HRC approach reduced human and robot idle time values by 34.5% and 58%, respectively. This indicates that, although the human idle time is reserved for safety reasons, the proposed prediction-based approach achieves fluent collaboration and efficient resource utilization in the collaborative assembly scenario.Here, we compare the total task finish time between the prediction-based HRC approach and speech commanded approach, as shown In Early prediction plays an important role in human-robot collaboration. This paper presents a prediction-based framework to understand, learn, and assist human coworkers in the HRC assembly scenario. The proposed model was implemented on a robot to help in the manufacturing assembly process by delivering parts for installation. Experimental results show that the model is effective in learning from demonstration, and the robot can predict the human partner’s future intentions and then naturally deliver corresponding parts to assist humans proactively in real time. The early prediction-based intention aware robot has better cooperation fluency, shorter suspended time, and higher collaboration comfort, compared to the traditional passive commanded one. These outcomes verify the effectiveness of the human intention prediction-based framework and demonstrate the importance of future task awareness in human-robot collaboration. In addition, the predictions inferred from human demonstration can be customized to different human partners according to their unique working styles. Compared with existing methods, the developed approach can help alleviate the need for heavy handcrafted models and complex settings. This model also proposes a potential way by which human coworkers can personalize the robot and let the robot assist them in the way they prefer. The proposed framework can serve as a core module in future human-robot collaboration applications and extend the exercise of more intelligent robot systems. There are limitations in the proposed method for predicting human intentions in some occlusion scenarios. The prediction results are affected by the limited workspace due to the view field of the camera. For instance, human workers sometimes need to clean the parts in areas that may not be covered by the camera during assembly manufacturing. Further developments of this work include more complex practices in industrial applications, a wearable-sensor-equipped system that is less susceptible to occlusions, and more methods to ensure safety and improve efficiency simultaneously.


Shadow detection plays a very important role in image processing. Although many algorithms have been proposed in different environments, it is still a challenging task to detect shadows in natural scenes. In this paper, we propose a convolutional block attention module (CBAM) and unsupervised domain adaptation adversarial learning network for single image shadow detection. The new method mainly contains three steps. Firstly, in order to reduce the data deviation between the domains, the hierarchical domain adaptation strategy is adopted to calibrate the feature distribution from low level to high level between the source domain and the target domain. Secondly, in order to enhance the soft shadow detection ability of the model, the boundary adversarial branch is proposed to obtain structured shadow boundary. Meanwhile, a CBAM is added in the model to reduce the correlation between different semantic information. Thirdly, the entropy adversarial branch is combined to further suppress the high uncertainty at the boundary of the prediction results, and it obtains the smooth and accurate shadow boundary. Finally, we conduct abundant experiments on public datasets, the RMSE has the lowest values with 9.6 and BER with 6.6 on ISTD dataset, the results show that the proposed shadow detection method has better edge structure compared with the existing deep learning detection methods. Shadows exist in most scenes in our daily life, which are shielded by light sources. Shadows can preserve important information about dynamic scene and objects, such as detection of buildings and vegetation areas, and detection of clouds through shadows in satellite images. On the other hand, shadows are also a major source of error and uncertainty  in the i − th image. fk is the corresponding domain classifier.where Hierarchical domain adaptation ensures that the intermediate features between the two domains have similar distribution, thus enhancing the robustness of the adaptation model. In the process of shadow detection, eliminating the data deviation between domains can improve the accuracy of shadow detection on the target data set. As shown in For computer vision tasks, the attention mechanism plays the role of generating weights for each pixel of the image. Ideally, the weight of foreground pixel will increase and the weight of background pixel will decrease gradually. Through the widening of the weight gap, the effect of different semantic separation will be achieved.Convolutional Block Attention Module (CBAM) is a reliable attention mechanism algorithm in computer vision tasks, which has a simple algorithm structure and considerable practical effect. Convolutional block attention module combines the space and channel of CNN to generate respective attention for images and feature maps of different attention domains, and guides the model to distinguish semantic information more efficiently.Convolutional block attention module is composed of spatial domain attention generation module and channel domain generation module, and the two modules need to be combined by weighted sum operation. Where, the space domain generation module can be expressed as:F output by the CNN, the global average pooling and global maximum pooling operations of the feature map are carried out simultaneously. Then, the results of the two pooling methods are connected based on channels, and a convolution network with the number of target channels is 1 and the convolution kernel is 7 × 7 is input. The number of channels is reduced to 1 without changing the length and width of the feature map. Then the activation function Sigmoid is used to transform the output into nonlinear data, and the spatial domain attention matrix Ms(F) is obtained. According to the feature map The channel domain generation module can be expressed as:F. Then the results of the two operations are respectively input into the same multi-layer perceptron, and the two vectors are directly added together. It inputs Sigmoid activation function, and outputs channel domain attention matrix Mc(F).In the attention module of channel domain, the average pooling and maximum pooling operations based on channel are carried out synchronously in the feature map The existing shadow detection data sets lack the soft shadow images with rich scenes because of the single acquisition method (using various shielders under the strong light source). Affected by the intensity of light source, soft shadow image does not have clear shadow boundary. However, many existing deep learning methods are unable to obtain good detection results on soft shadow images. As shown in In order to solve the above problems, the boundary adversarial branch model is constructed to predict the boundary structure results in the target data set. Boundary adversarial branch is designed to generate the shadow boundary image as shown S. Its label data is the ground truth mask ys. The target dataset T has no labeled data. Firstly, the generator Gb fits the shadow boundary in the image and generates boundary prediction results Gb(xs) and Gb(xt) for the light source shadow image xs and target shadow image xt, respectively. The visualization is shown in Db is designed to determine whether the boundary comes from the source or the target dataset. With the boundary adversarial branch, for the soft shadow image in the target domain, it can accurately identify the shadow region, as shown in Assume that the source data set is Db judges and punishes Gb(xs) and Gb(xt), respectively, as shown in Equation (8):For the source domain data set and target domain data set with domain label, boundary discriminator LB is binary cross entropy loss, which is defined as LB = −(ylnŷ + (1 − y)ln(1 − ŷ)); N and M are the number of images in the source data set and the target data set, respectively.where LGb of the generator is a weighted combination of the mean absolute error loss term on the source data set and the adversarial loss term on the target data set, as shown in Equation (9):The loss function where After the boundary adversarial branch, using the additional shadow mask generator directly generates zigzag shadow detection boundaries for the target dataset . InspireIn order to suppress the uncertain prediction results, the entropy adversarial branch first generates the shadow probability map for the shadow image. Based on the probability map, the Shannon entropy is used to transform the probability map into the entropy map. Entropy maps of the target domain and source domain are forced to be as similar as possible, so as to reduce the effect difference between the model on the target and source data sets. Finally, the quality of the generated image is improved by the idea of adversarial learning. The high entropy value in the entropy graph should only be around the shadow boundary. The reasonable entropy distribution corresponds to the shadow detection results with smooth boundary.Gm generates mask prediction results Gm(xs) and Gm(xt) for source and target images, respectively. Given the mask prediction result p of input image x, Shannon entropy can be used to calculate the entropy graph, as shown in Equation (10):Mask generator De aims to calibrate the distribution of E(xs) and E(xt). Similar to the boundary-driven adversarial learning, the entropy discriminator De determines whether the entropy graph comes from the source domain or the target domain. Its objective function is shown in Equation (11):Entropy discriminator LGm of the generator is a weighted combination of the pixel-level cross entropy loss on the source data set and the adversarial loss item on the target data set, as shown in Equation (12):The loss function where Firstly, the coherence block matching strategy is used to find the best matching non-shaded region block for each region block in the shaded region. Then, local illumination propagation and global illumination optimization were performed for each matching shaded and non-shaded area pair. Finally, the shadow boundary is processed to get the final result.A. Local illumination propagationThe shadow area is modeled and combined with the illumination propagation algorithm, the ratio of direct light and indirect light is calculated to obtain the video without shadow:Ii is the i − th pixel of the image in RGB space. ki ∈  is the degree of direct illumination of the pixel. Both Ld and Le are vectors of scale 3, representing the intensity of direct light and ambient light. Ri is the reflectance of the pixel, which is also a three-dimensional vector, and each dimension corresponds to a color channel of the RGB image. Equation (13) indicates that the pixel value of a pixel is obtained by the interaction of direct light and ambient light and multiplied by the reflectance of the pixel. The state of direct light in the image can be divided into three situations, namely, shaded area, non-shaded area, and semi-shaded area. When ki = 0, the pixel is not affected by direct light and belongs to the shadow area. When ki = 1, direct light completely acts on the pixel, and the pixel belongs to the non-shadow region. When ki ∈  is the shaded transition region.where Then for a pixel with the shadow removed, the relationship between its pixel value and the pixel value of the shadow pixel can be simplified as Equation (14):Ii is the RGB value of the i-th pixel of the original image.where B. Global shadow removalAlthough the local illumination propagation operation can remove the shadow in the shadow area block, it can not get the spatio-temporal coherent shadow free video result. After the local illumination propagation, the global shadow removal method should be used to make up for this deficiency. In order to obtain spatially coherent unshaded images Ns(p) is the block containing pixel p. wi = dist is the similarity distance between block si and its corresponding block Lj. Where NL(p) is the block set formed by the nearest nNs(p) corresponding to the bright region. p through both block Si and block Li. In summary, the pixel value of pixel p in the overlapping area is obtained by the weighted average of multiple blocks containing pixel p.where Using the above global optimization technique, spatially smooth shadow-free results can be obtained within the shadow. Using weighted average method to calculate the shadow free value of overlapping pixels can avoid or greatly reduce the fuzzy artifacts in the overlapping area. Minimizing the objective function ensures that the results are consistent in time.Vf is the final shadow-free result. S represents the shaded area. u(p) and v(p) are the forward and backward optical flow direction of pixel p. where The proposed network structure adopts encoder-decoder structure of U-Net , Balance Error Rate (BER), and Per pixel Error Rate (PER) to evaluate the proposed method.TP, TN, FP, and FN are the correctly detected shadow pixels, correctly detected non-shadow pixels, wrongly detected shadow pixels and wrongly detected non-shadow pixels, respectively.where Because existing shadow detection methods only have good performance on source data sets, a novel shadow detection method is proposed in this paper. The method aims to obtain the same accurate detection results on the target data set as on the source data set. Firstly, combined with the hierarchical domain adaptive feature extraction method, a domain classifier is added after each convolution layer in feature extraction process to reduce the data differences between domains, and thus it improves the robustness of the model. Secondly, boundary adversarial branch and entropy adversarial branch are used to obtain smooth boundary detection results. Compared with the most advanced shadow detection methods, the proposed method not only has a great improvement in the source domain, but also has advantages in the target domain. In the future research work, we will consider to further improve the performance of the model from the perspective of generating diverse shadow image data.The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding authors.All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.Key Scientific Research Project of Higher Education in Henan Province, Teaching Science and Technology [2021] No. 383. Project Number: 22B510017. Project name: Indoor positioning research of quadrotor UAV based on MULTI-sensor fusion SLAM algorithm.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


One of the causes of bleeding in subdural hematoma is cortical artery rupture, which is difficult to detect at autopsy. Therefore, reports of autopsy cases with this condition are limited and hence, the pathogenesis of subdural hematoma remains unclear. Herein, for the detection and morphological analysis of cortical artery ruptures as the bleeding sources of subdural hematoma, we used the tissue-clearing CUBIC  method with light-sheet fluorescence microscopy and reconstructed the two-dimensional and three-dimensional images. Using the CUBIC method, we could clearly visualize and detect cortical artery ruptures that were missed by conventional methods. Indeed, the CUBIC method enables three-dimensional morphological analysis of cortical arteries including the ruptured area, and the creation of cross-sectional two-dimensional images in any direction, which are similar to histopathological images. This highlights the effectiveness of the CUBIC method for subdural hematoma analysis. Cerebral contusion or bridging vein rupture resulting from severe head trauma is the main source of bleeding in acute subdural hematoma (SDH) ,2,3. EvePrevious reports have indicated that most cortical artery ruptures were macroscopically missed at autopsy ,52; hencMany researchers have developed various tissue-clearing methods since the high-speed three-dimensional (3D) imaging of cleared mouse brains by light-sheet fluorescence microscopy (LSFM) was first reported in 2007 . EmerginThe clear, unobstructed brain/body imaging cocktails and computational analysis (CUBIC) method combined with LSFM is a water-based tissue-clearing and imaging technique ,58. It iWe aimed to determine whether the CUBIC method with LSFM is suitable for the 3D imaging of cortical arteries in the human brain and morphological analysis of cortical artery ruptures. In this study, we evaluated the usefulness and limitations of the CUBIC method for the detection and morphological analysis of ruptured cortical arteries. Comparing this method with conventional macroscopic and histological examination, we found that the CUBIC method has advantages over conventional methods in the investigation of arterial rupture as a source of bleeding in SDH.This study was approved by the Research Ethics Committee of Niigata University (No. 2018-0224). The autopsies in this report were performed in accordance with the proper procedures under Japanese law, and all procedures were performed in accordance with the Declaration of Helsinki Principles. Since this study required past forensic autopsy records, and yet it was not possible to obtain informed consent for the use of the records from the subjects’ legal guardians, the Ethics Committee waived the need for written informed consent in this study. This decision was made in accordance with the “Ethical Guidelines for Medical Research Involving Human Subjects ,” Chapter 5, Section 12-1 (2) (a) and (c).Among the forensic autopsy cases performed in our institution from 2017 to 2019, there were six cases of SDH  cases) with cortical artery rupture detected histopathologically. All of these cases were included in this study .Based on our previous report , we idenTo clear the brain block samples containing cortical arteries using the CUBIC method, CUBIC reagents were prepared according to previous reports ,63. The The block of tissue was made transparent using the CUBIC method . Each fiEach sample with RI matching was immersed in an oil mixture (RI = 1.525), as previously described , and imaAfter LSFM imaging, the tissue was made detranslucent by PBS washing, and serial sections were prepared with Elastica van Gieson staining. Histopathological images were obtained under a light microscope  to detect arterial rupture.Image analysis was performed using a computer equipped with Imaris software . From the image data obtained by LSFM, we created 3D and cross-sectional images of arterial rupture regions identified by histopathological examination and compared them with magnified macroscopic and histopathological images.In cases 1–5, histopathology confirmed that arterial ruptures had occurred at or near the root of branches that originate from a cortical artery near the Sylvian fissure. Using the CUBIC method, we were able to visualize the arterial rupture clearly in four cases. In case 5, the rupture was out of CUBIC imaging range. In case 6, histopathology confirmed a small arterial tear in the focal contusion contained a hematoma, which interfered with LSFM imaging of the tore artery. The results are summarized in In cases 1 and 2, magnified macroscopic observation disclosed a defect in the cortical arterial wall with a small arterial branch, which was visualized in more detail in the CUBIC 3D images. In case 1, the CUBIC cross-sectional imaging reconstructed the morphology of the ruptured area with the same level of detail as the histopathological image  .In case 2, CUBIC cross-sectional images reconstructed the morphology of the ruptured cortical artery trunk as well as the histopathological image but indistinctly demonstrated the torn branch that was evident in the histopathological image .In case 3, histopathology revealed two ruptures in one cortical artery: two thin arteries branching off from the cortical artery, one ruptured at its root and the other ruptured immediately after further branching. However, magnified macroscopic observations could not reveal the rupture or even the two small arterial branches due to the agglutination of bleeding on the surface. Clearly, 3D-CUBIC imaging showed the presence of two arterial branches near the cortical artery that lacked continuity with the cortical arterial trunk: one ruptured at its root and the other ruptured immediately after further branching. Anatomically, each rupture forms two edges of the ruptured artery: one on the proximal  side and the other on the distal  side. Therefore, this case had four edges of the ruptured artery, all of which were detected in separate histopathological sections. The CUBIC images not only showed the same ruptures as the histopathological images, but also integrated all the rupture end sections into one image, which enables the observer to understand the interrelationship of the rupture sites .In case 4, magnified macroscopic observation did not reveal the rupture of the cortical artery or small branch, despite bleeding deposits not being detected at the rupture site. Clearly, 3D-CUBIC imaging showed the cortical artery ruptured at the root of a small branch. The CUBIC cross-sectional image showed not only the ruptured cortical artery at the root of the small branch but also the arachnoid tear and the small nodule near the rupture as it appears in the histopathological section .In case 5, arterial rupture was histologically detected outside the imaging range of CUBIC, so rupture findings could not be compared with CUBIC images.In case 6, macroscopic observation revealed a focal contusion in the cerebral cortex of the left temporal lobe near the Sylvian fissure, and histopathology confirmed that the contusion contained a subcortical hematoma with a small arterial tear . CUBIC iOverall, CUBIC processing had minimal effect on the histopathological observation of the artery in all cases, but small cracks occurred inside the brain parenchyma in some cases .Three-dimensional imaging using a tissue-clearing technique has been studied mainly in mouse organs and tissues ,59. HoweFirstly, CUBIC analysis is better at detecting arterial rupture than conventional macroscopic observation. In addition to cases where the presence of a rupture was macroscopically apparent, such as case 2, CUBIC analysis allows for the clear visualization of ruptures even in cases where there were only small superficial changes that were difficult to recognize as a rupture. For instance, cases 1 and 4 where there was no evidence of a rupture at all, and in case 3, where the artery itself was hidden by surface bleeding. Particularly in case 3, the presence of two ruptures in one cortical artery could not be recognized macroscopically but could be clearly visualized using the CUBIC method. This means that even if one rupture is detected by conventional histological examination, there may be other ruptures that could be missed. The CUBIC method is not only capable of detecting multiple ruptures in a single image, but it also improves the detection sensitivity of arterial ruptures compared with conventional macroscopic observation ,52.Another advantage of the CUBIC method is that it allows for detailed 3D morphological analysis of arterial ruptures and their adjacent arteries. Even in cases where morphological changes in the ruptured arteries were unclear or absent in conventional macroscopic observation, the CUBIC method allowed us to visualize them with high-definition in 3D images and enabled detailed morphological analysis. Likewise, the CUBIC method enabled visualisation where the small branches around the ruptured area were too small or hidden by hemorrhage to be recognized. Since conventional histopathological examination is based on the morphology of thin tissue cross-sections, each section only holds 2D morphological information. Although it is possible to obtain morphological information from thick tissue blocks by preparing a large number of serial sections, it is not easy to integrate them to obtain high-definition 3D images, which the CUBIC method can create. In five of our six cases, rupture of the cortical artery occurred at the root of its small branch. The characteristic morphology of the arterial rupture, termed “fire hose” rupture , was preHowever, the CUBIC method has some limitations that can occur not only under the specific condition of cortical artery rupture but also under common pathological conditions. The first limitation is that hematomas will not be sufficiently cleared, so their internal and surrounding structures will be obscured for LSFM imaging. Thin hemorrhages on the surface of the brain, as in case 3, did not interfere with the imaging of blood vessels, but in case 6, a subcortical hematoma around the ruptured artery inhibited the imaging of the lesion. The SDH itself can be removed from the brain during autopsy and does not interfere with CUBIC imaging. It would be difficult to adapt CUBIC imaging to conditions such as intracerebral hemorrhage or massive subarachnoid hemorrhage, where it is difficult to remove the hematoma during autopsy.The second limitation is the vulnerability of the transparent tissue blocks. Tissue blocks become soft like jelly during the transparency process, so they must be handled carefully to avoid breakage. In our study, even careful handling of normal brain tissue resulted in the partial destruction of brain parenchyma, so the application of CUBIC imaging to more fragile tissue, such as brain tumors with necrosis, seems to be a destructive analysis which opposes the benefits of this method. Since arterial tissue is tougher than the brain parenchyma, none of the arteries were seriously affected in our case series. However, mishandling can break weak arteries.Another limitation is that the size of tissue that can be imaged with CUBIC is limited by the condition of the tissue. Given that the fluorescence laser is scattered and attenuated as it travels through the tissue , the imaAlthough the CUBIC method can create virtual tissue sections computationally, it cannot completely replace the histopathological examination in all cases. As is evident in case 2, cross-sectional images of the CUBIC method might not reveal fine structures. In addition, although immunostaining is possible with the CUBIC method ,58, histBased on the advantages and disadvantages described so far, we further discuss optics, statics, and analytical chemistry in our CUBIC-based analysis as below. Since forensic autopsy samples include coagulated blood, it is of primary importance to decolor the blood in order to execute clearing-based 3D imaging. The CUBIC cocktails consist of the chemical reagents optimized for eluting heme molecules from the coagulated blood ,62. ThusCleared samples show no apparent change in size over a year, even though the CUBIC-R slightly swells the tissue sample in the process of tissue-clearing . Since CIn analytical chemistry, we performed tissue autofluorescence imaging with excitation wavelengths in the visible region, which is effective for analytical chemistry such as chemical labelling and immunostaining. The autofluorescence of human brain tissue is extended over a broad wavelength band. Since the intensity of autofluorescence and the light transmittance after clearing highly depend on the sample condition, an arbitrary excitation wavelength was selected in our experiment. If the sample would be stained by specific labelling, the autofluorescence must be suppressed by hydrogen peroxide treatment prior to labelling, since autofluorescence causes background noise, but this process may enhance sample vulnerability.The advantages and disadvantages of the CUBIC method described so far have only been observed in a few cases, and new advantages and disadvantages may become apparent when more cases are examined. This study does not describe the understanding of the pathogenesis itself of this type of SDH using the CUBIC method, and it is not clear at this time how much the CUBIC method can contribute to the elucidation of the pathogenesis. Based on our findings, we are currently applying the improved CUBIC method  to new cIn the investigation of arterial rupture as a source of bleeding in SDH, the CUBIC method clearly showed arterial rupture in 2D and 3D, although there were exceptions. The CUBIC method is a non-destructive method of tissue examination that does not hinder additional histopathological examination. This means that the addition of the CUBIC method to the conventional macroscopic observation and histopathological examination makes it possible to analyze the morphology of arterial rupture in more detail, which is useful for the analysis of SDH pathogenesis.