We report an experimental study demonstrating the feasibility to produce both pure and Ge-doped silica nanoparticles (size ranging from tens up to hundreds of nanometers) using nanosecond pulsed KrF laser ablation of bulk glass. In particular, pure silica nanoparticles were produced using a laser pulse energy of 400 mJ on pure silica, whereas Ge-doped nanoparticles were obtained using 33 and 165 mJ per pulse on germanosilicate glass. The difference in the required energy is attributed to the Ge doping, which modifies the optical properties of the silica by facilitating energy absorption processes such as multiphoton absorption or by introducing absorbing point defects. Defect generation in bulk pure silica before nanoparticle production starts is also suggested by our results. Regarding the Ge-doped samples, scanning electron microscopy (SEM) and cathodoluminescence (CL) investigations revealed a good correspondence between the morphology of the generated particles and their emission signal due to the germanium lone pair center (GLPC), regardless of the energy per pulse used for their production. This suggests a reasonable homogeneity of the emission features of the samples. Similarly, energy dispersive X-ray spectroscopy (EDX) data showed that the O, Ge and Si signals qualitatively correspond to the particle morphology, suggesting a generally uniform chemical composition of the Ge-doped samples. No significant CL signal could be detected in pure silica nanoparticles, evidencing the positive impact of Ge for the development of intrinsically emitting nanoparticles. Transmission electron microscope (TEM) data suggested that the Ge-doped silica nanoparticles are amorphous. SEM and TEM data evidenced that the produced nanoparticles tend to be slightly more spherical in shape for a higher energy per pulse. Scanning transmission electron microscope (STEM) data have shown that, regardless of size and applied energy per pulse, in each nanoparticle, some inhomogeneity is present in the form of brighter  features of a few nanometers. In material science, laser–matter interaction encompasses not only the study of basic mechanisms but also material machining/engineering. Such emphasis is partially related to the increasing need for optical, photonic and medical devices used in telecommunication, medicine, sensing and imaging applications –2 as welIn 1, Ge Lα1,2 and O Kα1 lines, respectively. The data have been recorded for the group of nanoparticles inside the red square named 1 in panel a. These three images indicate that all the three elements are detectable. As expected, the signals of the three elements are higher when the quantity of the sample is larger in the morphological image (SEM), implying a higher amount of Ge, O and Si atoms. On the other hand, the chemical composition appears to be qualitatively homogenous for sample A.In The origin of the CL signal has been investigated by spectroscopy. 19To further characterize the morphology of the produced nanoparticles, we recorded transmission electron microscopy (TEM) and scanning transmission electron microscope (STEM) images of the samples. Similarly, for the irregular ones the sizes range from a few tens up to about 100 nm. The state of aggregation/agglomeration is also evidenced. More results are further highlighted in Aiming to better understand the role of the energy laser pulse, we investigated a second sample produced with a higher energy per pulse, which is similar to the one used in . SEM, EDThe comparison with the SEM image  providesIn Some microparticles were also detected in sample B. Similarly, although in the SEM and TEM images, the spherical particles appear more frequent than in sample A, irregular particles of different sizes were detected, too.The last sample that we studied is a pure silica material, as reported above, where the employed energy per pulse was about 400 mJ. One of the SEM images recorded for this sample is reported in In the present investigation provide evidence for three main results: i) the comparison of samples produced by only varying the starting energy per pulse showed the generation of nanoparticles in Ge-doped silica at a laser energy per pulse lower than that in previous investigations ; howeverTo further understand the generation of the nanoparticles, it is relevant to note that for the Ge-doped samples only a few laser pulses (≈1 s of irradiation) were needed even at the lowest energy , to observe (by the naked eye) material removal; in the case of the pure silica samples, more pulses (10–20 s of irradiation) were needed and the irradiated parts become very white before the removal of matter. Such phenomenon, if present, was too fast to be observed in the Ge-doped samples. This phenomenon could indicate that a large amount of absorbing defects have to first be generated to achieve a sufficient absorption of energy at the wavelength of the KrF laser. Such an effect can be compared to the incubation effect previously reported in . This haActually, the decrease of the band gap implies that the order of the multiphoton absorption can be lowered and the presence of defects with optical transition at ≈5 eV allows a standard absorption from the excitation laser, eventually followed by a second absorption from the real excited state of the defects  to tThe higher efficiency of energy transfer to the network in Ge-doped silica is also suggested by the data of  and by tConsidering the two Ge-doped samples, there is only a small tendency in sample B to form more regular  particles. In this case, explosion of the sample occurred more frequently during irradiation. This experimental result indicates that it would be difficult to obtain monodisperse, regularly shape nanoparticles by increasing or decreasing the energy per pulse of a nanosecond pulsed laser with a spot size of millimeters.In the following we consider that the particles of irregular shape are generated mainly during explosion events and that spherical particles are induced by the melted and then rapidly quenched events. On one hand, increasing the energy per pulse allows a larger part of the sample to absorb a sufficient amount of energy to be melted. On the other hand, this growth of the melted mass and possible increase of the induced thermal gradient, which together can create stress or inhomogeneity and can induce explosion, implies a higher tendency to explosion.Furthermore, the laser spot has a spatial profile  which implies that different regions, within the spot, are irradiated differently. Our data indicate that for low energy per pulse (or fluence) the particle shape tends to be irregular. The use of the same mask could limit the contribution of the irradiated part far from the center , which seems to contribute to the generation of irregular particles.In any case, the presence of the Ge atoms and the consequent increase of the sensitivity to laser irradiation could be combined with other ablation procedures using a UV ns pulsed laser as the one reported in . In regaOur TEM experiments also highlighted the amorphous nature of the obtained nanoparticles for samples produced with various values of energy per pulse. Such results indicate that crystallization is not induced, so even if high energy absorption can induce melting of part of the sample, the quenching is sufficiently fast to avoid the formation of large crystalline regions.2 rich zones or densified parts for example. The presence of Ge-rich zones, even if of small size, has been proposed in Ge-doped bulk silica as a consequence of an imperfect mixing or composition fluctuations [The obtained nanoparticles differ in size and are in general larger than a few tens of nanometers. Such dimensions do not represent a relevant issue for some applications, as discussed in 13,35–37,tuations –23.In this work, we employed various techniques to produce Ge-doped and pure silica nanoparticles using nanosecond pulsed KrF laser irradiation. Our experiments demonstrated the possibility to generate nanoparticles using a laser pulse energy of about 33 mJ and 400 mJ for doped and pure silica materials, respectively. The difference in the required energy per pulse has been attributed to the high Ge doping level, which affects the optical properties of the silica. As a consequence of the presence of the GLPC, the emission activity was studied using cathodoluminescence measurements only for the Ge-doped nanoparticles. In addition to experiments using 33 mJ irradiation, an experiment was performed on Ge-doped glass with an energy per pulse of about 165 mJ. Comparing the samples produced with 33 mJ and 165 mJ (based on TEM images) we noted a slight tendency to form more spherical nanoparticles in the second case. In both cases we also noted a significant correspondence between the morphology studied by SEM images and those studied by cathodoluminescence, suggesting a certain homogeneity in the optical properties of both samples. The amorphous structure (as investigated by TEM) and qualitative homogeneity in chemical composition (as studied by EDX) of Ge/Si dioxide appears to be independent of the energy per pulse applied. Nevertheless, STEM data indicate the presence of some inhomogeneity inside each nanoparticle, likely arising from aggregation of elements or densification. Such a result is observed independent of the size or the energy used for their production. Thus, apart from the size, the produced samples are a set of nanoparticles with reasonable homogeneity in structure, chemical composition and emission features.For the present investigation, we used two different types of silica bulk materials. The first one is doped with 20 wt % Ge, produced by plasma-activated chemical vapor deposition (PCVD) ,30. The 2 considering a uniform distribution on the spot surface and neglecting losses along the optical path in air and water. This latter assumption on transparency of the employed liquid is generally confirmed, even if multiphoton absorption by liquid or other effects are present [In detail, employing a preparation procedure similar to that used in one of our previous studies , the sam present .The samples were moved along the plane orthogonal to the laser beam in order to refresh the irradiated region, and the laser focus was adjusted to keep it at the sample surface. A few seconds of irradiation were needed to observe the starting of the material removal from the silica sample.SEM images were acquired with a JSM 7100F (JEOL) instrument, using an electron energy of 20 keV and probe current of ≈10 nA. The samples were not coated with a conductive layer as in . The samTEM and STEM images were recorded with a JEM-2100F UHR (JEOL) device at 200 keV beam energy. STEM images were acquired with a dark-field annular detector.For all the experiments we used holey-carbon-coated grids on 200 mesh copper from SPI, since these grids allow the characterization of the same zones along the samples with the different techniques. The thin network-shaped carbon film helps to recognize the location of the investigated particles. The suspensions resulting after the laser irradiation (distilled water + silica particles) were kept at about 90 °C to reduce the amount of water. Then, we shook the solution and immersed one grid in it, and left it on a heating plate that was preheated to 60 °C. The grids were supported by an aluminium stub for SEM/EDX/CL characterization.


It is known that motor actions performed by individuals with autism spectrum disorders (ASD) are clumsy and a previous study revealed that children with ASD of around 8 years old showed less smooth movement and dysfunction of appropriate usage of online vision for grip aperture control. The present study investigates whether and how the kinematic properties of reach-to-grasp movements in older adolescents and adults with ASD [mean (±SD) age: 18.3 ± 2.1] differ from those in typically developing (TD) peers [mean (±SD) age: 19.1 ± 2.2]. Revealing the kinematic properties of reach-to-grasp movements in older adolescents and adults with ASD is indispensable in determining the developmental trajectory of this motor behavior in individuals with ASD. While wearing liquid crystal shutter goggles, participants reached for and grasped a cylinder with a diameter of either 4 or 6 cm. Two visual conditions were tested: a full vision (FV) condition (the goggles remained transparent during the movement) and a no vision (NV) condition (the goggles were closed immediately after the movement was initiated). These two visual conditions were either alternated with each trial in a single experimental session  or blocked within the session (blocked condition). We found that the reaching movement smoothness calculated as a normalized jerk score  of ASD participants did not differ significantly from that of TD peers although ASD participants showed smoother reaching in the alternated condition than in the blocked condition. The influence of online vision and its visual condition schedule on grip aperture during the in-flight phase was remarkably similar between the ASD and TD groups. Furthermore, we found that ASD group experienced a significant longer transition period from grasping end  to uplift initiation than the TD group. The results suggest that (1) deficits in movement smoothness and the use of online vision for motor control are rectified by the time individuals with ASD reach late adolescence and (2) older adolescents and adults with ASD still have difficulties chaining motor acts. Autism spectrum disorder (ASD) was first identified by The prospect for early detection of motor abnormalities in infants with ASD is still controversial , but movAmong the various hand/arm motor behaviors, the reach-to-grasp movement is fundamental to daily life and, since Recently, it has been demonstrated that grip aperture control can be modulated by the presentation order of trials of FV and NV conditions in healthy adults , 2015. SReach-to-grasp movements are usually performed with a subsequent motor act, depending on a final goal of the action e.g., . TherefoThe above studies regarding the kinematic properties of reach-to-grasp movements in individuals with ASD were focused on pre- and early adolescent children. No study has yet explored how older adolescents and adults with ASD perform simple reach-to-grasp movements, although pointing movements (without grasping) , 2009 orIn this study, we used motion capture system to investigate whether and how the kinematic properties of reach-to-grasp movements in older adolescents and adults with ASD differ from those of TD peers. Based on the studies mentioned above, we tested a task requiring individuals to reach to grasp the object and lift it up after holding it based on three functions: (1) movement smoothness, (2) grip aperture control modulated by availability of online vision  and its presentation order, and (3) chaining of motor acts (as an entire action). Specifically, in comparison with TD peers, we focused on (1) whether older adolescents and adults with ASD show clumsy movement , (2) whether people with ASD could use online vision and show a “homogenization” effect in relation to grip aperture control, and (3) whether people with ASD show a deficit in terms of chaining motor acts as an entire action .The experiment involved 12 individuals with ASD  and 12 TD individuals  Table  and had A Japanese version  of the aThe study was approved by the institutional ethics committee at the National Rehabilitation Center for Persons with Disabilities, and all the participants  provided written informed consent according to institutional guidelines conforming to the Declaration of Helsinki.As shown in Figure Each participant was required to place his or her right hand on the starting position before each trial. In this position, the lateral side of the little finger and ulnar palm was touching the surface of the button. Participants were also required to begin each trial with the tips of the thumb and index finger of the right hand touching each other. This pre-trial condition was consistent across every trial of the experiment.The task in each trial required each participant to reach out to grasp the target object at a comfortable daily life speed and then lift the object about 5 cm. After lifting up the object, the participant put the object back to where it had been and returned his/her hand to the starting position. With respect to grasping (holding) the object, the participants were instructed to approach the object laterally with their fingers and to use the thumb and index finger in such a manner that the line connecting the surface points of each thumb and finger passed through the center of the object’s horizontal circle. Use of the other three fingers was allowed to grasp the target object as long as the thumb and index finger were always the main digits used in the grasping action .The vision during the task was manipulated using liquid-crystal shutter goggles. The goggles were opaque before each trial, and participants started their movements after the goggles became transparent. This was accompanied by the experimenter’s voice cue . Two visual conditions during the movement were tested: an NV condition, in which the goggles closed immediately after the release of the hand from the start switch, and an FV condition, in which the goggles remained transparent during the entire movement e.g., , 2015. WThe experiment comprised two sessions , and the order of these sessions was counterbalanced across participants. Each session comprised four sub-sessions, each with 15 trials. In the blocked condition, either the FV or the NV conditions was repeatedly presented during the first two sub-sessions and the other was presented during the last two sub-sessions. Object size was blocked in an ABBA manner. The presentation orders were also counterbalanced across participants; therefore, in addition to the practice trials , each participant in the groups completed 120 trials [=15 trials × 2  × 2  × 2 ] across the entire experiment.The three-dimensional positional data given by Cartesian coordinates from the reflective markers were recorded and filtered offline by a second-order dual-pass Butterworth low-pass filter with a cut-off frequency of 15 Hz. Further offline analysis included computation of wrist velocity, acceleration, and jerk from the filtered position signal. We also calculated grip aperture as the distance between the positions of two reflective markers attached to the thumb and index finger.Movement onset was defined as the frame in which the tangential velocity first exceeded 50 mm/s, and reach-end time was defined as the frame at which the velocity fell back below this threshold. The typical aperture velocity profile for grasping movements shows positive values throughout the aperture-opening phase until the PGA is achieved; that is followed by negative values as the hand’s fingers close down upon the object. Grasp-end time was defined as the point in time when the negative grip aperture velocity crossed the criteria line (setting -20 mm/s) before returning to approximately 0 mm/s as the fingers made contact with the object.The time of lifting initiation was defined as the point in time when wrist height velocity exceeded 15 mm/s. Finally, reach duration denoted the time between movement onset and reach-end time, and movement duration was defined as the time between movement onset and grasp-end time .The current study focused on (1) smoothness of reaching movement, (2) grip aperture control according to online vision, and (3) smoothness of chaining motor acts  in autistic participants, compared to typical developing peers. To evaluate these functions, we calculated the following values.Firstly, the NJS, which is unit-free, was calculated as an index of movement smoothness using the equation shown below . Calculaj denote reach duration, total wrist displacement until reach end, and jerk, respectively. Secondly, the differences of the NJS and PGA between the FV and NV conditions (NJSDiff and PGADiff) were computed to evaluate whether a homogenizing effect induced by visual condition schedule appears in reaching movement smoothness and grip aperture adjustment. Lastly, the difference between the grasp-end time and the time of lifting initiation (DiffGrLf) was calculated as an index of the smoothness of chaining motor acts.In the equation, RD, WD, and In addition to these above-mentioned values, the values of the transport component , the time to PGA, and the reaction time  were measured.Mean values for each dependent variable (except NJSDiff and PGADiff) were entered into a four-way ANOVA with the group  as a between-participants factor and the visual context , object size , and visual condition  as within-participant factors. With respect to NJSDiff and PGAdiff, a three-way ANOVA was applied with the group  as a between-participants factor and the visual context , object size  as within-participant factors. If we found an interaction, the simple main effect analysis was examined with the Bonferroni correction.Furthermore, by pooling the data for the TD and ASD groups, a multiple linear regression and associated stepwise variable selection method were applied to analyze the relationships between the five subcategories of AQ scores  and each kinematic parameter to determine whether and which subscale scores would predict the performance of each kinematic parameter.The mean values of kinematic parameters in each experimental condition in the ASD and TD groups were shown in Table p > 0.109).No significant main effects on factors and no significant interactions were noted  = 28.344, p < 0.001, partial η2 = 0.563], and the interaction between object size and visual condition was also significant . The simple main effect analysis revealed that the movement duration in the FV condition was significantly shorter than that in the NV condition for both object sizes.Visual condition was found to have a significant main effect . The simple main effect analysis revealed that the time to peak wrist velocity for the 6 cm object was also significantly later than that for the 4 cm object in the ASD group while no significant time difference was found between the 4 and 6 cm objects in the TD group.The interaction between group and object size was significant , indicating that the NJS in the NV condition was significantly larger than that in the FV condition. The results also showed a significant interaction between group and visual context , as well as a main effect of visual context which indicated that the NJS in the blocked condition was significantly larger than that in the alternating condition . The simple main effect analysis revealed that the mean NJS in the blocked condition was significantly larger than that in the alternating condition in the ASD group, while there was no significant difference between the blocked and alternating conditions in the TD group.The main effect of visual condition was significant , object size , and visual condition  were found. The results show a larger PGA for the TD group than the ASD group, as well as a larger PGA for the larger object and the NV condition. Significant interactions between visual context and visual condition , and between size and visual condition  were also found. In particular, the simple main effect analysis found that the PGA in the NV condition was significantly larger when performing the task in the blocked condition than when performing it in the alternating condition, while the PGA in the FV condition was significantly larger when performing the task in the alternating condition.With respect to PGA, significant main effects of group  and size  were found, indicating that the value in the blocked condition was significantly larger than that in the alternating condition and that the value for the 4 cm object was significantly larger than that for 6 cm object. No significant main effect of group  was noted.With respect to PGADiff, significant main effects of visual context , indicating a longer time to PGA in the NV condition than in the FV condition when performing the task in the alternating condition. This interaction also denoted that the time to PGA in the alternating condition was significantly earlier than that in the blocked condition when performing the task in the FV condition.A significant interaction between visual context and visual condition was found , indicating that the ASD group took significantly longer time than the TD group to begin to lift up the object after the grasp-end time. A significant main effect of size  was also found, indicating that the value for the 6 cm object was significantly longer than that for the 4 cm object.The results showed a significant main effect of group [R2 = 0.172, p = 0.044), movement duration , time to peak velocity , and time to PGA . We also found that only the subcategory attention switch score was significantly correlated with DiffGrLf . The other kinematic values showed no significant correlation with the subcategory AQ scores.The multiple regression analysis revealed that only the subcategory social skill score was significantly correlated with reaction time (This kinematic study explored whether and how older adolescents and adults with ASD perform reach-to-grasp and uplift movements in comparison to TD peers. Our foci were (1) smoothness of the reaching movement, (2) grip aperture control according to online vision, and (3) smoothness of chaining motor acts.First, as for the smoothness of the reaching movement, the ASD group showed a significantly larger mean NJS in the blocked schedule than in the alternating schedule, while no significant difference of the mean NJS was shown between these two visual schedule conditions in the TD group. This result indicates that the modulation patterns of reaching movement smoothness according to the visual context itself are different between ASD and TD. Specifically, alternating FV and NV conditions from trial to trial in a session, in contrast to blocking these visual conditions, contributed to reducing the NJS  for both FV and NV conditions in the ASD group while visual context had no influence on the NJS in the TD group. At the same time, the main effect of the group  on the mean NJS was not significant, suggesting that reaching movement smoothness in the ASD group was comparable to that in the TD group. Furthermore, the homogenizing effect did not operate on the NJS in either the TD or ASD group. This finding strengthens the previous findings by Second, grip adjustment according to online vision and its context was remarkably similar between ASD and TD peers. Specifically, homogenization in PGA due to visual context , 2015 ocThird, the significantly longer DiffGrLf  in the ASD group than in the TD group suggests that ASD participants have difficulties chaining motor acts smoothly and appropriately. It is noteworthy that this significant longer DiffGrLf in ASD group emerged despite no significant difference in reaction time or movement duration between the ASD and TD groups. Namely, the longer transition period from grasping end to uplift initiation in the ASD group cannot be attributed simply to general movement slowness.As we introduced in the section “Introduction,” With respect to chaining sequential motor acts, By comparing the current results concerning older adolescents and adults to previous studies mentioned above concerning school-age children, we can deduce the following developmental trajectories of prehension movements in ASD: First, impairment of reaching smoothness and grip adjustment in school-age children could be compensated by their developmental processes, and their visuo-motor transformation processes would be comparable to TD peers when they grow older . By contrast, organizing their sequential motor acts as an entire action is difficult even for older adolescents and adults.In addition to comparing ASD and TD participants, we investigated the relation between autistic traits and kinematics by pooling the data on the TD and ASD groups. The transition period from grasping end to uplift initiation (DiffGrLf), which showed a significant difference between the ASD and TD groups, was significantly correlated with the subcategory attention switch. Although we found no significant difference between the ASD and TD groups for the following values (except time to peak wrist velocity for the 6 cm condition), the subcategory social skill score showed a (mild) significant correlation with the temporal components . Although the reason for this specific subcategory’s significant correlations with specific parameters will need to be clarified in future studies, the current results suggest that the AQ subcategory scores could be useful for predicting motor performance for the pooled population of ASD and TD individuals.The lack of significant differences in parameters between ASD and TD during reach-to-grasp movements (except for time to peak wrist velocity for the 6 cm condition) may be due to the ease of the task in the current experiment, since the target was presented at one fixed location and each target size was blocked in each session. Furthermore, participants were instructed to perform the task at a comfortable daily life speed in the current study cf. . Of courOlder adolescents and adults with ASD and their TD peers exhibit a similar grip aperture modulation according to online vision and its context in the current experiment, while school-age children cannot perform such a modulation . That isWith respect to the effect of visual context on reaching smoothness (NJS), the significant reduction of NJS in the alternating condition compared to the blocked condition was found in the ASD group, while no significant difference in NJS occurred for the TD group. While this latter result for the TD would be due to the ease of the current task , the increase in reaching smoothness in the alternating condition for the ASD group would support investigations of therapeutic interventions for movement disturbance. For example, it may be better to train motor behaviors across several visual contexts, rather than in fixed visual condition. Although no significant difference in NJS between the ASD and TD groups was found in the current study, future studies using a higher sampling frequency motion capture system in a slightly larger sample size are needed to clarify detailed properties of reaching smoothness.The current prehension task is simple and performed by one single person. Recently, however, this prehension action was incorporated into a task to investigate the visuomotor processes of joint action carried out concurrently by two people e.g., . ReasoniIn sum, the use of online vision and its context for motor control, which is not fully exploited in school-age children, may be compensated for when individuals with ASD reach late adolescence; however, older adolescents and adults with ASD still have difficulties chaining motor acts.TF conceived, designed, and performed the experiments, analyzed the data, and wrote the manuscript. SK and KN diagnosed the individuals with ASD. MiS confirmed the diagnoses using the Japanese version of the ADOS. AT, MaS, and HA performed psychological the tests. TF, RF, YN, and MW contributed reagents, materials, and analysis tools. All authors discussed the results and commented on the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


The goal of this paper is to suggest a system for intelligent learning environments with robots modeling of emotion regulation and cognition based on quantitative motivation. A detailed interactive situation for teaching words is proposed. In this study, we introduce one bottom-up collaboration method for emotion-cognition interplay and behaviour decision-making. Integration with gross emotion regulation theory lets the proposed system adapt to natural interactions between students and the robot in emotional interaction. Four key ideas are advocated, and they jointly set up a reinforcement emotion-cognition system (RECS). First, the quantitative motivation is grounded on external interactive sensory detection, which is affected by memory and preference. Second, the emotion generation triggered by an initial motivation such as external stimulus is also influenced by the state in the previous time. Third, the competitive and cooperative relationship between emotion and motivation intervenes to make the decision of emotional expression and teaching actions. Finally, cognitive reappraisal, the emotion regulation strategy, is introduced for the establishment of emotion transition combined with personalized cognition. We display that this RECS increases the robot emotional interactive performance and makes corresponding teaching decision through behavioural and statistical analysis. One of the most significant discussions of intelligent interactive robotics is the collaboration of cognition and emotion adjustment. The internal processes of emotion-cognition interplay appear in the form of behaviour performance, and positive objects  tend to be more acceptable than negative ones . Agents Relationship between emotion and cognition is bidirectional . EmotionThis study is conducted to provide an intelligent learning environment for students and robotic teachers with the emotion-cognition collaboration ability. The robotic teacher can be enabled to select an emotional facial expression and provide a difficulty level of the next English words' meaning multiple choice question. The difficulty level is more in line with robot's preference according to the student's current performance. The robotic teacher focuses on two external factors: the students' scores and their emotional state. Moreover, it has two intrinsic factors: motivation and emotion, which could be a cooperative and competitive relationship, jointly determining the behaviour output.Our paper is organized as follows. In In cognitive engineering and cognitive psychology, cognition is generally assumed to be a participant in information processing . MotivatLike cognition and emotion, emotions and motivation are related structures but are not exactly the same . StudiesMotivation has neurobiological origins in basal ganglia and midbrain marginal dopaminergic pathways. Chumkamon et al.  made synMoreover, emotion is one kind of internal state constantly motivated and experienced by the individual. For the reason that emotion is awoken consciously or unconsciously and can be considered an emergent property of motivationally driven neural activity , the emon-bit (≥3) parity problems, inspired by the neurocircuits' competitive property.In the recent literature, emotional modeling is motivated by two main theories: anatomical approach  and apprAppraisal theory emphasizes on the dynamic emotional processes, considering that emotion derives from a person's cognitive interpretation of environmental relations . These tIn the natural emotional interaction, the emotional change will be affected by a series of external and internal factors. It is affected not only by external emotional stimulation and the current emotional state of the impact but also by the individual's own emotional cognitive ability.Appraisal theorists typically treat appraisal process as the cause of the emotion, or at least of the physiological, behavioural, and cognitive changes associated with emotion , 34, 35.Cognitive reappraisal occurs before the emotional response, and the emotional state is reappraised and adjusted; expression inhibition occurs after the emotional response behaviour. Cognitive reappraisal strategy as a priority regulation strategy reduces the negative emotional experience better, and emotional state tends to be alleviated. Gross suggests that emotion regulation refers to the process of influencing emotions, experiences, and expressions. More generally, emotion regulation involves the change of emotional latency, time, duration, behaviour expression, psychological experience, physiological response, etc. Therefore, we establish a collaboration system to describe the dynamic process. Thompson  argues tFor intelligent learning environment, advanced robotic teachers need emotion regulation ability, which helps robots generate more positive emotional state, while ensuring the smooth transition of emotions and modest changes based on cognition.H by support vector regression (SVR) through detected physiological signal, and the information on the user's operations is obtained through the touch screen; cognitive structure and emotion generation generate responses emphasized on emotion regulation; behaviour decision relies on the competition (Winner-Take-All) results of emotion or initial motivation.Our research rather lies in a system level. Frenzel  describeH), which refers to the emotional pressure of learners and whether the answer is correct. The external factors in the cognitive layer involved are the expectation to achieve more teaching tasks and learner's state composed of emotional pressure. There is competition between the two goals, based on the output expectation vs. learner's pressure, to determine the next phase of the study. The reinforcement learning process is used to measure the emotional influence on the final motivation. In other words, the learner's emotion factor also helps to achieve the expectation. It appears in the robot's internal emotion regulation that learner's negative emotion leads to delay in teaching process. And, robotic positive emotional state generated by emotion regulation can be detected by RL. Last, in the behaviour layer, the unconditional response shows the words meaning answer, and for the conditional response, the robot has two kinds of behaviour: selection of words according to the difficulty level and the facial expression representing the real emotional state. Generally, the emotion regulation occurs on an emotion-cognition collaboration level but appears in the expression level.Our collaboration system tends to research the effect of emotion regulation on interaction. In order to validate the research instrument, we invited 24 Chinese undergraduates of the same 20 years of age. Before taking part in the study, all the undergraduates had received more than six years of formal EFL education and passed the EFL test of the National College Entrance Examination in China with same scores.We used a MATLAB simulator to achieve perception and reinforcement learning process. This experiment was performed on a noninvasive physiological collected wrist strap and a robotic platform with one contact interface, one loudspeaker, etc. For valence extraction, we used DEAP dataset for the training model. Besides, the communication protocol between the robot and wrist strap is Bluetooth 4.0.Each student makes sure these subjects keep calm at first and then let them scan the words list without meaning, scoring according to the degree of familiarity quicklyAccording to the specific student, the words list is imported into a corresponding robot systemThe robot teaches the word to each student by providing multiple choice questions for choosing the right meaning of the wordIn the teaching process, the robot provides a random word at the beginning, and when it receives the feedback from the learner, it selects the difficulty level of the word and makes emotional expression based on RECSEach student is allowed to answer 40 questionsThe specific experiment steps are as follows:As for experimental comparison, we divided the 24 students into four groups with four different configurations (the details are shown in Results) with six students per groups. Each group has different system parameters leading to discrepant emotional state and behaviour output.H through the physiological features and train the regression model with the real-time features. The training labels (contain instantaneous labels) are self-reported even in short-time events (5 s) that ensure the accuracy of emotion prediction in real-time interaction. Apart from this, we consider that recognizing an interactive stimulus implies the recognition of the sense of students' interactive effect . The memory unit is required to hold this information of mapping relationships and stimulus's level. The combined information is a vector recording questions' difficulty and accuracy of answers. Specific scores are artificially defined for different kinds of questions' difficulty. Once sensors receive the new external stimulus, vector's value are updated and delivered to the next layer. There are two conditions. They are as follows:The stimulus in the interaction process derives not only from sensory level but also from analysis of motivation. For teaching words task, the stimulus is obtained from two modules: interactive stimulus (scores) and valence extraction. The emotion of interactive students could influence agent's cognition. Thus, we extract valence characterized as the emotion which is measured as the positive degree. More specifically, emotionally valenced  sensory and physiological signals give the agents a subjective and motivated perception of their interactive behaviours. Their sensations, as well as their actions, are no longer neutral and objective but are rather emotionally coloured. In our experiment, the subjects wear supplementary noninvasive wrist straps which record physiological signal synchronously. SVR algorithm is used to recognize valenceTransferring to the behaviour layer directly, i.e., the direct stimulus without cognitive and emotional modulation. In the specific task, it means giving students the right answers.Transferring to cognitive layer and then obtaining the next question and agent's emotional expression based on emotion-cognition collaboration.As the previous sections described, the emotion modulation happens in the cognitive layer. Thus, the cognitive structure deals not only with output as an effect of cognition to emotion and behaviour but also with input as an effect of emotion and the environmental stimulus to cognition. Agents need motivation to reduce the difference between environmental stimuli and ideals. Thereby, it can notice when the current situation is different from what it recognized. The motivation is extracted from two quantitative goals : teachinIn neurobiology, dopamine is one of the factors that determine motivation, and there is selective depletion process of forebrain dopamine. Therefore, different from self-organized cortical cognitive maps, the presented method of motivation acquisition focuses on the evolution of the whole in the time dimension, and external stimulus still serves as the only input to the network. As shown in D0 is an output of the neuron with the weights W and external stimulus S(t) input:W also represents inner default state of agents. We measure the effect of t − i time as follows:For simplification, normalizing both ranged from  before further calculation. In order to represent the current motivation, we rely on competitive network structure and establish competitive neurons considering about short history effect. The network is composed of two aspects: competition between current external stimulus and default inner state and competition between current differences and history differences. Thus, current difference β is the attenuation weight and is given as follows:δ represents the descending speed of the history effect.The idea of measuring novelty of prediction errors for the purpose of self-improvement has been considerably exploited in the research on intrinsic motivation . But lony(t) is the intermediate value participating in the time-series process, and sigmoid function is used to ensure that the emotional output sequence is between 0 and 1.The sensory cortex receives a signal that is transmitted to the amygdala via the thalamus, producing an emotional state. It exists in the brain in the activity form of “feeling stream.” The inner limbic structure and the thalamic system, including specific hormones and chemical neurotransmitter activities, ensure the persistence of the feeling stream . TherefoThis module can be written as a simple Taylor expansion to represent the nonlinear process by using nonlinear kernels up to the first order:ϑ(Mot(t)) represents the effect of motivation on emotion transition. The effect of motivation uses Gaussian kernel function equations applied to produce the value corresponding motivation with emotional state:vR(t − 1) is the previous valence value. Thus, the nonlinear kernel can be set asMotivation is one of the influencing factors, and the nonlinear kernel can be used to measure its influence in the formula. τ is used to represent the level of this ability, which is achieved in the sigmoid function:For description of cognitive reappraisal ability, The sigmoid function can be described asR (in different levels of representations) can trigger emotional facial expressions through stored mapping relationships.Because the emotion regulation influences the sensing-related and the action-related  processes, the module contains unconditional response, facial emotional expression based on emotion, and the behaviour based on the collaboration. On the other hand, the input is driven by motivation extraction, emotion generation, or sensory level. The three paths deliver the signal to behaviour decision in parallel. Besides, for excitation from emotion, valenceThe facial emotional expression ranges in six states (from positive to negative) . In thisWe use the reinforcement learning  method tReinforcement learning enables a robot to autonomously discover behaviour outputs under the influence of emotion that chooses difficulty levels of the next question through accumulated reward from the first derivative of emotion. Instead of explicitly detailing the solution to a problem in reinforcement learning, the design of reward provides the competition between motivation and emotion in terms of real-time emotional transition.Observations: the robot generates the emotional state after the series processes of cognition-emotion collaboration. Observations at each moment serve not only as the current final emotional state of the robot but also as the source of assessment for reinforcement learning.Goals: the paper focuses on accumulated emotional changes as the outlet of emotional behaviours. Stable changes in robotic emotion prefer to let initial motivation control behaviours. However, when the fluctuation of the emotion is intense, the robot prefers to take “irrational” behaviours controlled by emotional influence.Reward design: it is important to select the time when emotion state is significant enough to control the behaviour decision. We provide the first derivative of emotion as the reward which is a continuous value to represent how much the emotion intensity accumulates:It means the final i value at the first time max(R(t − i), 0)=0. Winner-Take-All is used to make competitions between the initial motivation and a certain percentage of the reward.Based on the literature presented in the previous studies, the key ingredients of the reinforcement learning setup are observations, goals, and reward design, which are explained as follows:There are four configurations, compared pairwise, considering two experiment goals: verification of the impact of different robotic preferences and cognitive reappraisal ability on behaviour or its own emotion. C1, C2, C3, and C4 mean configuration 1st (set high preference to emotion), configuration 2nd (set high preference to students' scores), configuration 3rd , and configuration 4th . For obtaining the clear results of the comparison, the first two set the same cognitive reappraisal ability , and in the rest two, fair competition is provided between the emotion and scores.To compare the full system described, a total of 24 interaction processes are performed, 6 for each configuration. In this section, the full system performance is described in the first part, about the robotic motivational and emotional effect on final behaviour decision. Second, we show the impact of different robotic preferences on behaviour or its own emotion. Finally, the effectiveness of cognitive reappraisal is provided. It is noteworthy that we use word “motivation” in the name of initial quantitative motivation, contrasting with emotion, although emotion generation is related to motivation.X is the robotic reward or motivation vector while Y is the selected difficult level of questions representing the behaviour output or the opposite one. Based on correlation coefficient, the correlation between emotion and behaviour is generally lower than the one between motivation and behaviour. This indicates that the behaviour is mainly driven by motivation and we cannot confirm that the emotion did not play a role. As for the influence from motivation to emotion generation, we can see in For detailed observation, we provide two typical systematic activity plots: C1 vs C2 in For the incipient process, most of the previous external stimuli tend to be positive. Thus, when the agent receives the negative stimulus, it generates a more negative emotional state. We note that the initial decision of the questions' difficulty level has more positive correlation with the agent's emotional state. When motivation continues to decline, the emotional state tends to be more negative. And, motivational fluctuation within a certain range cannot influence the robotic emotional state significantly. It proves the emotion stability during a short time. Moreover, accumulated emotional decline causes the high reward.For the middle process, agent's emotional fluctuation tends to be mild although interactive stimulus rises and falls frequently . It is noteworthy that the agent prefers to choose more difficult questions once receiving negative scores during the positive condition of students. And, the agent pays more attention to the fluctuation of the student's emotion. During this period, the robotic emotional state has a mild curve, though the external stimulus is being changed. It is worth noting that at the 20th time, the positive stimulus cannot have a significant influence on behaviour, for the reason that robotic emotion wins the competition while robotic positive emotion causes the tolerant decision.The later stage of the experiment illustrates that the situation with the high fluctuation of scores and mild human emotional state curve causes motivation about expectation. We can see that continuous negative stimulus causes negative emotional state and negative or weak motivation.For intuitive comparison, C1 and C3 provide the systematic activity plot using the data during the real interactive process, while C2 and C4 show the simulation results using the same data recorded by C1 and C3, respectively.p=0.748) and scores with behaviours . And, the Mann–Whitney test has the following results: U = 20, p=0.409 and U = 11, p=0.1548. K-W and M-W tests show no significant effect on these configurations, which means the correlation coefficients in C1 or C2 follow the same distribution.The fact we considered the influence on behaviour regardless of which preference is set does not allow us to conclude on the results, success or failure. Because whatever is students' emotion or score, will cause the motivation, and the K-W and M-W tests can prove that the motivational influence on behaviour follows the definite distribution. And, the effectiveness of these parts can be confirmed by the region of correlation coefficients distribution. In The same reason can be proved in As p=0.039). Besides, the M-W test provides the following result: U = 0, p=0.01, which also proves the effectiveness.And, in The results presented above highlight the interest of using the effect of emotion-cognition collaboration for a teaching words task. The prototypical behaviours we observe mainly describe four kinds of situations.Failure: failure of the learners let the robot generate negative emotion and may provide to select easier question: long-time failure leads to stable negative situations and temporary one just leads to the immediate impactSuccess: the success of the learners leads to a positive emotion of robots, and the robot may choose more difficult questions for its expectation or may choose more easy questions within pleasant emotional statePreference: the robotic teacher has different behaviour preferences when it prefers to care student's emotion or their scores. It proves that the motivation has obvious influence on emotion and behavioural outputs.Cognitive reappraisal: high cognitive reappraisal ability leads to more positive emotion of the robot. The comparison chart shows the emotional influence on behavioural output.According to the above method, the RECS focuses not only on learners' response score but also on their current state of physiology. Long-term failure will give learners more negative emotions and physical condition. None of these emotional experiences is considered positively. Therefore, emotional regulation is used to avoid these deadlocks and tries to keep the robot in a state of positive development of the emotional state, which could output more positive emotions to students. The evident “instability” of this system is due to interactive stimulus from the variety of uncontrolled circumstance. The impact of robotic emotional influence on behaviours is not obvious because some stable external stimulus conditions have been provided, and the lack of these situations in which bigger enough reward causes the emotional behaviour is also the reason for this. Statistical analysis validates the effectiveness of whatever the preference configuration or cognitive reappraisal configuration. The system provides not only the generation of motivation and emotion but also the emotion-cognition collaboration in terms of behaviour output and emotion regulation.For teaching tasks, the introduction of emotion regulation strategy is not disjointed with the whole content. Because for intelligentized, humanized robotic teachers, emotion regulation strategy can make the robot more intelligent in the generation of the state of emotion, which considers not only the more positive emotional generation but also the natural emotional transition. Of course, avoiding aforementioned deadlocks also is the major reason.Though all selections of behaviour from the emotion-cognition influence are the preconditions we set, this paper emphasizes the changeable behaviours and robotic emotion according to the emotion-cognition collaboration. In detailed teaching environments, whether the teacher should avoid certain emotions, attempting to express others, does not allow us to confirm that we should promote positive emotions to exclude negative emotions, or at least not under all circumstances. The complex cognitive processes should need more environment knowledge and more complicated cognitive system. This can be our future research efforts.This paper addresses the emotion-cognition collaboration for the teaching words task and focuses on the competition between the motivation and the emotion. As for motivation, the extraction method is provided, and a different robotic preference  is considered. As for emotion, we suggest autoregressive time series as the emotional transition framework and introduce the cognitive reappraisal to provide as much positive teaching interaction as possible. The experimental results show the effectiveness of the RECS.Initial motivational effects consider not only current sensorimotor experience but also the memory. The competition exists not only between different stimulus but also between current and memory.Initial motivation can be the stimulus of emotional generation, and emotion transition is also related to the emotion of the previous time.Accumulated emotion effects are represented in the rewards in RL as the bargaining counters that compete with the initial motivation.High cognitive reappraisal ability can make the robot generate more positive emotion. And, it is meaningful in teaching environments.To summarize, the major ideas advocated in this paper are as follows:


A novel lightweight diagnostic film with sensors/actuators and a multiple-path wiring option using inkjet printing was developed. The diagnostic film allows for systematic, accurate, and repeatable sensor placement. Furthermore, the film is highly flexible and adaptable for placement on complex configurations. The film can be attached to the surface of the structure through a uniform secondary boundary procedure or embedded within the composite layup during curing. The surface-mounted film can simply be peeled off for repair or replacement without scratching or damaging the part. The film offers significant weight reduction compared to other available technologies. A set of extreme temperature, altitude, and vibration environment test profiles were carried out following the Radio Technical Commission for Aeronautics (RTCA) DO-160 document to assess the durability and performance of the diagnostic film for onboard application. The diagnostic film was shown to be durable and reliable in withstanding the variable operational and harsh environmental conditions of tests representing the conditions of regional aircraft. Structural health monitoring (SHM) based on ultrasonic techniques is now considered to provide an alternative reliable, noninvasive way to test critical structural components. In this case, SHM may involve the employment of macrofiber composite (MFC) piezoelectric actuators ,2, fiberOne of the major impediments in the application of structural health monitoring using PZT-based ultrasonic-guided wave damage detection in aeronautics is the added weight, which is due mainly to the wiring. Added complications include directing and securing the wires through a complex structure such as a fuselage with frames and stringers. For the application of SHM systems at the industrial scale , it is necessary to automate the installation. The factors that are important for scaling up the technology are precision and repeatability in placing the transducers to ensure reliability in acquisition. Another requirement is flexibility of the installation to adapt to any part with complex geometry  without adversely effecting its operation.The decision to employ an SHM system as a maintenance strategy aboard a structure is directly related to the additional cost and reliability of the system as compared to existing schedule-based maintenance and SHM techniques. Furthermore, the installed sensors are usually not protected from the aircraft’s operational environment, making PZT sensor failure a distinct possibility ,21. In tA possible solution to the aforementioned issues is provided by the use of additive manufacturing. Over the last few years, inkjet printing, and especially drop-on-demand (DoD) inkjet printing, has been employed as an effective and cost-efficient technique for the fabrication of electronic devices such as organic thin-film transistors ,21, solaTM transducers integrated between two layers of uncured ethylene propylene diene monomer (EPDM) rubber. The cables for each sensor were soldered prior to their integration. The resulting layer is a flexible array that will be co-cured during the same autoclave cycle as the prepreg, which also acts as a protective layer once it is cured 9 cycles, using a hydraulic INSTRON universal testing machine equipped with a 100 kN load cell  were inkjet-printed on the Kapton surface and five DuraAct sensors were used for actuating and sensing Lamb waves. The thin film was then attached to a CFRP surface and ultrasonic waves were excited with a 250 kHz five-cycle Hanning tone-burst with a signal amplitude of 6 V. Prior to the ultrasonic measurements, electromagnetic impedance spectroscopy (EIS) was employed.As expected, the length of the printed wires had a negligible effect on both the EIS measurements and the signal features, such as amplitude. As stated, the relatively high electrical conductivity of the inkjet-printed wires resulted in a minor attenuation of the excitation signal; see Initially, the diagnostic film was mounted onto the surface of the CFRP and exposed to the environmental test profile depicted in The evolution of the impedance magnitude spectrum for both PZT sensors before and after thermal loading and humidity exposure is presented in The thermal and humidity exposure of the plate to the operational temperature profile did not affect the recorded signals. For both excitation frequencies (50 and 250 kHz), the features of the recorded signals showed minor changes, indicating that the developed thin film acted as a protective layer for both the PZT transducers and the inkjet-printed circuits see .In this section, examples of embedding or surface-mounting of the developed diagnostic film are presented. The diagnostic film can also be embedded into composite. At the end of the curing process a, the coIs flexible, so that it can adapt to any surface.Can be surface-mounted as well as embedded in composites.Is repairable, so it can be easily removed from the surface of the structure and replaced in case of failed sensors.Has an adaptable design: the wires can be of any shape (curved or straight), so it is applicable to any complex geometries and can be readily scaled up.Has low weight .Has a uniform and repeatable bonding quality.Exhibits reliable functionality under operational conditions of aircraft.The design, manufacturing, and testing of a novel flexible diagnostic film for SHM application to aeronautical structures was presented. The diagnostic film has the following novel characteristics:The integrity and reliability of the signal acquisition was assessed under the operational conditions of an aircraft. For that purpose, the developed diagnostic film was subjected to extensive testing that included electrical and mechanical fatigue and thermal loading. The data acquired before and after each test showed successful application of the diagnostic film under loading conditions representative of an aircraft, without any permanent change. The collected data can then be reliably used for any diagnostic application, including impact and/or damage detection and identification of aeronautical structures.


Since the introduction of porcelain laminate veneers in the early 1980s, the anterior esthetic rehabilitation with PLVs has been provided with a conventional restorative technique for many years. Recently, a new concept named digital workflow has been raised that will lead to the abandonment of conventional procedures during the fabrication of PLVs. Digital workflow is a new concept facilitating applications by adapting digital systems to clinical applications, including intra- and extraoral photographs, diagnostic wax-up, mock-up, intraoral camera, and use of the CAD/CAM device. The aim of this case report is to describe the clinical procedures of porcelain laminate veneers using digital workflow with a facilitating step in a single session and to evaluate the PLVs after one year of clinical service. During the last few decades, the demand for the computer-aided design and manufacturing systems in dentistry has dramatically increased because of the rapid developments in computer technology and science . The ideThe clinical technique for porcelain laminate veneers includes bonding very thin restorations to tooth adhesively in order to correct an unesthetic appearance of the anterior teeth . The maiSeveral medium-to-long-term clinical studies have evaluated the clinical performance of PLVs produced by conventional techniques. The success of conventional PLVs in these clinical trials ranges from 100% to 67% [A healthy 19-year-old male patient attended to our clinic with a chief complaint of maxillary anterior teeth because of the fractures Figures . All theCast models were provided from the impressions and a wax-up model was prepared by free-hand technique. The wax-up model was duplicated, and vacuum sheet was prepared on the stone cast model for mock-up. At the second appointment, the final volume of the provisional restorations was made with temporary flowable composite resin  using transparent, rigid, and vacuum-shaped sheets . The patient was able to preview the estimated finished restoration from the provisional restorations. After patient approval of the mock-up , the firAfter finishing the preparations of the teeth, the second digital impression of the maxilla, as well as the first digital impression of the mandible and occlusal bite registration, were taken with the CEREC Omnicam. The mock-up model was copied to the computer, errors on the copy were corrected manually on the computer, and designs of the restorations were completed . The mes2). Restoration margins were finished and further polished with extrafine diamond finishing burs (Diatech), polishing cups , and interproximal polishing strips . Finally, the occlusion was checked in protrusive and lateral movements of the mandible.A light-curing adhesive resin cement  was used for the adhesive cementation of the PLVs according to the manufacturers' instructions. The adhesion surfaces of all the veneers were etched with hydrofluoric acid  for 60 s and subsequently rinsed with water and dried. Monobond S (Ivoclar Vivadent) was applied as a silane for 60 s to the inner surfaces of the veneers. Phosphoric acid  was applied to the prepared tooth surfaces including enamel for 30 s and dentine  for 15 s. Adhesive bonding agent  was applied to both the adhesion surfaces of the teeth and the PLVs for 10 s. Resin cement in the selected translucent value  was applied to the inner surfaces of the veneers. After these procedures, the PLV restorations were positioned, and excess luting cement was removed with hand instruments and a brush. Before final curing, PLVs were cervically precured for 5 s to remove excess resin cement completely from the cervical and interproximal areas using hand instruments and dental floss without any pressure. For each of the PLVs, these processes were separately made and the PLVs were cemented one by one before the final cure. Final curing was performed according to the manufacturer's instructions for 40 s on each surface  with a light-emitting diode polymerizing unit (Elipar S10, 3M ESPE; Neuss, Germany; light output: 1200 mW/cmPLV restorations produced in a single session with CAD/CAM provided the patient's aesthetic rehabilitation and satisfaction quickly Figures  and 12. This clinical report described how to achieve an esthetic outcome for anterior teeth as minimal invasive as possible with PLVs produced by digital workflow. In addition, to facilitate the computer design, a wax-up model was prepared in the laboratory from the impressions taken immediately in the first visit of the patient. Esthetic rehabilitation of the 4 maxillary anterior teeth was quickly provided with the chairside porcelain laminate veneers produced by CAD/CAM, and the patient satisfaction was provided. Even after one year, the restorations were in very good condition.The production of a restoration with CAD/CAM includes step by step processes that consist of selection of the tooth, type of restoration, material to be used, selection of the restoration design, taking the digital impressions with intraoral camera, designing the restoration, and finally sending the data to the milling machine to print out the restoration . Three oDifferent clinical methods can be found in the dental literature to produce porcelain laminate veneers with CAD/CAM. Zandinejad et al.  preparedA recent clinical study was published and reported the survival of chairside CAD/CAM porcelain laminate veneers . The autPLVs can be produced with CAD/CAM systems in perfect harmony to the teeth with minimal error. The digital workflow facilitates the relationship between dentist and patient. The mock-up model to be prepared can be created and applied by the computer first and can also be implemented by the free-hand technique with heated wax-up before computer design. The clinician can perform whatever technique is more feasible for him or her. A differently modified technique, as here, can help the clinician in designing the restorations with CAD/CAM, especially for the multiple anterior teeth.


Landmark-based vehicle localization is a key component of both autonomous driving and advanced driver assistance systems (ADAS). Previously used landmarks in highways such as lane markings lack information on longitudinal positions. To address this problem, lane endpoints can be used as landmarks. This paper proposes two essential components when using lane endpoints as landmarks: lane endpoint detection and its accuracy evaluation. First, it proposes a method to efficiently detect lane endpoints using a monocular forward-looking camera, which is the most widely installed perception sensor. Lane endpoints are detected with a small amount of computation based on the following steps: lane detection, lane endpoint candidate generation, and lane endpoint candidate verification. Second, it proposes a method to reliably measure the position accuracy of the lane endpoints detected from images taken while the camera is moving at high speed. A camera is installed with a mobile mapping system (MMS) in a vehicle, and the position accuracy of the lane endpoints detected by the camera is measured by comparing their positions with ground truths obtained by the MMS. In the experiment, the proposed methods were evaluated and compared with previous methods based on a dataset acquired while driving on 80 km of highway in both daytime and nighttime. Vehicle localization is one of the key components of both autonomous driving and advanced driver assistance systems (ADAS). Accurate localization results can improve performances of other key components such as perception, planning, and control. The most widely used vehicle localization method is global navigation satellite systems (GNSS). This method provides global positions and its error is not accumulated, but produces inaccurate results when the GNSS signal is reflected or blocked. To alleviate this problem, GNSS has often been fused with dead reckoning (DR). This method is robust against the status of the GNSS signal and provides accurate results in a short period of time, but its error can accumulate over time. Recently, to overcome these drawbacks, localization methods that utilize a perception sensor and digital map have been widely researched . These mVarious types of landmarks have been used, and road markings are one of the most widely used landmarks for vehicle localization. In terms of the perception sensor, road markings are relatively easy to detect because their shapes are predetermined by regulations and their color and reflectivity are significantly different from those of the road surface. In terms of the digital map, road markings can be stored at low capacity because their shapes are relatively simple compared with other road structures. Among road markings, lane markings are most widely used for vehicle localization. Lane markings provide abundant information on lateral position, but lack information on longitudinal position. To alleviate the drawback of lane markings, methods that utilize stop lines, crosswalks, arrows, and letters have been suggested. Those road markings are useful in urban situations because they frequently appear on urban roads. However, in highway situations, stop lines and crosswalks rarely exist, and arrows and letters seldom appear. To overcome the limitations of the abovementioned road markings, lane endpoints can be used as landmarks to improve longitudinal positioning accuracy. The lane endpoint indicates the location where a lane marking starts or ends, so this paper classifies it into a lane starting point and a lane ending point. This paper proposes two methods concerning the use of the lane endpoints as landmarks for vehicle localization. First, it proposes a method to efficiently detect the lane endpoints using a vehicle-mounted monocular forward-looking (front) camera, which is the most widely installed perception sensor. Second, it proposes a method to reliably measure the position accuracy of the lane endpoints detected from images taken while the ego-vehicle is moving at high speed.Since the front camera module is responsible for detecting various objects such as lane markings, vehicles, pedestrians, traffic signs, etc., an algorithm added to this module should require less computational costs. Thus, this paper proposes a method that can efficiently detect lane endpoints by combining simple algorithms. The proposed method first extracts lane pixels using a top-hat filter, and detects left and right lanes based on RANSAC. Once the lanes are detected, profiles of the top-hat filter response are generated along the detected lanes. Lane endpoint candidates are generated by finding local minima and maxima from differentiation results of the profiles. The lane endpoint candidates are verified by a two-class classifier after converting them into bird’s-eye view images. Finally, the positions of the lane endpoints from the camera are calculated based on the camera’s intrinsic and extrinsic parameters.In general, the landmark-based localization system estimates the position of the ego-vehicle by comparing the positions of the detected landmarks from the ego-vehicle and those stored in the digital map. This means the position accuracy of the detected landmark is directly related to the performance of the vehicle localization system. In addition, it is necessary to have a distribution of the position error of the detected landmark in order to effectively apply extended Kalman or particle filtering methods. However, it is quite difficult to precisely measure the position accuracy of the landmarks, which are detected from images taken while the ego-vehicle is moving on a highway at high speed. According to our literature review, there has been no previous work attempting it. To reliably measure the position accuracy of the landmark, this paper proposes a method that uses a mobile mapping system (MMS) after examining various approaches. Since the MMS consists of high-precision positioning equipment and high performance LIDARs, it produces highly accurate positions and dense 3D points with reflectivities. If the front camera is attached to the MMS, 3D locations of the detected lane endpoints from the front camera can be accurately obtained while the ego-vehicle is moving at high speed by using the precise positions and dense 3D points produced by the MMS. Thus, this paper reliably measures the position accuracy of the lane endpoints by comparing its position obtained from the camera and the corresponding position obtained from the MMS.In the experiment, the proposed method was evaluated based on a dataset acquired while driving on approximately 80 km of highway. Half of the dataset was acquired during daytime and the other half at night. The proposed lane endpoint detection method shows 96.1% recall and 99.7% precision in daytime, and 94.7% recall and 100% precision at night. As a result of the MMS-based position accuracy evaluation, the detected lane endpoints show 7.8 cm and 21.6 cm average position errors in lateral and longitudinal directions during daytime, and 8.2 cm and 48.2 cm average position errors in lateral and longitudinal directions at night. In terms of execution time, this method requires only 4.35 ms per image so that it can handle 230 frames per second.The rest of this paper is organized as follows: The perception sensor and digital map-based vehicle location methods can be categorized based on the information acquired by the perception sensor and stored in the digital map. According to this criterion, previous methods can be categorized into range data-based, feature point-based, and road marking-based approaches. Since the proposed system is categorized into the road marking-based approach, the other two approaches are briefly introduced in this section. The range data-based approach acquires range data using perception sensors and matches them with range information stored in the digital map. Since this approach mostly utilizes active sensors such as LIDARs and Radars, it is robust against illumination conditions and textures of surrounding objects. However, its performance can be degraded in cases where a few fixed objects and many moving obstacles are presented on a broad road. The methods in this approach recognize specific objects  with moment invariants , differeThe road marking-based approach extracts markings on road surfaces and matches them with road markings stored in the digital map. Since road markings are visually distinctive and under government regulations, they can be reliably detected compared to other objects. However, localization performance can be degraded when the road markings are worn or covered with snow. The road marking-based approach can further be categorized into signal-level, feature-level, and symbol-level approaches according to the level of information being used.The signal-level approach uses raw data of road surfaces obtained from the perception sensor. Mattern et al.  matches The symbol-level approach recognizes a variety of types of road markings and matches them with those stored in the digital map. Lane markings are the most popularly used symbols for vehicle localization. Nedevschi et al. , Jo et aThe lane endpoints are expected to be good landmarks for vehicle localization when applying the symbol-level approach to highway situations. This paper proposes two essential components for its application: a cost-effective method to detect the lane endpoints using a conventional monocular front camera and a reliable method to measure the position accuracy of the lane endpoints based on the MMS. According to our literature review, there has been no previous work that explicitly detects the lane endpoint for vehicle localization purposes and measures its position accuracy.h can be described asn is an n-dimensional row vector whose elements are all ones. The top-hat filter can successfully detect lane candidate pixels if w fits the lane marking width. The lane marking width changes according to the vertical coordinate of the image because of the perspective projection. Thus, this paper pre-calculates the lane marking widths at different vertical coordinates based on the camera’s intrinsic and extrinsic parameters and the width of the lane marking in the regulation. In this pre-calculation, it is assumed that the roll and yaw angles of the camera are close to zero and only markings of the ego-lane are considered. The response of the top-hat filter at location , r can be cost-efficiently calculated using only four operations regardless of w when a horizontal integral image, hI is used as:For lane departure warning (LDW) and lane keeping assist (LKA) systems, distant lanes and their curvatures should be obtained. However, in this paper, lane detection results are used only to restrict the search range of lane endpoints. For this purpose, it is enough to detect the lanes close to the ego-vehicle. Since the highway has a large turning radius, the lanes close to the ego-vehicle can be approximated as straight lines. Thus, the lanes within 20 m of the front camera are detected by finding a pair of straight lines. To detect left and right lanes, this paper first designates regions of interest (ROIs) for both lanes by considering the camera’s intrinsic and extrinsic parameters and the lane width of the highway in the regulation. Two pairs of blue lines in N  consecutive images. Two consecutive vanishing points are considered to be located in a similar position if the distance between them is less than a predetermined value. Once the lane markings are stably detected, the ROIs for left and right lanes are limited to the narrow areas near the lane markings detected in the previous image. Even though this paper detects lanes using a combination of simple algorithms to reduce computational cost, more sophisticated methods can also be used.p can be represented as:p(y) indicates the value of the lane profile at the vertical image location, y. N and M are the starting and ending locations of the lane profile, and they correspond to the road surface of 5 m and 20 m in front of the front camera, respectively. p(y) is calculated from the top-hat filter response as:r is the value of the top-hat filter response at the image location , and Lx(y) is the horizontal image location of the lane marking at y. p(y) is calculated not only using r(xL(y),y) but also using its horizontally neighboring values, r(xL(y) − 1,y) and r(xL(y) + 1,y) in order to reduce the influence of the lane detection error on the generation of the lane profile. y, d(y) is calculated by subtracting the averages of the lane profile values before and after 1 m from y as:ym−1 and ym+1 are the vertical image locations before and after 1 m from y, respectively. The lane endpoint candidates are generated along the detected lanes. To this end, lane profiles are first calculated by following the detected left and right lanes. Those profiles are not generated based on the brightness values of the image, but the response of the top-hat filter already obtained in the lane detection procedure. Direct use of the brightness values is sensitive to lighting conditions because even objects of the same color in the real world can have different brightness values in images depending on local lighting conditions. However, since the response of the top-hat filter is calculated based on the brightness difference between surrounding areas, it is relatively robust against lighting conditions. The lane profile, Red and blue triangles in In real situations, the candidates of the lane endpoints (including starting and ending points) may include some false detections due to preceding vehicles, stains, shadows, etc. From the viewpoint of the vehicle localization, the false detections can more severely degrade the localization accuracy than the miss detections. If landmarks are miss-detected, the matching between the map and landmarks is omitted and the vehicle localization is performed with DR, which slightly degrades the localization accuracy. However, if landmarks are falsely detected, the localization accuracy may be significantly degraded due to the matching of the map with the wrong landmarks . TherefoTo this end, the area of 1 m × 2 m around the lane endpoint candidate is first transformed into a bird’s-eye view image. In this transformation, the vertical direction of the bird’s-eye view image is set to coincide with the direction of the detected lane. Crosses in xi is the i-th support vector and N is the number of support vectors. iα is a coefficient of xi, and b is a bias. x is an input feature vector and iy indicates the class of xi, which is either −1 or +1. Φ is a kernel function. In case of the linear SVM, (6) can be simplified as:The resolution of the bird’s-eye view image is 48 × 96 pixels, and the parameters of the HOG are as follows: number of orientation bin: 9; cell size: 8 × 8 pixels; block size: 2 × 2 cells; and block overlapping: 50%. Thus, the HOG feature dimension is 1980. After extracting the HOG feature, the SVM is used to determine whether the candidate is correct. In order to reduce the computational cost, the linear SVM is selected. In general, the classification function of the SVM is defined as:iα, iy, and xi can all be determined at the learning stage, w can be calculated in advance. Therefore, the test stage can be simplified by adding the bias, b to the inner product of two vectors, w and x regardless of the number of support vectors. This simplification significantly reduces the computational cost. If a lane endpoint candidate is classified as a correct one based on the HOG-SVM classifier, it is confirmed as a final detection result. Details of the learning and evaluation of the HOG-SVM classifier will be described in the experimental section. Since B2C and tB2C are rotation matrix and translation vector that represent a rigid transformation between the checkerboard and camera coordinate systems, an image location,  and a 2D location in the XY-plane of the checkerboard coordinate system,  are related by a homography, H:To effectively utilize the detected lane endpoints for vehicle localization, their locations with respect to the camera should be calculated. This paper calculates the location of the lane endpoint based on the camera’s intrinsic and extrinsic parameters, which are obtained during the precalibration stage using planar checkerboards. In case of calibrating the extrinsic parameters, the longitudinal direction of the vehicle is set to the Y-axis, the ground to the XY-plane, and the direction perpendicular to the ground to Z-axis in the checkerboard coordinate system. If RSince the origin of the 2D location of the lane endpoint should be the origin of the camera, H in (8) is manipulated as:CX, CZ) can be calculated from the image location,  using the homography, H’ as:CX and CZ are called the lateral and longitudinal locations of the lane endpoint, respectively.Based on (9), the 2D location of the lane endpoint having the camera location as its origin,  is one of the most widely used instruments, and is mainly used for obtaining high resolution 3D models of large structures such as dams, mines, and terrain. To evaluate the position accuracy of the road marking using this method, TLS and the monocular front camera are first installed together and the rigid transformation between them should be precalibrated. The ground truth position of the road marking is obtained from 3D points produced by the TLS, and its position is transformed into the camera coordinate system using the rigid transformation between the TLS and camera. Finally, the position accuracy is measured by comparing the transformed ground truth position and the position calculated by the image of the camera. This method has an advantage of providing a dense and precise 3D shape of the road marking. However, it can be used only in cases where the vehicle is stationary because the TLS takes a long time to scan surrounding areas. For the same reason, it is difficult for the TLS to scan long and narrow areas such as highways. Due to these limitations, it has been concluded that this method is inappropriate to use in highway situations.The on-vehicle sensor-based method measures the relative position from the camera to the road marking using various ranging sensors that have already been mounted on commercialized or autonomous vehicles. The most widely used ranging sensors mounted on those vehicles are a stereo camera and multi-layer LIDAR. This method is similar to the measuring instrument-based method. However, there is a major difference, which is the sensing speed. The sensing speed of the vehicle mounted ranging sensors are designed to be fast so that they can quickly obtain the ranging data of the surrounding area while the vehicle is moving at high speeds. To evaluate the position accuracy of the road marking using this method, the stereo camera or LIDAR along with the monocular front camera are first installed on the same vehicle and the rigid transformation between those sensors should be precalibrated. The ground truth position of the road marking is obtained from a dense disparity map of the stereo camera or 3D points produced by the multi-layer LIDAR, and its position is transformed into the camera coordinate system using the rigid transformation between those sensors. Finally, the position accuracy is measured by comparing the transformed ground truth position and the position calculated by the image of the camera. This method has an advantage that it is applicable to situations where the vehicle is moving at high speeds. However, in the case of the stereo camera, its ranging accuracy severely deteriorates when the distance of the road marking increases. In the case of the multi-layer LIDAR, it provides a high ranging accuracy, but its range data is not dense enough to accurately localize the road marking. Due to these limitations, it has been concluded that this method is inappropriate to use.The MMS-based method measures the relative position from the camera to the road marking using dense 3D points and accurate positions obtained by the MMS. The MMS consists of high-precision positioning equipment and high performance LIDARs, and those sensors are usually mounted on a vehicle. Since the MMS can measure dense and accurate 3D points while the vehicle is moving at high speeds, it has an advantage of measuring long and wide areas such as highways. To evaluate the position accuracy of the road marking using this method, the monocular front camera is first attached to the MMS and the rigid transformation between the camera and MMS should be precalibrated. The ground truth position of the road marking is obtained from 3D points and its position is transformed into the camera coordinate system using the precalibrated rigid transformation and accurate positions produced by the MMS. Finally, the position accuracy is measured by comparing the transformed ground truth position and the position calculated by the image of the camera. Compared to the measuring instrument-based method, the MMS-based method has an advantage that it can be used while the vehicle is moving at high speeds. Compared to the on-vehicle sensor-based method, its 3D structure is denser and more accurate. Because of these advantages, this paper has selected the MMS-based method for evaluating the position accuracy of the road marking. A detailed explanation of this method will be presented in the following sections.The MMS used in this paper consists of a high precision LIDAR as a ranging sensor and a combination of real-time kinematic-global positioning system (RTK-GPS), high precision inertial measurement unit (IMU), and distance measurement instrument (DMI) as a positioning sensor. The mapping and positioning sensors are mounted on an off-the-shelf vehicle as shown in To evaluate the position accuracy of the lane endpoint, the ground truth position of the lane endpoint is manually designated using the 3D points obtained by the MMS. Since the manually designated position is in the 3D point cloud coordinate system, it is converted into the camera coordinate system using the precise camera position at the time of capturing the image, which is provided by the MMS. The position accuracy is evaluated by comparing the ground truth position and the position produced by the proposed method.To manually designate the lane endpoints in the 3D points, roads and lanes should be visually distinguishable. However, since the roads and lanes are located on the same plane, it is hard to distinguish between them using their 3D information. Thus, this paper utilizes the infrared reflectivities provided by the LIDAR of the MMS to visually distinguish the roads and lanes. The 3D points of the lanes have high infrared reflectivities compared with the roads because they are drawn with a highly reflective paint. The positions of the lane endpoints are manually designated by displaying the 3D points based on their infrared reflectivities. GP in the point cloud coordinate system is first transformed into the camera coordinate system, GC as:PC2 and tPC2 are the rotation matrix and translation vector that represent a rigid transformation between the point cloud and camera coordinate systems. RPC2 and tPC2 are obtained from the high precision positioning sensor of the MMS. GC is manipulated as:BC2 and t′BC2 are the rotation matrix and the translation vector that represent a rigid transformation between the camera and checkerboard coordinate systems, which has been already explained in (8) and (9). GX and GZ are the ground truth position of the lane endpoint in lateral and longitudinal directions, respectively. The position accuracy of the lane endpoint is calculated by comparing (GX and GZ) in (12) and (CX and CZ) in (10).Since the position of the lane endpoint is calculated with respect to the camera position in Experiments were conducted using a dataset acquired by the monocular forward-looking (front) camera and the MMS mounted on the roof of the vehicle while driving on highways during the day and night. The experimental setup has been shown in This section presents the performance evaluation and comparison results of the lane endpoint verification explained in This paper compares two methods: the HOG-SVM and CNN. The CNN-based classifier has been widely used for a variety of applications. This paper chose the one that has been used for classifying road markings in  for perfTwo classifiers were trained by the same training samples and tested by the same test samples shown in This section presents the performance evaluation and comparison results of the lane endpoint detection, which consists of the lane endpoint candidate generation and verification. To this end, this paper randomly selected 1200 images from the test dataset. Of these, 600 images are from the dataset taken in daytime, and 600 images at night. The daytime and nighttime images include 1504 and 1290 lane endpoints, respectively. This paper manually confirms whether the locations of the detected lane endpoints match the location of the actual lane endpoints. The performance evaluation and comparison were conducted based on the following three criteria: recall, precision, and F-measure. The better the classification performance, the greater the value of all three criteria:The proposed method was quantitatively compared with the method suggested in . The metlone), lateral error (late), and Euclidean error (euce) as:GX, GZ) and  are the locations of the ground truth and detected lane endpoints, respectively. lone indicates the position error in the vehicle traveling direction, late indicates the position error in the direction perpendicular to the vehicle traveling direction, and euce indicates the Euclidean distance between two positions.The position accuracy of the lane endpoint was evaluated using the MMS-based method explained in lone, late, and euce, respectively. In nighttime, the proposed method gives 48.2 cm, 8.2 cm, and 49.9 cm for lone, late, and euce, respectively. In the case of the highway situation, the lane width is more than 3.5 m and the distance to the preceding vehicle is several tens of meters. Therefore, it can be said that the longitudinal and lateral errors of the lane endpoints detected by the proposed method are sufficient to be used as landmarks of the vehicle localization system for highway autonomous driving. According to Furthermore, at night, the longitudinal errors of the LEP and REP are larger than those of the LSP and RSP by approximately 20 cm. This is due to the nature of the motion blur that occurs at the lane endpoints. When the front camera is moving at high speed and capturing images, the motion blur makes the brightness of the lane starting point (LSP or RSP) slightly dark. Since the lane marking is much brighter than surrounding areas, a slight darkening does not affect the locations of the detected lane starting points. However, in the same situation, the motion blur makes the brightness of the region near the lane ending point (LEP or REP) fairly bright. This makes the proposed method erroneously detect the location brightened by the motion blur and it leads to a relatively larger longitudinal error.In To increase the accuracy of landmark-based vehicle localization, this paper proposes using lane endpoints as landmarks. Two methods are proposed regarding the lane endpoints. One is to efficiently detect the lane endpoints using a conventional front camera, and the other is to reliably measure the position accuracy of the detected lane endpoints. In the experiment, it was found that the proposed detection method can accurately find the lane endpoints in both daytime and nighttime with a small amount of computation and the proposed MMS-based method can reliably measure the position accuracy of the lane endpoints detected from images taken while the vehicle is moving at high speed. In the future, we are planning to develop a method that can reduce the influence of the motion blur when calculating the positions of the lane endpoints at night, and extend the proposed position accuracy evaluation method to other landmarks such as arrows, letters, and traffic signs.