One of the main factors in controlling infectious diseases such as COVID-19 is to prevent touching preoral and prenasal regions. Face touching is a habitual behaviour that occurs frequently. Studies showed that people touch their faces 23 times per hour on average. A contaminated hand could transmit the infection to the body by a facial touch. Since controlling this spontaneous habit is not easy, this study aimed to develop and validate a technology to detect and monitor face touch using dynamic time warping (DTW) and KNN (k-nearest neighbours) based on a wrist-mounted inertial measurement unit (IMU) in a controlled environment and natural environment trials. For this purpose, eleven volunteers were recruited and their hand motions were recorded in controlled and natural environment trials using a wrist-mounted IMU. Then the sensitivity, precision, and accuracy of our developed technology in detecting the face touch were evaluated. It was observed that the sensitivity, precision, and accuracy of the DTW-KNN classifier were 91%, 97%, and 85% in controlled environment trials and 79%, 92%, and 79% in natural environment trials (daily life). In conclusion, a wrist-mounted IMU, widely available in smartwatches, could detect the face touch with high sensitivity, precision, and accuracy and can be used as an ambulatory system to detect and monitor face touching as a high-risk habit in daily life. Stopping or slowing the spread of infectious diseases, particularly COVID-19, is a top priority for governments at the national and community levels around the world. Measures and regulations are strictly enforced to control both direct and indirect transmission of infection . Face toThe measurement systems for human motion tracking and gesture recognition can be categorized into two types: (1) stationary motion tracking systems which usually use cameras, including stereo cameras and depth-aware cameras that are with or without markers –9., and Flex/stretchable and EMG sensors are among the common wearable sensors that have been used to recognize the wrist and finger motion by mapping the skin and muscles to a specific wrist or finger motion , 15, 21.Wearable sensors were also used to detect face touch. Previous research studies proposed methods for face touch detection using an accelerometer, magnetometer, and acoustic-based system and studied the validity of these methods –33. MaruBased on IMU’s technical and practical capacities, we hypothesized that the recordings of a wrist-worn IMU representing the wrist motion could be used to recognize the face touch gesture with high sensitivity and precision. High sensitivity is required to minimize the risk of unrecognized face touching, while high precision is needed to minimize the number of false alarms that help increasing the technology’s compliance during daily life. Therefore, the objectives of this study are to (1) develop a technology to detect and monitor the face touch using a wrist-worn IMU based on a machine learning technique, (2) validate the developed technology in controlled environment trials and (3) investigate the accuracy of our developed technology under natural environment (daily life).To have a minimal effect on the pattern of hand motion and daily life activities, a stand-alone IMU module  that included an accelerometer and gyroscope was used to record the three-dimensional (3D) acceleration and angular velocity at a sampling frequency of 128 Hz. The IMU was mounted on the dominant wrist of the participants using a Velcro strap , and theEleven volunteers  participated in the study. The experiment procedure was approved by the Research Ethics Board Committee of the University of Alberta (Pro00102526) and all methods were performed in accordance with the approved experiment procedure. Participants were asked to read and sign the written informed consent form prior to participating in the experiment. The experiment included two parts: controlled environment and natural environment trials.During the controlled environment trials, the participants were asked to perform six different scenarios  each for five times. Each scenario contained at least five repetitions of a single gesture leading to at least 25 repetitions per participant for every single gesture. They performed these motions with their dominant hand while the IMU recorded their wrist motion . In addiIn addition to controlled environment trials, natural environment trials were included to investigate the ability of our developed algorithms in detecting gestures in daily life activities. In natural environment trials, participants were asked to follow their daily life activities while occasionally performing the abovementioned gestures  and log their activities for 20 minutes. As a result, 20 minutes of the participants’ routine daily life activity which contained a known number of face touches were recorded while the IMU was mounted on their dominant wrist. Similar to the controlled environment trials, each 20-minute trial was also started and ended with 10 seconds of in the rest position recording.Participants logged the approximate instants in which face touches were performed in a log sheet which was then used to label the IMU recording. Additionally, a repetition of face touching gesture was judiciously selected from the previously recorded data as a template which was then used for feature extraction.th-order low-pass Butterworth filter with a cut-off frequency of 30Hz. Then, the IMU readouts were transformed from the IMU’s coordinate system into a vertically aligned coordinate system .Feature extraction was performed on the 3D acceleration and angular velocity recordings obtained by IMU that represented the hand motion during the trials. The similarity of the template to the 3D acceleration and angular velocity recordings was obtained using DTW. A repetition of face touch gesture was judiciously selected from the previously recorded data as a template. The template consisted of acceleration and angular velocity recordings. In order to calculate the similarity score between the template and the recorded acceleration and angular velocity, we constructed a cost matrix. Then, the total distance with the least cost was defined as the DTW score . DTW wask nearest neighbours in feature space. The major voting classification was performed for the k equal to 3 to 21. For the purpose of training, the feature vectors among the controlled environment and natural environment trials were randomized separately and then split into 70% and 30% to form the training and test set, respectively. A total number of 82,822 instants that included 1,562 instants of face touching and 38,523 instants of the confounding gestures existed in controlled environment trials datasets, and a total number of 67,190 instants that included 761 instants of face touch existed in natural environment trials datasets.We chose the KNN classifier which is a non-parametric classification method to classify the data and detect face touches. First, the feature space containing the feature vector (DTW scores) and class labels was created  using thk (number of neighbours) in the range of 3 to 21, each instant in the test set was classified using the KNN model and compared to the true label vector  for 11 folds were calculated.The classification was performed for each instant in the test set using the KNN model developed based on the training set. Then, for each l vector . Note thk in the range of 3 to 21  as the sensitivity, precision, accuracy, FPR, and FDR did not considerably change with k. In natural environment trials, the KNN classifier was able to detect the face touch with substantial sensitivity, precision, and accuracy for k ranging from 3 to 21. Yet, unlike controlled environment trials, the sensitivity and accuracy decreased by increasing the value of k, caused by an increase in FN and FP. This agrees with the fact that as k increases the neighbourhood becomes less distinct and the classifier becomes less sensitive (to decrease the sensitivity the average of the three feature vectors based on acceleration and the average of the three angular velocities were added to the feature space). Similarly, the sensitivity, precision, and accuracy tended to be higher for k<13 during both controlled environment and natural environment trials. Based on our observations, we recommend either k = 3 or k = 5 for face touch detection in the controlled environment and natural environment trials. The average sensitivity, precision, and accuracy values obtained for different k in Leave-One-Subject-Out Cross Validation were higher during the natural environment trials compared to the controlled environment trials. That could be due to the fact that confounding gestures occurred more in the controlled environment trials compared to the natural environment trials that could increase the chance of mislabeling of the face touches. The limited information about the task performed during the natural environment prevent us from any further investigation.The sensitivity, precision, and accuracy obtained with the DTW-KNN method for detecting the hand gesture and distinguishing the face touch from the confounding gestures in our present study were in the same range with the ones previously obtained for face touch and other type of hand motion. Our obtained sensitivity, precision, and accuracy agreed with the results presented in studies that investigated the validity of face touch detection using wearable sensors –34 and iThere are a couple of limitations to this study that should be mentioned. First, our proposed method was developed based on data collected from the dominant hand of the participants. Second, although it is very common to use the feedback mechanism to improve personal behaviours like activity level, the effect of feedback mechanisms on behaviour change is not fully discovered. Future works should investigate the effectiveness of feedback that alert a person in lessening the number of face touches. Third, a larger sample size and a more complex experimental protocol should be conducted to account for more variation of face touching to improve the developed data-driven models and assess the efficacy of our proposed DTW-KNN method. Fourth, due to the fact that the DTW score calculation is computationally demanding, in future studies, the feasibility of using our proposed method in real-time detection of face touch should be investigated. Fifth, the train data set was unbalanced and might result in misclassification. In future work, undersampling and oversampling method could be used to balance the training set. Sixth, because the exact time of the face touch was not recorded in the log sheet, the labelling procedure implemented for the natural environment trials may introduce errors that should be considered when discussing the results.In this study, we proposed a system to detect face touch and distinguish it from other confounding hand motions using the recordings of a wrist-worn IMU and based on a DTW-KNN model. We investigated, (1) the practicality of using KNN-DTW method in detecting face touch, (2) the validity of our method in daily life circumstances, (3) the feasibility of distinguishing the face touch from other confounding gestures including eating, drinking, scratching head and reaching and picking an object is investigated. To do that, the experimental procedure and the sensor were selected in a way to have minimal effect on the participants’ pattern of motion. Additionally, the data was collected using a stand-alone sensor (that feels like a smartwatch on the participants’ wrist) while the participants were busy with their daily routines without any interruption. Our proposed method was able to detect face touches and distinguish them from confounding motions such as drinking, eating, scratching the head, and reaching and picking an object. The highest sensitivity, precision and accuracy were obtained by the DTW-KNN model with, either 3 or 5 clusters. Our proposed wearable technology can be used for ambulatory monitoring of the hand motion and sending alarm on face touching as a high-risk habit during daily life.


This paper deals with the development of dimensional control technology for the production of ADT (accessory drive train) gearbox housing. The project included the development of a robotic geometry inspection station for thin-walled aerospace casings. Laser profilometers from two brands, Cognex and Keyence, were used to measure. A proprietary software solution for arc measurements is presented. The obtained solutions were compared and verified in accordance with the requirements of the manufacturer Pratt and Whitney Rzeszów S.A. The results of the work indicated that correct solutions were obtained with a very large reduction in control time. In addition, the measurement is fully automated and transferable to the company’s electronic systems. During the production and quality inspection of thin-walled aerospace casings, a great number of geometric parameters must be measured. The parameters can be acquired with three different methods: by taking imprints and displaying their values with a measurement projector; a contact method using a hand-operated laser profile measurement tool; or a contact method using a contact probe operated by a CMM (coordinate measuring machine). Each of these methods has advantages and weaknesses.When taking imprints, its slicing and display with a measurement projector is a time-consuming process, burdened with the potential for measurement errors as it is directly handled by a human (human factor). A manual measurement method with a hand-operated laser profile measurement tool does not ensure a reliable result given the potential incorrect orientation of the tool’s head, which should be perpendicular to a tangent of the measured surface, difficult access to the measured area, measurement operator’s fatigue, etc. The first of the two methods discussed requires considerable time, is prone to human factors and does not facilitate the automatic generation of measurement reports.Measurements with CMMs can only be undertaken in suitable laboratory conditions. This method requires the removal of the casing to be measured from the machine fixture, handling the casing to a testing laboratory, waiting until the casing reaches temperature equilibrium, and installing the casing with the reference systems retained inside of the measuring area of the CMM. This requires more time and space. The advantages of CMMs are the great flexibility of applications and accuracy. The greatest weakness of this method is the high purchase and maintenance cost of a CMM and the long measurement time ,2.Minimizing production costs implies, among other things, reducing time-consuming product inspection processes, which results in the search for alternative methods. Operations of dimensional and geometric tolerance control have so far often been performed on coordinate measuring machines (CMMs). Contact measurement with a CMM has well-defined accuracy and measurement uncertainty . A cheapMeasuring geometrical features such as full circle radii, chamfer widths, small diameters, or distances with a 2D scanner is a typical application solution and does not present a problem. The main problem solved in this paper is to give an algorithm for measuring radii performed as blunts of sharp surfaces. The difficulty arises from the need to adapt the measurement field, the number of selected points, and to propose a way to average the measurement. This is an approximation problem of inscribing a radius into a point-defined curve, realized under the software limitations of the Keyence and Cognex 2D scanner controllers being used.In the era of automating industrial solutions, a robotised geometric inspection of workpieces was proposed; the system is based on laser beam measurements with profilometers manufactured by Cognex and Keyence, whose displacement, positioning, and orientation are performed by a robotic arm. The specific nature of geometric measurements required the development of processing algorithms for the results and the design of a robotised inspection station. A verification measurement could be triggered during the machining of the casing or directly after the machining process, with the measurement results stored in a database to retrieve a final inspection report.A Pratt & Whitney-approved measurement method using imprints was used as the standard to which the obtained measurement results were related.An example of thin-walled aerospace casings is the ADT (accessory drive train) gearbox. The ADT  is a jetEach transmission consists of several hundred components, the most important being the main body gearbox and cover. The gearbox must meet the highest requirements in terms of quality and precision, which is why it is essential to check the geometric parameters during production and at the release stage. The production of ADT gearboxes for the P&W turbofan engine PW1500G series used in Airbus A220 aircraft involves manufacturing thin-walled aluminium alloy castings with a complex geometry . The approximation error is measured with the Chebyshev standard .‖f − F‖continuous approximation, where function z = f(x) is determinate in the interval of . The approximation error is expressed with the following relation:For a mean-square approximation, two cases may exist:discrete approximation, where function z = f(x) is discrete, meaning that its known values can be tabulated.For the inspection software discussed here, a continuous mean-square approximation was performed. It was not possible, however, to intervene in the method of approximation or to determine the approximation error; controller software developers do not provide this option. The only accessible, programmable parameter was the measurement area width, or the interval of . To determine the relation between the measurement area width and the measurement accuracy, it was decided to measure the gauge elements made on the CNC machine with the accuracy of 0.05 mm, and this value was taken as the nominal one in the tests . In thisDuring the tests, for known fillet radii the measurement area width was increased in increments of 0.1 mm .The starting width of the measurement area was 0.3 mm. It was found that the measured fillet radius value was below the nominal value, followed by an increase and stabilisation around the nominal value in several successive steps of digitisation. When the limit value of the measurement area width was exceeded, the measured value greatly exceeded the nominal values. The tests were completed for a range of fillet radii from 0.25 mm to 3.5 mm. The completed tests facilitated the determination of the relation between measurement area width and inspection system error and the development of a measurement algorithm that could be implemented with the available hardware .The algorithm  was succIn the course of the tests of the developed inspection system solutions, the functioning of the proposed algorithms was verified.A comparison was made between the results produced with the algorithms developed for the Keyence and Cognex solutions with the method of taking an imprint by a qualified employee of Pratt and Whitney Rzeszów S.A. To test the functioning of the inspection systems, fillet radii R01, R02, and R03 were measured in the assumed range. Each fillet radius was measured five times. The reference value adopted was the imprint measurement method, as this method was validated and approved by Pratt & Whitney S.A. Arithmetic mean was determined from the measurement values, and standard deviation was determined for the measurements made with laser profile measurement systems. The produced results proved to be satisfactory, according to prior assumptions.For this paper, a robotised geometric inspection of thin-walled aerospace casings was developed and characterised. Existing methods of geometric feature measurement were analysed, and their advantages and weakness were listed. Automatic measurement methods viable for application with industrial robots were reviewed. The measurements with laser profile measurement systems were selected, and two solutions for these measurements, each provided by a different manufacturer, were tested. The main contribution of the research is the development of the algorithm and its implementation in the measuring device. This algorithm is designed to increase or decrease the width of the measuring field depending on the tested radius (0.4 to 1.8 mm) and to determine its different values automatically. The developed geometry inspection systems, together with the implemented algorithms, successfully performed the designed test tasks. The robot’s parameters, such as accuracy and repeatability, had no effect on the measurement accuracy. The test results achieved were satisfactory; following consultation with the representatives of Pratt and Whitney Rzeszów S.A., the discussed geometric inspection system solution can be potentially applied in a production cell. The robot ensured satisfactory repeatability of positioning and orientation relative to the workpiece measured by the scanner heads discussed here, which, serving as geometric measurement tools, proved to be very precise and fast in operation (with approximately 3 s for a single measurement point). Following the measurement inspection, the geometric values acquired were stored in an electronic database for post-processing. What is also especially important, the measurements could be integrated with the ADT casing machining workstation. In the process of machining, check inspections of specific parameters can be performed and correct the features that these parameters are .


In vitro testing of the fabricated prototype has been carried out inside a multi-layer porcine slab to verify the effectiveness of this unique SAR reduction technique.In this paper, a conformal absorber metasurface has been designed and used for reducing the specific absorption rate (SAR) of an implantable antenna. SAR reduction of implantable antennas is one of the significant design aspects to be considered for their use in modern-day healthcare applications. The introduction of the absorber metasurface restricts the back radiation of the antenna to control the SAR value. This technique decreases the maximum SAR value by 24% and also reduces the average SAR distribution significantly without affecting the desired antenna gain. A reduction in SAR value indicates the decrease in radiation absorption by human tissue, and thus, decreases the possibility of health hazards due to EM radiation. Later, this antenna-absorber system is designed as a capsule module for increased mobility and less-invasiveness. The redundancy of invasive surgery increases acceptance of the capsule module designs of implantable antennas and devices for various biomedical usages.  Biomedical implants revolutionize the approach of the modern-day healthcare industry by continuous and accurate monitoring of patients' conditions. The implantable antennas are largely sought for as transceiver units to communicate wirelessly with and between these implants. These antennas come with the perils of electromagnetic (EM) radiation hazards over the human body , as the Specific absorption rate (SAR) is the estimation of EM absorption by a unit mass of the human body. An excessive SAR value indicates greater amount of absorption by human tissue and hence more significant damage might be done to the human under consideration . VariousDue to the above limitations, some direct methods of SAR reduction have been analyzed over the years. Amongst the techniques, electromagnetic shielding is widely used for wearable antennas . RecentlSAR reduction of the implantable antenna is far more essential as the entire antenna is placed inside human tissue and internal human organs are in direct exposure to the antenna radiation. In a recent approach to reduce the SAR in implantable antenna, we had proposed a technique using a ferrite superstrate to reduce the discontinuities in near E-field and hence reduce the SAR . Though In another aspect, placement of the implantable antenna along with other implant electronics requires excessive planning and attention. Keeping in mind the complexity and delicate nature of the human structure, an implantable antenna must be flexible, compact, and bio-compatible. Capsule-shaped antennas, being the solution to all the design challenges mentioned above, are widely used for digestive and ingestible monitoring systems . This lein vitro conditions. Finally, we have incorporated the capsule-shaped design to make the antenna system more suitable for implantable applications as well as various ingestible practices. This capsule antenna design will be beneficial for batch production of the antenna and amalgamation of the antenna with other implantable medical devices (IMDs). The final antenna-absorber capsule module has been investigated inside a multilayer porcine slab.Absorber metasurfaces are engineered periodic structures, used for EM radiation absorption. A bianisotropic  metasurfMajor contributions to this work can be summarized below,A novel topology creates a breakthrough for SAR reduction of an implantable antenna without affecting other antenna performance.A new bianisotropic periodic structure has been designed for this topology to utilize the absorbing property as well as reduce the field scattering.The antenna-absorber system has been crafted as an ingestible capsule module to enhance mobility and reduce invasiveness.A = 1−T−R, where A is absorption, R is reflection, and T is transmission. In terms of S parameters, it can be deduced as, The metamaterial unit cell has been designed using a four Ω-shaped structures placed diagonally along with an X-shaped metallic strip . In geneurvature . Validaturvature .Flexibility is one of the essential requirements for IMDs and implantable antennas. The proposed absorber must have this virtue so that the entire low SAR antenna system can become conformal in design. The unit cell has been designed on PLA, which is a flexible bio-plastic. Though the designed absorber unit cell can be bent physically, the performance analysis has been done to validate whether the unit cell is flexible in nature without affecting the absorber properties. It is evident from the , that thA printed dipole antenna  is desig2O3) . Using this technique, the thickness of the alumina layer is controlled and a thin layer of coating is provided on the proposed capsule module. For further study, this conformal capsule module is placed inside a human torso model  . To deposo model  in Ansysso model  of the cSAR is a measure of EM radiation absorbed by a unit mass of human tissue. Mathematically SAR of an implantable antenna over a unit volume of human tissue can be expressed as,E is the electric field due to the antenna radiation and ρ is the density of the human tissue. So it can be seen that, for a fixed amount of EM radiation this absorption becomes the property of that tissue itself. Hence the existing techniques, those used for SAR reduction in wearable antennas, may cause a paradoxical situation in case of implantable antennas. Antennas, which are implanted inside the human body, radiate inside human tissue and make the tissue exposed to EM radiation. The use of EM shielding might reduce the SAR value by limiting the antenna radiation, it will certainly, reduce the antenna gain drastically. As, for implantable antennas, all the essential and spurious radiation necessarily goes through the human tissue. Hence, lowering SAR with the existing technique might reduce the antenna gain and efficiency extensively, so the antenna performance might be affected. To overcome this paradox, in previous literature  and 50Ω SMA connector probes. We have used a porcine slab, as its dielectric properties are similar to those of the human tissue equivalent, to establish an in vitro setup using VNA (in vitro analysis purposes. Measurement yields almost similar return loss characteristics compared to the simulated analysis (In the previous section, a printed dipole antenna was designed and analyzed. Also, an absorber metasurface has been proposed to reduce the SAR of that implantable antenna. Both the antenna and metasurface have been fabricated on a PLA substrate . The prosing VNA . These panalysis . The effanalysis .in vitro using a multi-layered porcine slab. This technique helps us to reduce the SAR of the implantable antenna without affecting the antenna gain much. Also other parameters, such as antenna efficiency, impedance bandwidth, etc. remain almost unaffected due to the inclusion of absorber metasurface. Hence, this technique contributes to curtailing EM radiation absorption significantly, which results in biologically safer usage of implantable antennas. Using this technique the SAR of the proposed antenna has been reduced from 453 to 339.7 W/Kg. However, this reduced value is beyond the permissible value of the SAR according to FCC standards, i.e., 1.6 W/Kg. To attain that a well-used practice is to reduce the input power of the antenna. However, with the technique proposed here, we can reduce SAR value so that more input power can be given to the antenna for efficient communication. In this work, we can provide a maximum input power of 5.5 mW instead of 4.8 mW due to the incorporation of this proposed SAR technique. While this input power is sufficient to establish an in-body to an on-body communication link, a tradeoff between the higher input power and lower SAR value will always remain a key factor for implantable antenna design. In future development based on this literature, the maximum input power may be increased for better in-body communication. In this paper, a unique technique has been proposed to reduce the SAR of the implantable antenna using absorber metasurface. The redundant back radiation of the printed dipole antenna has been absorbed by the metasurface to reduce the average SAR by 24%. The antenna-absorber system is designed as a compact capsule module to enhance the conformal nature of the system and also, make the system less-invasive. This capsule module can be attached to any implantable medical device as a transceiver unit. This ingestible capsule, made of PLA, is biocompatible in nature. The fabricated module has been tested The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.Ethical review and approval was not required for this study in accordance with the local legislation and institutional requirements.SD performed the design and experiments of the proposed antenna and wrote most of the article. AC designed and fabricated the capsule. DM and BM were involved in the critical analysis of the sensor's working. SD and BM were involved in the simulation software. RA has done the review, proofread, and provided antenna design techniques. RA and DM were involved in the supervision of the project, conception, design of the research, and approved the final version of the manuscript. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


In radar detection, in order to make the beam have variable directivity, a Capon beamformer is usually used. Although this traditional beamformer enjoys both high resolution and good interference suppression, it usually leads to high sidelobe and is sensitive to array steering vector (ASV) mismatch. To overcome such problems, this study devises a novel, robust adaptive beamformer that is robust to ASV mismatch under the constraint where the sidelobe is oriented to the ground. Moreover, to make full use of the transmit power, the constraint of a low peak-to-average power ratio (PAPR) is also taken into consideration. Accordingly, this robust adaptive beamformer is developed by optimizing a transmitting beamformer constrained by ASV mismatch and low PAPR. This optimization problem is transformed into a second-order cone programming (SOCP) problem which can be efficiently and exactly solved. The proposed transmit beamformer possesses not only adaptive interference rejection ability and robustness against ASV mismatch, but also direct sidelobe control and a low PAPR. Simulation results are presented to demonstrate the superiority of the proposed approach. The proposed method can make the peak sidelobe level (PSL) level on the ground side below −30 dB. In modern radar systems, the adaptive array is usually used to suppress interference . When coThe research on ADBF technology has become comprehensive in recent years. Practically, many factors that seriously deteriorate the target detection performance cannot be ignored, including wavefront distortion, incoherent local scattering, terminal pointing errors, and antenna array calibration errors . ConventThe subspace approach is another technology that works against ASV mismatch. Zhang estimated noise subspace and interference subspace via constructing an interference-plus-noise covariance matrix . When ASA new robust adaptive beamforming (RAB) method was proposed recently. It eliminates the influence of the desired signal by reconstructing the interference-plus-noise covariance (INC) matrix. In the literature , Gu propJoint radar and communications on a single platform can reduce the cost of the platform, share spectrums, and enhance performance via the cooperation of radar and communications ,17,18. J(1)The maintenance of the main lobe performance for radar beamforming;(2)The suppression of the sidelobe of radar beampattern;(3)Introduction of the PAPR constraint to improve radar detection performance.In this paper, a new transmitting beamformer is proposed. Considering the disadvantages of conventional beamformers, such as high sidelobe and sensitivity to ASV mismatch, the direction of the sidelobe facing the ground is limited, and it is robust under ASV mismatch. In addition, to make full use of the amplified power of the transmitter, better suppress the large-area interference, and irradiate the radar energy to the target area as much as possible, the constraint of a low peak-to-average power ratio (PAPR) is also considered. Simulation results show the superiority of this method. Generally, the major contributions of this paper include:The rest of this paper is organized as follows. In A uniform linear array (ULA) composed of nth array sensor. The transmit beampattern can be expressed asA plane wave with a It is well known that conventional adaptive beamformers, e.g., the minimum variance distortionless response (MVDR) beamformer , minimizPractically, a radar system that aims to detect low altitude, slow-speed, small targets should transmit all energy into the target area in an ideal condition so the receive module can obtain a stronger SNR and a better performance. However, if the radar transmitter is amplitude weighted, the radar emission energy is weakened. Considering that a radar transmitter amplifier usually works in a saturated state and cannot implement the amplitude modulation on the waveform, the emitted signal is often required to be unimodular or have a low PAPR. The low PAPR constraint is the relaxation of the unimodular constraint so that the radar transmitter can transmit as much energy as possible to the target area. With the above under consideration, we impose a low PAPR constraint on the TWV. According to the definition, the constraint on the PAPR of the TWV can be expressed asFurthermore, there may be mismatches in the array element position, channel amplitudes, and channel phases, which leads to ASV mismatch. The SMI beamformer in (11) is highly sensitive to ASV mismatch. In a mismatch case, the actual beampattern is distorted so that the spatial filtering capability is reduced, and other substantial performance degradations are caused.nth element of the array be Let the position of the The ideal weighting vector of the beamformer isAssume that The actual beam magnitude response isℓ2 norm. With constant mismatch variances, According to (16), with tiny mismatches in the above-mentioned cases, the expectation of the actual beam magnitude response isIn engineering applications, much more clutter and many more moving targets exist on the ground than in the air. Thus, the echoes from the ground are much higher than those from the air. In order to mitigate the echoes from the ground clutter and ground moving targets at low altitude, the power emitted to the ground should be reduced. In the transmit beampattern, the sidelobes facing the ground side should be suppressed to a lower threshold, while the sidelobes facing the air side can be raised, as there are few interferences on the air side. The constraint on the sidelobe levels on the ground side can be written asWhen the constraints on the PAPR, robustness, and sidelobe levels are considered, the optimal TWV cannot be obtained directly by (11). To this end, we establish the following optimization problem:Since med thatω2=ωHω=ωHN. The problem in (27) contains two linear matrix inequalities (LMIs) and one linear matrix equality (LME). Thus, in the interior-point method (IPM), the number of iterations required to reach the optimal solution is Computational complexities: The complexities of the proposed approach mainly depend on the number of antenna elements In this section, representative simulations are carried out to assess the effectiveness of the proposed, robust adaptive transmit beamforming scheme. As shown in To show the sidelobe quality, the indicators to measure the performance of sidelobe suppression are the peak sidelobe level (PSL) and the integrated sidelobe level (ISL). The PSL is the ratio of the maximum sidelobe power to the peak power. The ISL is the ratio of the total side lobe power to the peak power. The PSL and ISL are calculated as follows:The most basic DBF beamformer is the static beamformer. Its weighted value is the desired signal steering vector. Because it only has a phase shift, it is the simplest to implement and has a low sidelobe level. The static beamformer and SMI beamformer are realized asIn this study, firstly, the beampattern of the proposed method is compared with classic beamformers in order to analyze the beampattern promotion. To show more details of the promotion and the comparisons of different beamformers on the transmit weight vector, zero depths are given in In this subsection, the beampattern of the proposed method is compared with the SMI beamformer, the static beamformer, norm-constrained Capon beamformer (NCCB), and worst-case performance optimization (WCPO), denoted by In this example, the threshold for sidelobes on the ground side is still set to −30 dB, while the PAPR tolerance is set to 1.2, 1.4, 1.6, and 2.0. The sidelobe threshold To block interference suppression, e.g., ground, a robust adaptive transmit beamforming under the constraint of a low PAPR is proposed in this paper. In this approach, we allow the sidelobe to face the ground under the threshold so that strong ground jamming is suppressed. The constraint on the PAPR of the TWV is used to avoid excessive energy attenuation. We then give the final form of this optimization model. By transforming the optimization model into an SOCP problem, it can be solved easily using the CVX toolbox. Simulation results show that the proposed method well suppresses the interference on the ground side and air side. The sidelobe level of the ground side is low, and the excess signal energy is transmitted to the air side. By employing the proposed method, the detection performance of the radar can be effectively improved.


Autonomous vehicle perception systems typically rely on single-wavelength lidar sensors to obtain three-dimensional information about the road environment. In contrast to cameras, lidars are unaffected by challenging illumination conditions, such as low light during night-time and various bidirectional effects changing the return reflectance. However, as many commercial lidars operate on a monochromatic basis, the ability to distinguish objects based on material spectral properties is limited. In this work, we describe the prototype hardware for a hyperspectral single photon lidar and demonstrate the feasibility of its use in an autonomous-driving-related object classification task. We also introduce a simple statistical model for estimating the reflectance measurement accuracy of single photon sensitive lidar devices. The single photon receiver frame was used to receive 30 12.3 nm spectral channels in the spectral band 1200–1570 nm, with a maximum channel-wise intensity of 32 photons. A varying number of frames were used to accumulate the signal photon count. Multiple objects covering 10 different categories of road environment, such as car, dry asphalt, gravel road, snowy asphalt, wet asphalt, wall, granite, grass, moss, and spruce tree, were included in the experiments. We test the influence of the number of spectral channels and the number of frames on the classification accuracy with random forest classifier and find that the spectral information increases the classification accuracy in the high-photon flux regime from 50% to 94% with 2 channels and 30 channels, respectively. In the low-photon flux regime, the classification accuracy increases from 30% to 38% with 2 channels and 6 channels, respectively. Additionally, we visualize the data with the t-SNE algorithm and show that the photon shot noise in the single photon sensitive hyperspectral data contributes the most to the separability of material specific spectral signatures. The results of this study provide support for the use of hyperspectral single photon lidar data on more advanced object detection and classification methods, and motivates the development of advanced single photon sensitive hyperspectral lidar devices for use in autonomous vehicles and in robotics. Autonomous driving is expected to be one societally significant reform coming to modern society during the next 20 years. In some estimates, up to 15% of new cars will be completely autonomous in some US city areas in early 2030. Based on US studies, the further impacts of autonomous driving include improvement of fuel economy , and decAutonomous vehicles often include a large number of sensors mounted on-board due to the need to observe the environment all around the vehicle and also to position the vehicle. The perception sensors include ranging sensors and vision-based sensors. Ranging sensors, such as lidar, radar, and sonar, provide 3D measurements of the surroundings . They arEven though multi-sensor approaches are used for road environment target detection, classification, and tracking, there are several shortcomings with the current autonomous sensors: (1) in some use cases it is necessary to see 200–300 m ahead and be able to detect the objects, (2) current systems provide non-optimal accuracy of main road-user classification, (3) energy usage of the future cars using current state-of-art sensors is too high, and (4) sensor technology should work in all weather and climate conditions. In order to solve these problems, the sensor technology is leading to an integrated lidar and camera solution. Multispectral or hyperspectral, single-photon, possibly frame-based solid-state, lidar is one potential candidate technology for the future.In a paper , the conIn this paper, we describe the prototype hardware for hyperspectral single photon solid-state lidar, introduce a statistical model for estimating the spectral reflectance measurement accuracy in a low-photon flux regime  laser scanners. In contrast to laser scanning systems, no mechanical moving parts are needed. Flash lidar is also capable of composing a 3D image of a scene in just one shot. As the cameras reach up to several hundreds frames per second, they are ideally suited to be used in real-time applications. Other flash lidar advantages include lightweight, blur-free images without motion distortion, no need for precision scanning mechanisms, thus, no moving parts are needed, and ability to discriminate distributed targets through range-gating. Ref.  present Various measurement models have been proposed for extracting the return pulse signal or reconstructing the 3D scene from noisy, low-photon flux, single-photon sensitive measurements, for example, in a multiple target scenario , and wheMultispectral lidars typically implement return pulse separation into wavelength channels by passing the incident photons into a spectrograph in the receiver side ,40,45. TRecently, large form factor SPAD arrays have become more common. A 512 × 512 pixel SPAD array with range gating possibility was introduced in . In 50]50], a SPThe applied detector was a 32 × 32 SPAD array, Kestrel (Princeton Lightwave). Each element has a TDC  with timing resolution of 250–1250 ps and measures the time between the initialization of the sensor and the first detected photon hitting the element. The sensor is initialised by a trigger pulse from the supercontinuum laser source (Leukos Samba 400) when a laser pulse is generated. After a set time period of 2–10 The laser pulse was collimated by a refractive collimator and transmitted to a target using an adjustment mirror for beam alignment . The scaThe other end of the fibre was connected to detection optics, where the light was collimated with a parabolic mirror to a diffraction grating with 150 lines/mm . An achrBeam scanning can be realized on the prototype hyperspectral single photon lidar by employing standard beam scanning mechanisms, such as rotating multi-faceted, Palmer scanning, or oscillating mirrors. Additionally, the optical head assembly can be designed to be operated on a rotating assembly.A fundamental statistical property of light incident on a detector element is that the arrival times of individual photons follow the Poisson distribution. This property can be derived both from the semi-classical and the quantum mechanical model of coherent light . The desIn practice, the Poisson distributed photon counting statistics hold for SPAD array sensors if the average photon flux incident on a single pixel is less than one detection event per exposure period ,53. For b, scale parameter 1, and shape parameter l. The variable d.We begin our derivation by considering a Poisson-distributed random variable with a rate parameter puted as :(1)Q:(2)rTo derive an expression for the confidence limits of channel-wise reflectance values, given a certain observed signal level interval −α·100%, Equation . Then, tA more useable form of the above expression can be obtained by computing the ratio of the upper and lower confidence limits to the reflectance estimate (shifted relative error). In the case of overabundant photon counts, the observed reflectance value is overestimated and the upper confidence limit for the ratio is given by:Similarly, when there is a deficit in the photon counts compared to the expected photon flux, the observed reflectance value is underestimated and the lower confidence limit for the ratio is given by:It is a well-known property of single-photon sensitive imaging schemes that the sample mean of the photon count is close, or equivalent, to the Cramér–Rao lower bound . It can The objectives of this research are to examine the feasibility of hyperspectral single photon lidar data for autonomous driving purposes and to investigate the limitations of spectral reflectance measurement accuracy that might exist in the low-photon flux regime. Specifically, we compare the statistical model for spectral reflectance measurement accuracy in the low-photon flux regime . We use 30 different spectral channels , each with a bandwidth of 12.3 nm, from the wavelength band 1200–1570 nm. The dataset classes were selected with the criteria that they should represent some of the most common objects and materials found in the driving environment. The classes include dry asphalt, gravel road, granite, moss, white plaster wall, snow covered asphalt, car body , spruce, wet asphalt, and grass. With this dataset we tried to simultaneously explore the properties of the low-photon count regime and the potential that is available with the hyperspectral sensing capability as compared to the commonly available monochromatic lidar technology. To our knowledge, the combination of these two dimensions of measurement are discussed to a smaller extent in previous literature, and are hardly experimented with together at all. To date, several valuable studies ,33,58 haEach sample in the dataset is a set of 10,000 consecutive frames that have been acquired by firing the supercontinuum laser towards the same spatial spot (beam steering was not used) at a pulse repetition rate of approximately 30 kHz . The spatial location of the target spot, and the incidence angle between the target surface and the laser beam, were widely varied between the sample measurements. The targets were located at distances ranging from 15 to 100 m. The total exposure time was in the order of one ® diffuse reflectance standard plates were used with reflectance values of 20% and 40%, respectively. The targets were optically flat over the measurement wavelength band with a relative error of In order to perform white balance calibration to the system and to examine the photon counting statistics at the low-photon flux regime, we collected four samples of Spectralon targets in a low ambient irradiance environment. Two SpectralonThe distance to the Spectralon targets was approximately 16 m in the first set of measurements and approximately 11 m in the second set of measurements. Therefore, in addition to the system dependent white balance calibration values, also the factor originating from the optical properties of the air mass between the lidar and the target plates has been included in the white balance calibration measurements.In order to explore the magnitude of dark current noise in our measurement data, we collected a set of 100,000 consecutive frames in such a way that any external illumination was prevented from entering the SPAD sensor array . The magnitude of the dark current noise was approximately The theoretical model for the spectral reflectance measurement accuracy in the low-photon flux regime   algorithi’th sample in class c. The relative reflectance spectra was used instead of the regular reflectance spectra in order to concentrate the analysis on the spectral shape dependent properties of the data without adding an external error source due to calibration related non-idealities. The data were normalized before feeding into the t-SNE algorithm by mean-centering and rescaling the spectral channels to unit variance:n denotes the row index of As an input data We computed the t-SNE embedding for three different block sizes  of each class in the autonomous-driving-related dataset see  as an inDue to the small size of the dataset, 5-fold cross-validation was applied. At each fold, the training set size was 240 samples (24 samples per class) and the test set size 60 samples (6 samples per class). Two hyperparameters were varied between the tests: the block size (the photon count was accumulated over a block of multiple consecutive frames) was altered in the range The random forest classifier was trained from the ground up for each spectral resolution value (number of channels), and for each cross-validation fold. In the training phase, the training samples consisted of the white balance normalized relative reflectance spectra that had been accumulated over the full range of 10,000 consecutive frames per sample. This was completed in order to ensure that the classifier would learn, for each class, an estimate of the spectral signature that would be as close as possible to the underlying noise-free spectral signature.k and with the test set sample index of i, respectively. The experiments were carried out by using the random forest classifier implementation from the scikit-learn library . The numThe accuracy of the trained random forest classifiers is compared to a baseline accuracy of 10% which can be obtained by predicting the sample class by randomly selecting it from a discrete uniform distribution over the class labels.The following sections describe the process of estimating the target distance, computing the relative spectral reflectance curve from a single frame, or from a sequence of measurements, and reducing the spectral resolution by channel binning. The data processing pipeline has been visualized in i and j denote the pixel indices in the intensity and wavelength directions, respectively, and k denotes the location of the frame in a sequence of measurements. The pixels are triggered either by photons originating from the target return pulse or ambient illumination source, or alternatively by SPAD array intrinsic noise, such as dark current noise [Each measurement cycle with our hyperspectral single photon lidar produces a data frame nt noise , afterpunt noise , or crosnt noise ,67,68,69x denotes the timer histogram values, A is the signal intensity dependent amplitude scaling factor. The constant term The target distance was estimated by a following process: first, a timer histogram was computed from the SPAD array pixel counter values Equation ) was fitOnce the estimates for the distance uated as:Sλx=∑i=13When operating in a low-photon flux environment, it is important to capture most of the signal photons due to the inherently low signal-to-noise ratios. Therefore, the timer value filter window spersion ,70,71 anFollowing the computation of the channel-wise photon count In order to obtain an estimate of the shape of the spectral reflectance curve Then, the normalized photon count vector Equation ) in ordeThe spectral reflectance curve obtained in this way is relative, in the sense that the scale does not represent the true target reflectance value, but the shape of the curve resembles the shape of the absolute spectral reflectance curve In expression , the deni and k represent the location of the frame in the measurement sequence, and The maximum signal capacity for a single frame measurement on our hyperspectral single photon lidar has been limited by the SPAD array resolution to 32 photons per channel. The limitation was addressed in the experiments by artificially increasing the signal photon count by employing block-wise signal accumulation scheme where the photon count from a block of two or more consecutive frames was combined together:In the block-wise signal accumulation scheme, the expected photon count scales linearly with respect to the number of frames:This allows the signal-to-noise ratio (SNR) in the block-wise signal accumulation scheme to be expressed as:k refers to the left-edge of the binned channel in the full spectral resolution coordinates. The index k was selected, such that the channel binning operation was applied only to non-overlapping channels in the original spectrum Channel binning was applied in the classification experiment in order to capture the effect of the spectral resolution in the classification accuracy. The channel binning operation was carried out by summing the photon counts at two or more adjacent channels together:The effect of channel binning on the spectral resolution has been illustrated in The autonomous-driving-related dataset has been visualized in The visualization in The relative spectral reflectance curves of the Spectralon measurements have been visualized in In order to investigate the convergence of the Spectralon spectra towards the ideal white balance spectrum, the root-mean-square error (RMSE) was calculated between the Spectralon spectra and the ideal white balance spectrum with respect to the block size. The resulting error graph can be observed in The average block-wise photon count of the Spectralon 40% sample, in addition to the sample standard deviation of the block-wise photon count, has been visualized in Additionally, the water vapour absorption peak in the air mass can be observed in The theoretical relative confidence limits The visualization reveals that the sample observations for the lower confidence limit seem to be in agreement between the theoretical values. However, the theoretical upper confidence limit has been estimated slightly more conservatively when compared to the confidence limit given by the sample observations. A similar tendency to overestimate the theoretical upper confidence limit when compared to the sample observations was also observed in The visualization in The results of the dataset separability experiment have been illustrated in The purpose of the classification experiment was to demonstrate the use of the hyperspectral single photon lidar data in an autonomous-driving-related perception task, and, also, to establish the feasibility of the data for classification purposes in the low-photon flux regime. The results of the experiment have been visualized in The mean classification accuracy in the test set reflects the theory in multiple ways: First, the rate of improvement in the classification accuracy is the highest when the block size is relatively small, while the rate of improvement stagnates when we move towards larger block sizes. The behaviour reflects closely the asymptotically convergent shape of the relative reflectance confidence limits. In the low-photon flux regime the confidence limits substantially improve as the signal photon count is increased, and then, at the high-photon flux regime, shrink towards better reflectance estimation certainty in much smaller increments. Second, at the high-photon flux regime  only when the block size is extended above 10 frames, which translates into signal photon counts in the order of One of the initial objectives in this study was to find theoretical limits to the spectral reflectance measurement accuracy in the low-photon flux regime. The simple theoretical model that was derived based on the quantile function of the gamma distribution fits the data relatively well, although the higher confidence limit was overly conservative when compared to the sample observations with small number of photon counts. The difference of the derived model to the measurement results might be explained by the pile-up effect , which dThe pile-up effect exists only for high-intensity return pulses, which explains why the lower confidence limits are in agreement between the theory and the sample observations. Further, the expected frame-wise photon count For the purpose of computing the spectral reflectance curve as accurately as possible from the data, it is recommended to use a proper photon counting statistics model ,73 that The results of the dataset separability experiment indicate that the main factor contributing to the separability of the hyperspectral single photon data is, most probably, the spectral reflectance measurement accuracy, which, in turn, is highly dependent on the return pulse photon count. In many cases, the spectral signature of important targets differs substantially across the target classes . It can,It was observed in the experiments that a few of the dataset classes were intermixed in the t-SNE embedding even in a scenario where the spectral reflectance measurement accuracy was adequately high. This is quite natural when the underlying material properties in the measurement setup are considered more closely. For example, at the time of the dataset collection the grass field was slightly moist, which, to a certain degree, explains the similarity of the spectral features of the “grass” class when compared to the “wet asphalt” class. Likewise, the dry asphalt surface that was used as one of the dataset targets was deteriorated due to age, which made the gravel particles in the surface course stick out. The gravel particles provided a reflective surface area for most of the illumination pulse photons instead of the asphalt matrix material, which, in turn, made the spectral signature of the “dry asphalt” class resemble the “gravel road” class.Further study is required to investigate the degree to which the noise properties of single point measurements affect high-capacity machine learning methods, such as deep learning networks, when the input data are a point cloud instead of a set of individual points. It might be possible that the internal representations formed by the deep learning networks learn to perform, in a sense, signal accumulation over finite spatial regions with object borders declared by the geometry of the object. The internal representations could in principle, with high enough spatial point density, achieve a similar noise reducing effect that was achieved with the block-wise measurement scheme, but without requiring the measurement cycle to be run multiple times for a single spatial location.The impact of the spectral reflectance measurement accuracy on the dataset separability could also be observed in the results of the classification experiment. Although the dataset was small, the strongly increasing trend in the classification accuracy with respect to the block size (correlates with the channel-wise photon count) was still clearly visible. Additionally, it is interesting to note that the results of the classification experiment support the idea that high spectral resolution is preferable to low spectral resolution, even when the reduced spectral resolution would account to higher spectral reflectance measurement accuracy at the binned channels, at least at the wavelength band used by our lidar. Channel binning is therefore not advisable without optimizing the individual channel bin widths in such a way that the spectral signature separability is maximized simultaneously to minimizing the channel-wise photon shot noise. This process is naturally heavily dependent on the application, and is therefore an important subject for further study.The findings in this study, while preliminary, may help us to better understand the challenges and strengths associated with next-generation automotive lidar technology. In earlier research it has been noticed that many autonomous-driving-related perception methods benefit from the use of lidar reflectance or intensity channel information in conjunction to the 3D point cloud coordinate information, even when the lidar operates on a monochromatic basis ,76,77,78The advantages of the hyperspectral single photon lidar architecture introduced in this study include the low-complexity of the data processing phase and the ability to simultaneously measure the spectral signature over the whole wavelength band from a single spatial location with only a single illumination pulse. In this regard, the lidar architecture, when combined to a beam steering unit, would be suitable for use in autonomous vehicles, where achieving minimal latency from the target detection to the control system output is often desired. There are, however, some disadvantages that might reduce the practical operating envelope of the sensor, such as the loss of SPAD array capacity in long-range measurements due to the accumulation of dark noise- or ambient photon triggered pixels, and the limited dynamic range with current, fairly low resolution, pixel-wise TDC-based SPAD arrays.The issue of reduced SPAD array capacity at long-range measurements could be addressed either by delaying the initialization of the sensors TDC counters after a trigger pulse has been detected, or by operating the lidar in range-gated mode ,82,83. Cambient channel for each point cloud point. With spectrograph based hyperspectral single photon lidar, the lidar device could also be used as a hyperspectral camera. This would be beneficial in situations, where, for example, due to high solar irradiance, the normal operating conditions of the lidar device would have been deteriorated.The high-sensitivity of single photon detectors provides the possibility to combine functionalities of camera and lidar devices in the same sensor. This has been already implemented in multiple Ouster lidar models , which pIn this feasibility study, a frame-based single photon sensitive hyperspectral lidar has been developed and introduced in the context of autonomous vehicle perception. The lidar architecture allows the simultaneous measurement of geometric and spectral information from each point with a single supercontinuum laser illumination pulse. The measurement approach might be less prone to motion blur, require less illumination power on average, and also be computationally less demanding than comparable actively illuminated spectral measurement methods.We introduce a statistical model for estimating the accuracy of the spectral curve in the low-photon flux regime, and demonstrate how the model can be used to calculate confidence bounds of the relative spectral reflectance curves of single photon sensitive spectral reflectance measurements. Further, we explore the separability of the single photon hyperspectral lidar data as a function of the photon count with a focus on increasing the robustness of autonomous vehicle perception systems. Finally, we test the feasibility of the data for classification purposes in an autonomous-driving-related classification task.The results show that the statistical spectral reflectance accuracy model conforms closely to the observations, disregarding the SPAD sensor non-idealities. The model indicates that in order to obtain adequately high spectral reflectance measurement accuracy, with relative error of less than de lidar .The separation of the data into class-specific clusters in the 2D t-SNE embedding was observed to be highly dependent on the amount of photon shot noise in the spectral curves. A few of the dataset classes were already separated into class specific clusters when the signal levels were in the order of 10 photons per wavelength channel, but the unambiguous delineation of most of the class clusters was achieved only after the signal level reached approximately The results of the classification experiments imply that the measurement data provide useful information about the material specific spectral signatures, even with relatively low photon counts of less than 10 photons per wavelength channel. For a single frame measurement (maximum intensity of 32 photons), the mean classification accuracy in our dataset with a random forest classifier was approximately 35% with 30 wavelength channels, which is a substantial improvement over the baseline model with an accuracy of 10%. However, achieving mean classification accuracy of over 90% with the full spectral resolution requires the detection of multiple hundreds of photons per wavelength channel, which can be accomplished by using high laser repetition rate and block-wise measurement scheme, or by increasing the SPAD array resolution.Additionally, it was found that increasing the spectral resolution improves the classification accuracy, not only in the high-photon count regime, but also when the photon count is low. Therefore, we believe that hyper- and multispectral lidar devices, single photon sensitive or not, can increase the robustness and accuracy of many lidar based autonomous driving and robotics perception methods in the future.


Performing the machining of complex surfaces can be a challenging task for a robot, especially in terms of collaborative robotics, where the available motion capabilities are greatly reduced in comparison with conventional industrial robot arms. It is necessary to evaluate these capabilities prior to task execution, for which we need efficient algorithms, especially in the case of flexible robot applications. To provide accurate and physically consistent information about the maximum kinematic capabilities while considering the requirements of the task, an approach called the Decomposed Twist Feasibility (DTF) method is proposed in this study. The evaluation of the maximum feasible end-effector velocity is based on the idea of decomposition into the linear and angular motion capabilities, considering a typical robot machining task with synchronous linear and angular motion. The proposed DTF method is presented by the well-known manipulability polytope concept. Unlike the existing methods that estimate the kinematic performance capabilities in arbitrarily weighted twist space, or separately in the translation and the rotation subspace, our approach offers an accurate and simple solution for the determination of the total kinematic performance capabilities, which is often highly required, especially in the case of robot machining tasks. The numerical results obtained in this study show the effectiveness of the proposed approach. Moreover, the proposed DTF method could represent suitable kinematic performance criteria for the optimal placement of predefined tasks within the robot workspace. Collaborative robotic technologies enable humans and robots to work close together in a shared manufacturing environment. Contrary to classical industrial robots, which are primarily programmed to perform deterministic repetitive tasks, a new paradigm of collaborative robots encourages the development of flexible and intelligent robotic systems, especially for the needs of small and medium enterprises with low-volume, high-mix production ,2. In orA robot’s workspace is usually defined as the maximum reachable volume whereby the end-effector is capable of reaching each point in at least one orientation. The reachable workspace characteristics can be computed offline and visualized as a discretized map, which is called a reachability or a capability map ,10. The A detailed understanding of robot movement capability within its workspace is crucial for the successful planning and execution of more complex surface machining tasks that require the simultaneous tracking of the desired workpiece surface all the time along the path  ,12,13,14If only the kinematic of the robot is considered, a qualitative measure of the ability to move the end-effector toward an arbitrary direction is most often based on analyzing the manipulability ellipsoid that was first defined by Yoshikawa . YoshikaA suitable performance metric to determine the maximum directional kinematic capabilities of the robot end-effector was proposed in , that beBoth methods may result in a combination of translational and angular motion capabilities in a single scalar performance measure, which makes them physically inconsistent . SeveralThe more objective approach seems to be the partition on the translational and rotational motion capabilities. Through the decomposition of the task space into a translational and rotational subspace, the translational and rotational manipulability polytope in a weak and strong sense was derived in , similarFor this reason, we propose a method called Decomposed Twist Feasibility (DTF) in order to determine a feasible linear and angular velocity for robot machining purposes. In contrast to the existing methods that estimate maximum directional capabilities, it can precisely determine maximum motion capabilities for translation and rotation while considering the exact joint velocity limits and specific task requirements. Physical consistency can be achieved since the translational and rotational performance capabilities are considered individually. In contrast to the polytope approach, our solution can be computed in a time-efficient manner. The test results based on a generic 3 DOF planar robot manipulator and a collaborative robot UR5e demonstrate the usefulness of the proposed DTF method. The remainder of this article is organized as follows. The related research based on the manipulability ellipsoid and polytope is described in n degree-of-freedom serial robotic manipulator, which we consider to be operating in an m-dimensional Euclidian operational space, describes the relation between the end-effector velocities t, called twist, and joint velocities i = 1…m, j = 1…n, and The forward velocity kinematics of an The most widely used kinematic performance measure, which is based on the Jacobian matrix, was introduced in . The velNote that n matrix . FurtherA robot’s movement capability in any other direction can be geometrically described as the distance from the center of the ellipsoid to the point where the line along the direction of interest intersects the ellipsoid surface. Let The scalar  by Chiu :(6)μ2=u^The velocity transmission ratio (6), also known as the manipulator velocity ratio  or direcThe value of this index defines how effectively the robot’s end-effector can move in the direction he twist , which ihe twist .i-th joint if we assume that the maximal joint velocity is limited by the unit value of 1. The bounds on the individual joint velocity form a convex joint velocity polytope n-dimensional joint space, which can be written in the following compact form:The main disadvantage of the manipulability ellipsoid is the fact that it does not transform the exact joint velocity constraints into the task space ; howeverpolytope :(10)PtwiSince we are interested in quantifying the ability to change the position and orientation of the end-effector along a specified task direction, the maximum feasible magnitude of the end-effector velocity along  follows :(11)μ∞=(ity norm :(12)μ∞=tSimilar to the manipulability ellipsoid concept, the manipulability polytope concept also suffers from physical inconsistency due to the unwanted mixing of the linear and angular velocity in a single scalar value. Furthermore, it may be subjected to an algorithm seeking a computationally complex and less tractable minimum infinity norm solution ,45,46.Due to the physical inconsistency of the twist space manipulability ellipsoids and polytopes, the capabilities of translational and angular velocity should be handled individually. For this purpose, Yoshikawa  decomposSimilarly to the Yoshikawa approach, in the case of a manipulability ellipsoid, the translational and rotational velocity capabilities were also handled separately for the polytope approach . Since tn) submatrices:The translational  velocity polytope Note that mapping In n-dimensional polytope is mapped to a space of lower dimension (k < n), some of the vertices of the original polytope are mapped into an internal region of the k-dimensional polytope [When an polytope . For thi methods ,45,48 asOne may select one of the subspace manipulability polytopes, depending on the task at hand. If the main concern of the task is maximal linear velocity, then the vector length of the translational velocity polytope will be considered, and vice versa. Since translational subspace is considered separately from rotational subspace, multiple solutions may exist in the joint space, which satisfies constraint (8). The minimum infinity norm solution, illustrated by Besides polytope analysis, performance indices based on the vector expansion method  offer a The value The polytope The polytope In terms of trajectory planning optimization, an analysis of the translational manipulability polytope in the e faster  or with If we add another constraint that requires joint velocities to be projected onto the null space of The translational manipulability polytope The translational  manipulability polytope in the strong sense n degree-of-freedom, non-redundant, robotic serial-link manipulator in a non-singular configuration.The estimation of the maximum feasible linear and angular velocity for simultaneous linear and angular motion using the proposed DTF method will be explained in this section. The problem can be formulated as follows: We seek a maximum linear (angular) velocity in the desired direction, in a rotational  motion synchronized with the specified direction, under joint velocity constraints (8). In the following, we assume an n translational Jacobian submatrix:Unlike the majority of existing methods that take the translational  Jacobian submatrix ojection ,38. Let  and J˜R ,36:(23)Jh is a task-dependent velocity ratio factor that reflects the synchronization of translational and rotational motion.In contrast to the existing methods, where the desired motion synchronization cannot be controlled , the proposed method also takes the desired motion synchronization into consideration. The ratio of these magnitudes denoted by The linear velocity vector and the angular velocity vector can be represented by:The maximum linear and angular velocity can be found by scaling under the constraints (8), considering (26). This yields the following equations:To provide an intuitive understanding of the proposed DTF method, the maximum linear and angular velocity vectors  Concave path segment with h = 0.21, (ii) Convex path segment with h = 0.42, (iii) Flat path segment with h = 1000, (iv) Flat convex path segment with h=1.25. Each of the polytope iso-curves is related to the evaluation of the twist vector tips with the same task requirements at the specific value of the synchronization parameter h . It can be noted that each iso-curve lies entirely on the surface of the manipulability polytope, thus satisfying the constraint given by (8). If the reverse direction of angular velocity is considered, the set of resultant tips are mirrored to the opposite side of the twist manipulability polytope.In the middle of The maximum linear and angular velocity vectors of each example (i)–(iv) is additionally shown in detail by two different projection views, i.e., the The example (i) represents the case where the curvature of the path is high, and the change in the end-effector orientation is much bigger than the change in position . The stwistsee b. Since tope see b since tAn example of the path segment with a convex function shape (tope see c in thisered see d. The maIn the example (iii), almost only translational motion is considered. The magnitudes of the linear velocity vectors are equal to the vector lengths of the translational manipulability polytope in the strong sense ongT see e. The retion see f. Althouh and the direction of angular velocity In contrast to (iii), example (iv) represents a more flattened path segment. The magnitudes of the feasible linear velocity vectors are larger than the magnitudes of the angular velocity vectors. Especially in h, which are introduced as subregions within the Note that To demonstrate the effectiveness and potential usefulness of the proposed DTF method, we present two case studies in which the maximal end-effector capabilities were evaluated, i.e., for a generic 3DOF planar robot manipulator and for the 6DOF collaborative robot UR5e. To ensure the physical consistency of such evaluation in the case of a robot machining task, where the simultaneous control of synchronous translational and rotational motion is required, the maximal linear and angular velocity of the end-effector will be given individually for a predefined path segment. We will show that the maximal end-effector capabilities do not only depend on the configuration and direction of the desired linear and angular motion, but they are also strongly dependent on the specific task requirements. Additionally, in the case of the collaborative robot UR5e, it will be shown that this criterion could be useful in the case of a task-oriented workspace analysis and an optimal task placement.We consider the robot machining trajectory along a complex curved surface that is divided into The proposed DTF method was firstly applied to a generic 3 DOF planar manipulator, which enables an easier graphical representation of the manipulability polytope in a twist space. The 2D curve shown in h, converged to 0. For the flat path segment (iii), the synchronization parameter h value was the highest since the linear motion dominated over the angular. The values for the considered TCP position, the direction of the linear velocity vector h of each path segment are summarized in At each of the path segments under consideration, the direction of linear and angular velocity was determined along the path. Since simultaneous synchronous linear and angular motion was required along the path, the relative importance of each motion type plays a crucial role in the accurate determination of the maximal end-effector capabilities. In the case of following the curved path, the ratio between the translations and the rotations depends on its curvature. During the motion along the strongly curved path segment, as in case (ii), a significant change in the end-effector orientation was required, while a relatively small change in position was performed; therefore, the ratio between the magnitudes of the linear and angular velocities, denoted by the synchronization parameter h.Based on the proposed DTF method, the maximum feasible directional kinematic capabilities (The highest maximum feasible linear velocity was achieved in the case of segment (i), where the required change in position was approximately the same in relation to the change in the orientation (For each of the considered examples (i), (ii), and (iii), the twist polytope h, the maximal linear velocity vector It can easily be verified that the translational manipulability polytope his case c,f,j. FoThe ane view b,c,e shoThe proposed DTF method was also demonstrated and verified on the collaborative robot UR5e, which is of 6DOF, thus enabling robot operation in a 3D space. In this case, the 3D curve on the workpiece surface was determined as the path that the robot tool had to follow. A set of waypoints was placed along the path, with the orientation normal to the surface. The robot end-effector was to simultaneously reach each of them with the desired position and orientation. In this experiment, the proposed DTF method will be used to determine the maximum feasible linear velocity at three different parts of the curved path, as shown in h of each segment are shown in At each of the segments under consideration, the directions of the linear and angular velocities were determined along the path. During the motion along path segment (ii), a significant change in the end-effector orientation was required, while a relatively small change in position was performed; therefore, the ratio between the magnitudes of the linear and angular velocities, denoted by The maximum linear velocity For each of the considered examples (i), (ii), and (iii), the 3D translational manipulability polytope If the main concern was the linear velocity , (ii), and (iii)  of the base robot frame {B}, as shown in iii) see , which rAt each point, the inverse kinematic solution was firstly computed for the considered robot configuration. Only those points for which the inverse kinematic solution exists were included in further workspace analysis; the others were disregarded. The maximum linear velocity ment see  are denoAs shown in iii) see , the besThe feasible directional kinematic capabilities for the best placement of each path segment are presented in By the optimal placement of the workpiece with the considered machining path, a great improvement in the linear velocity amplitude can be achieved, while keeping the same desired base task parameters  method has been proposed in this paper. The basic idea behind the proposed DTF method is to find the maximal feasible directional kinematic capabilities in an operational space, individually for translational and rotational performances, under the robot joint velocity constraints. The method was algebraically derived and described by the manipulability polytopes as well. In contrast to the conventional manipulability methods, the proposed DTF method can provide accurate and dimensionally homogeneous information on how fast the robot end-effector can move under the constraint of joint velocities while maintaining the base task requirements. It can give us reliable information about the maximum linear and angular velocities when synchronized translational/rotational motion is required. A graphical representation of the proposed DTF method was presented by the twist manipulability polytope in the case of a generic 3DOF planar mechanism. Although we gave individual consideration to the translational and rotational capabilities in the proposed DTF method, the presented results show that the tip of the resulting maximum feasible twist velocity vector obtained by (30)–(33) always ends on the surface of the well-known twist manipulability polytope at the given joint velocity constraint (8), which verifies the method. In comparison to the conventional analysis of the twist manipulability polytope—which requires a computation of its vertices, the construction of the corresponding convex hull, and a further computation analysis of the convex hull in order to determine the intersection point with the polytope surface in the desired direction—the computational complexity can be reduced significantly by the DTF method, and the accuracy of the resulting information is maintained as well. The usefulness was demonstrated, especially in the case of the robot machining task purpose on a generic 3DOF planar mechanism and the collaborative robot UR5e. The maximum feasible linear velocity, considering the desired rotational motion, was determined in order to follow a specific path segment on the workpiece surface. The regions with the highest and lowest maximum feasible linear velocities were determined and then illustrated by the colormap, which forms the so-called task-oriented kinematic capability map.In conclusion it should be noted that the proposed DTF method addresses a purely kinematic performance measure. To provide feasible and optimal workpiece placement, the dynamic performances regarding the given task parameters should also be considered. In order to find an optimal workpiece placement for the entire tool path, the maximum linear velocity should be calculated for each path segment, or at least for the most critical ones. Thus, the proposed DTF method could be used as a suitable optimization criterion for optimal task-oriented workpiece placement within the robot’s workspace.