Under such conditions, AIC preferred the complex mixture models and BIC preferred the simpler mixture models. The mixture models selected by AIC had a better performance in estimating the edge length, while the simpler models selected by BIC performed better in estimating the base frequencies and substitution rate parameters. In contrast, AIC and BIC both prefer simpler partition models over more complex partition models under nonstandard conditions, despite the fact that the more complex partition model was the generating model. We also investigated how mispartitioning  affects both the performance of partition models compared with mixture models and the model selection process. We found that as the level of mispartitioning increases, the bias of AIC in estimating the expected Kullback–Leibler divergence remains the same, and the branch lengths and evolutionary parameters estimated by partition models become less accurate. We recommend that researchers are cautious when using AIC and BIC to select among partition and mixture models; other alternatives, such as cross-validation and bootstrapping, should be explored, but may suffer similar limitations .In molecular phylogenetics, partition models and mixture models provide different approaches to accommodating heterogeneity in genomic sequencing data. Both types of models generally give a superior fit to data than models that assume the process of sequence evolution is homogeneous across sites and lineages. The Akaike Information Criterion (AIC), an estimator of Kullback–Leibler divergence, and the Bayesian Information Criterion (BIC) are popular tools to select models in phylogenetics. Recent work suggests that AIC should not be used for comparing mixture and partition models. In this work, we clarify that this difficulty is not fully explained by AIC misestimating the Kullback–Leibler divergence. We also investigate the performance of the AIC and BIC at comparing amongst mixture models and amongst partition models. We find that under  The ongoing development of high-throughput DNA sequencing techniques provides us with large amounts of data for molecular phylogenetic inference. Heterogeneity among homologous sequences in these data may be due to lineage- and site-specific differences in the evolutionary process. This means that simple homogeneous models, which assume equal rates for all sites and lineages, may be inadequate and lead to incorrect phylogenetic inference .Two types of models are commonly used to incorporate heterogeneity in the evolutionary process when analyzing sequencing data: partition models and mixture models. A partition model divides the alignment into subsets of sites (blocks). All sites from the same block are assumed to have evolved under the same evolutionary process, and a different evolutionary model, including a substitution model and a tree, is fit to each block . DependiPartitionFinder and PartitionFinder2 implement heuristic methods to construct preferred partitioning schemes. Users need to predefine an initial set of blocks. The partitioning scheme is then selected by merging blocks together. The decision to merge blocks is typically based on criteria such as AIC, AICc, and BIC , the evolutionary processes are assumed to be reversible  over each edge. The most commonly used likelihood-based model selection tool in a phylogenetic context  include q.AIC incorporates the Kullback–Leibler divergence (KLD) , which mq ln(n), where n is the sample size)  after taking into account the penalty factor  . Similarle size) .AIC and BIC are both widely used in model selection, and the best-fit models for the observed data chosen by AIC and BIC are often different . Dziak eThere is a rich body of literature investigating the use of AIC and BIC in selecting simple homogeneous models in phylogenetics . There hIn phylogenetics, the nonstandard conditions are not rare scenarios and can often occur . Using AIn this study, we aimed to investigate the use of AIC and BIC to compare partition models and mixture models in standard and nonstandard conditions and we incorporated simulations to address three main questions: (1) Is AIC an unbiased estimator of the rEKL when applied to either partition models or mixture models under either standard or nonstandard conditions, and if so, is there a bias that differs systematically for partition models and mixture models? Does it differ based on the accuracy of the proposed partition? (2) Are models chosen by AIC the same as the ones chosen by BIC? (3) Do the models preferred by AIC and BIC under various levels of mispartitioning generally lead to accurate phylogenetic inference in terms of tree topologies, edge lengths, and substitution model parameters?To create heterogeneous data, we simulated two multiple sequence alignments (MSAs) under two different simple homogeneous models of DNA evolution. Each model includes a substitution model and an edge-weighted phylogenetic tree (the tree topology was fixed). The differences between these two models were only in the edge lengths and the model parameters. Each MSA contained 8 taxa and 1000 sites. These two MSAs were then concatenated together giving an MSA with 2000 sites. This was equivalent to generating the concatenated MSA under a two-block unlinked edge lengths partition model (P-UEL).To simulate a situation where the initial choice of blocks does not properly account for the heterogeneity in the concatenated MSA , we randomly selected a proportion of 0%, 5%, 10%, 15%, . . ., up to 50% of sites from each block and swapped them. That is, the sites drawn from the first block were placed in the second block, and the sites drawn from the second block were placed in the first block. This process was repeated 100 times for each proportion of mispartitioned sites giving a total of 1100 MSAs.Three sets of simulations were created under standard, mildly nonstandard and extremely nonstandard conditions . The generating trees for all simulations had the same topologies but different edge lengths . We usedThe parameters of the generating substitution models are the same for all three simulations . The metWe fitted two partition models and two mixture models to each MSA. These models are a two-block P-UEL partition model, a two-block linked edge lengths partition model (P-LEL), a two-class linked GTR parameters GHOST mixture model (M-LGP) and a two-class unlinked GTR parameters GHOST mixture model (M-UGP). A detailed explanation of these models is shown in the Models and Performance Measures section.10.5061/dryad.1jwstqjwj).The data were generated under Seq-Gen-1.3.4  and modeK is the total number of classes, Mk and Tk are the substitution model and the tree of the D is an alignment, Di is site i in D, kth class for site i and where In K is the total number of partitions, and if site i is in the set Sk = {i: wik = 1}, otherwise wik = 0.where The four models we used in the simulations were P-LEL, P-UEL, M-UGP, and M-LGP. The first two models are both two-block partition models and the last two models are both two-class GHOST mixture models. In a P-LEL model, all blocks of sites share the same set of edge lengths but have their own substitution rate parameters. The partitioning schemes used for both fitted partition models were the same, treating the first 1000 sites as a block and the second 1000 sites as another. For the groups of MSAs with different proportions of mispartitioned sites, this was equivalent to fitting the partition models with an incorrect partitioning scheme.In the two fitted GHOST mixture models, each class always has its own set of edge lengths, but may or may not share the same GTR substitution model. In an M-LGP model, each class shares the same GTR substitutional model, while in an M-UGP model, each class has its own GTR substitutional model.Using the same notation as in The details of the fitted models and the number of the free parameters are shown in rEKL is the measure we wanted to estimate in this study and is derived from the KL divergence between the true distribution and an approximating model .Let ation is :L of X.where p*(X) and p*(Y) are the true probabilities of alignments X and Y, and Y given the MLE of X. The first term in where The term p*(h) is the true probability of the site pattern h, h under the MLE of the candidate model, and N and n are the total number of taxa and the total number of sites in an alignment, respectively. It is worth noting that where As in c is the total number of simulated MSAs, and h under an approximating model when fitted to the lth simulated alignment.where We modified h indexes the site patterns, k indexes the partitions to which the sites have been assigned, j indexes components of the generating process (the truth), the set Sj indexes the true partition structure , p*J(h) is the true probability of site pattern h for the h for the kth partition when fitted to the simulated alignment l.where p*j(h) is the true probability of site pattern h for the jth class, h for the kth estimated class when fitted to the simulated alignment l, l, and where The values of AIC, BIC, and the bias in the AIC estimate of rEKL are also recorded. The equation of AIC for the model of interest is We used a different version of BIC for partition models. The BIC for mixture models is the same as the conventional one: nk is the number of sites in the kth partition, qk is the number of parameters unique to the kth partition, qc is the number of parameters common to all partitions, and where The branch score, developed by We performed element-wise comparisons between the estimated base frequencies and the generating base frequencies, and between the estimated substitution rate parameters and the generating substitution rate parameters. For the partition models, we compared the base frequencies and substitution rate parameters estimated from the first and second blocks of the MSA to the generating base frequencies and substitution rate parameters from the first and second blocks of MSA, respectively. For the mixture models, we compared the estimated base frequencies and substitution rate parameters from the classes with the two sets of generating parameters based on the allocation where the edge lengths match best.Under standard conditions, AIC was an unbiased estimator of the rEKL. The models with the smallest mean AIC also had the smallest rEKL values, that is, both the AIC and rEKL favored the same models . For MSAUnder nonstandard conditions, AIC was a biased estimator of rEKL ,c, and tUnder standard simulations, the result is the same as the result from AIC. That is, BIC preferred the P-UEL partition models from MSAs with 0% to 25% mispartitioning sites, and for the rest of the groups, BIC chose the M-UGP mixture models . Under mIn the standard and mild simulations, models chosen by AIC and BIC all recovered the true topologies very well, 100% and nearly 100% of the time for all MSAs, respectively . In the Mispartitioning had no impact on the accuracy of the topologies inferred by the partition and the mixture models . The proUnder standard conditions, mixture models had a better (lower) branch score than partition models when the proportion of incorrectly partitioned sites was 15% or greater . Under nrCG from the substitution rate parameters from block one of the partition model and one class from the mixture model  performed better at estimating the base frequencies than the partition models when the mispartitioning level was 20% and above . Under nIn general, the accuracy of the estimated substitution rate parameters for the mixture models is similar to the ones from the partition models . Under sIn summary, under nonstandard conditions, compared with the two partition models, BIC chose the simpler partition models (P-LEL) across all levels of mispartitioning. The AIC preferred the simpler partition models over the more complex partition models when there are more than 15% of sites are mispartitioned in the MSAs. Between the two mixture models, BIC preferred the simpler mixture models across all data sets. However, AIC chose the complex mixture models over the simpler ones for all the simulated data. Under nonstandard conditions, the simpler partition models chosen by AIC and BIC performed better than the complex partition models in estimating the edge lengths ,d. HowevAIC and BIC are two popular tools to use when analyzing simple homogeneous models in phylogenetics. There is limited work on the performance of AIC and BIC in selecting among partition models and mixture models. Here we have shown that AIC underestimates the rEKL under nonstandard conditions. This result is consistent with The inconsistent bias of AIC in estimating the rEKL among models, under different nonstandard conditions, suggests an incorrect calculation of the effective number of parameters. Under nonstandard conditions, in which the parameters are near the boundary of the parameter space, the effective number of parameters should not be calculated using such an approach . In a liThis study verified the results from n = 2000). The simpler partition models may be the best optimized solutions for both AIC and BIC despite the fact that AIC and BIC choose models that give different aspects in terms of the bias variance trade-off. In these circumstances, AIC and BIC may not fail, a simpler partition model may just explain the true process that generated the MSAs better than the other models for a smaller n. With a larger n, the choice of AIC and BIC may differ. This result also implies that, under nonstandard conditions, AIC and BIC may favor partition models with an underpartitioned scheme over partition models with an overpartitioned scheme.Our simulations showed that, under nonstandard conditions, AIC and BIC chose the same simpler partition models (P-LEL) when more than 15% of sites are mispartitioned. This result is interesting in terms of the bias variance trade-off. BIC favors an underfitting model which has a high bias and a low variance (specificity) and AIC prefers an overfitting model which has a low bias and a high variance (sensitivity) . HoweverThere is a slight difference in the results between BIC and AIC under nonstandard conditions when there was none  or some small amounts  of mispartitioning sites in the MSAs. That is, BIC always preferred the simpler partition model , while AIC selected the generating model (P-UEL). This result is consistent with the assertion made in Furthermore, our simulations showed that when dealing with heterogeneous data, both partition models and mixture models performed well in estimating different aspects of phylogenetic inference, and the computational time for fitting the GHOST mixture models to the data is similar to the time for the partition models in IQ-TREE2. This result is consistent with Overall, AIC and BIC may not be appropriate to use in selecting among partition models and mixture models under nonstandard conditions. Nonstandard conditions can occur in many situations: for example, when an MSA has some closely related species, or when partitioning data into small subsets, nonstandard conditions may occur. As Caution should be taken if using AIC and BIC to select among partition models and mixture models when dealing with these types of MSAs. We recommend that when fitting a partition model and a mixture model to the data, the inferred phylogenetic trees should be inspected prior to final model selection. Furthermore, when partitioning data into a large number of subsets, such that each subset has a very small sequence length, AIC and BIC may become problematic to use. This is because if the sequence length is very small, then the expected number of substitutions per alignment may be less than 5, creating a nonstandard condition. We recommend that when fitting a partition model to the data, the sequence length of the partitions should be examined. We also recommend that alternative estimators of the rEKL are worth investigating and considering in selecting between partition models and mixture models. These estimators include cross-validation scores proposed in http://dx.doi.org/10.5061/dryad.1jwstqjwjData available from the Dryad Digital Repository:


This work presents a novel framework for web-based environment-aware rendering and interaction in augmented reality based on WebXR and three.js. It aims at accelerating the development of device-agnostic Augmented Reality (AR) applications. The solution allows for a realistic rendering of 3D elements, handles geometry occlusion, casts shadows of virtual objects onto real surfaces, and provides physics interaction with real-world objects. Unlike most existing state-of-the-art systems that are built to run on a specific hardware configuration, the proposed solution targets the web environment and is designed to work on a vast range of devices and configurations. Our solution can use monocular camera setups with depth data estimated by deep neural networks or, when available, use higher-quality depth sensors  that provide a more accurate perception of the environment. To ensure consistency in the rendering of the virtual scene a physically based rendering pipeline is used, in which physically correct attributes are associated with each 3D object, which, combined with lighting information captured by the device, enables the rendering of AR content matching the environment illumination. All these concepts are integrated and optimized into a pipeline capable of providing a fluid user experience even on middle-range devices. The solution is distributed as an open-source library that can be integrated into existing and new web-based AR projects. The proposed framework was evaluated and compared in terms of performance and visual features with two state-of-the-art alternatives. Augmented reality (AR) consists in augmenting the real world with virtual layers of information . Since iWhile all AR systems need pose tracking to align virtual content with the environment, most systems still do not possess the capability of dynamically adjusting content to their surroundings, limiting the immersion of the AR application, an essential factor for the effectiveness of these systems as discussed by .Environment awareness in AR systems is an important step not only from an interactive and immersive perspective but also to improve usability, for example by adjusting fonts or content colour to improve readability based on the environment, as studied by .The computing power of mobile devices has improved significantly over the last decade and these devices have become the most common scenario for AR applications. Currently, mobile devices have powerful systems on a chip (SoC) that contain motion sensors capable of good position and orientation tracking, which, paired with modern visual camera systems composed of multiple image sensors, make smartphones a natural fit for AR experiences.Although smartphones are the most common type of devices used for AR, advanced features and more precise tracking are available on dedicated wearable devices, such as Microsoft HoloLens 2 , Magic LTo accelerate the development of AR applications, vendors have introduced dedicated APIs  that simplify the integration of advanced hardware features that provide details about the environment in which the user is located .Web-powered AR offers the capability of a setupless experience, without the need to download and install applications while offering developers the possibility of developing cross-platform AR experiences. Using modern application programming interfaces (API) , web-based AR experiences can achieve performance close to native applications, while being faster to develop due to the simplified nature of the web ecosystem.WebXR  is a webWith the rise in real-time collaborative AR experiences, the usage of web technologies has become more relevant as they allow for cross-platform communication and collaboration from a single application. Works such as the ones proposed by  or 14],,14], wheIn this document, a novel framework for environment-aware AR applications based on WebXR is presented. The novelty of this work stems from the integration of many state-of-the-art concepts in AR such as occlusion rendering, environment lighting, shadows, and physics interaction between virtual and real objects into a single performant pipeline.The solution uses physically based shading, combined with environment information that results in the rendering of 3D objects that are consistent with the real-word surroundings as demonstrated in This work was motivated by the lack of cross-device AR solutions that provide environment-aware rendering and physics for AR. Current solutions for cross-device AR  are limited to basic AR functionalities such as tracking and scene alignment .The rest of the paper is structured as follows: Traditional AR applications require previous knowledge of the environment or the usage of discrete markers to align the virtual information with real-world images and provide adequate tracking between the virtual and real worlds.The search for solutions capable of mapping and recognizing the environment for AR applications has been a topic of research for as long as the AR term has existed; an early solution for environment interaction in AR with depth estimation from a stereo camera was presented in .In 2014, Google introduced the Tango SDK, used to build AR devices equipped with depth sensors and motion tracking, capable of obtaining a detailed 3D mapping of the environment using simultaneous location and mapping (SLAM) algorithms. This innovative platform kickstarted the growth in mobile environment-aware AR and opened new possibilities regarding environment tracking and geometry occlusion between real and virtual worlds ,18.In 2017, Apple introduced a concurrent to Google Tango, the ARKit, which used the already existing hardware of iPhone and iPad devices for environment tracking, depth sensing, and lighting information.In 2018, an efficient algorithm for the densification of depth points obtained from monocular visual feature tracking integrated into a SLAM process using deep neural networks (DNN) capable of running in real time was proposed by . It prodIn the same year, Google introduced a new API for AR applications named ARCore. As its counterpart (Apple ARKit), ARCore could run on already available smartphones by using DNN to densify depth data obtained from a single camera alongside with phone position and orientation provided by the inertial measurement unit (IMU). It was also compatible with dedicated depth hardware such as time-of-flight (ToF) cameras that can provide more precise depth information. Alongside with device tracking and environment geometry information, ARCore also introduced the possibility of obtaining lighting information, image tracking, human face recognition, and mechanisms for real-time collaborative AR among devices by sharing environment features across devices .Although ARCore and ARKit proposed similar functionality, there were differences in their technical approach that were investigated by ,22, wherIn 2020, a solution for environment-aware AR named DepthLab  was intrAlso in 2020, Apple introduced solid-state Light Detection and Ranging (LIDAR) technology in smartphones. LIDAR provides high-accuracy dense depth data without the need for additional processing from DNN, improving both performance and quality of the depth information. Two examples of applications using this solution are Snapchat, which provides accurate geometry occlusion and relighting of a real world image, and Polycam, which uses the geometry provided from the LIDAR to perform a 3D reconstruction using SLAM.The web environment offers one of the most complete toolsets for Graphical User Interface (GUI) development with the possibility of developing complex 3D systems using WebGL.AR.js was the first framework widely adopted for web AR targeting smartphone devices with marker-based tracking. This approach requires the usage of markers such as QRCode, Aruco, or pretrained images to track the environment using the device camera.The usage of markers with data storage also provides context for the AR system . Without markers, AR systems must rely on object detection and classification techniques that can be computationally expensive for mobile devices.With the introduction of the first versions of WebXR, markerless web-based AR solutions have started to appear in scientific works and commercial applications. However, while WebXR provides the required technology to implement AR experiences, the inherent limitation of the web environment regarding data bandwidth and the complexity of leveraging the high-performance functionality still impose a barrier for fully featured experiences leading to badly implemented solutions, as studied in .In 2020, a solution for AR interaction with point cloud data using three.js and WebXR for environment tracking and interaction was proposed, providing a limited but performant example of a web application .The real-time realistic rendering of virtual images has been a research topic widely explored over the years from simulation applications, gaming systems, and even industrial applications. In the last few years, we observed an exponential increase in the processing capabilities of graphics hardware. Ray tracing is now a viable solution for hybrid rasterized systems in desktop computers and gaming consoles allowing the simulation of paths of light. Low-power mobile devices are now also capable of realistically rendering images using techniques that were previously not feasible such as physically based rendering (PBR) or the usage of advanced screen-space effects .AR realistic rendering techniques have been constrained by the lack of environment information  and limitations in performance. In 2003, image-based lighting (IBL)  was intrIn 2019, DeepLight , a DNN-bFor the implementation of the solution, the three.js graphics library and the JavaScript programming language were selected. The three.js library was chosen based on its rendering capabilities with PBR and WebGL 2.0 support and on its popularity to increase the reach of the solution proposed to researchers and developers. Three.js has a much larger community of users when compared to the closest competitors Babylon.js and Play Canvas.For GUI elements, the “dom-overlay” functionality, which allows the integration of document object model (DOM) elements in the AR scene, was used to implement an abstraction layer that projected and aligned these elements into the 3D scene using CSS transforms extracted from the camera projection model. These elements could also be used in 2D for on-screen user inputs and information.For the development and tests, a Xiaomi Mi 10 Lite 5G (M2002J9G) smartphone  was selected as a representation of a middle-range device that supports ARCore depth functionality. The device was equipped with a Qualcomm 765G SOC and 6 GB of RAM using the operating system Android 10. The code was tested using Chrome 88.0.4324.68. Models used for testing were obtained from the Khronos Group glTF 2.0 sample repository.In this section, the object-picking module of the solution used for interaction with the 3D virtual environment from a 2D screen points using the “hit-test” functionality of WebXR is presented. The hit-test system assumed that all interactions were performed over plane surfaces; it was capable of handling multiple planes simultaneously and provided the normal of the plane surface. The API received a ray described by its origin and direction vector and responded with the estimated distance to a real-world surface, if any. Points obtained from the hit-test were stored internally by the API as environment anchors used to realign the scene when the tracking was lost, ensuring the objects placed in the scene were always correctly realigned after tracking problems.The object placement mechanism was used to introduce virtual 3D objects using a position calculated from the hit-test result. The solution allowed the user to select screen-space points for which the origin and direction of the ray were calculated based on the device position (obtained from tracking) and the camera projection characteristics.Object picking could be combined with dedicated controllers using the Gamepad API and motion-sensing-enabled controllers  to further improve immersion. Using the object placement module, we also implemented sample measurement tools to evaluate distances, angles, and areas using the GUI toolkit.The accuracy of the tracking solution provided by the API was analysed in depth by , where tIn this section, a Graphics Processing Unit (GPU) based solution for object occlusion is presented. The quality of the depth data available depends on the system configuration and sensors existing on the device. The approach used allowed for high-precision depth data when available, by integrating the raw depth data into the rendering pipeline. The test device used a monocular depth estimation provided by the AR Core obtained using a DNN-based algorithm  with a rThe rendering pipeline for the geometry occlusion is presented in The depth data were provided as an array of 16-bit depth values, efficiently stored in a WebGL texture using the luminance alpha mode, which allowed for two 16-bit unsigned byte colour channels, exactly matching the size of the input data. The depth data provided were not normalized by default so the depth buffer coordinates may not correspond to screen-space coordinates. To convert from depth buffer coordinates, a normalization matrix was provided by the API that had to be applied to correct the screen coordinates.The resolution of the depth data could be different from the screen resolution depending on the system. In the used test scenario, the depth resolution of To reuse the existing materials from the three.js library, we analysed how the materials shaders were built from the material description. The shaders were built from a set of OpenGL Shading Language (GLSL) shader chunks, and each material had its own shader program, which, after being compiled, was managed by the internal renderer of the library.The shader code generated could be accessed from the “onBeforeCompile” call-back executed before the GLSL code was compiled, where the code for the depth occlusion was injected into the existing shader code. In that phase, uniform variables could be registered, specifically, the depth texture shared between all shaders, the normalization matrix, and the screen size. These uniforms were mapped from their JavaScript attribute name to the shader code exactly and had to be managed manually to prevent memory leaks.To blend the result between virtual environments and real-work images, a preprocessing step was applied to prevent the overdraw of fragments that would be latter discarded. To ensure no loss in precision, we compare the depth data encoded in data texture against the projected vertices z passed to the fragment shader. This approach avoided the truncation of the precision of the depth data to match the depth buffer accuracy.The screen-space depth coordinates were obtained through the rectification matrix to reconstruct the 16-bit depth value from the two 8-bit components, obtaining the original depth value in millimetres. Any fragments that had a lower depth value, meaning that they were further away from the projection origin than the real-world surface, were discarded from the result before any additional processing was performed.Combining real and virtual objects using the method proposed provided visually accurate results, where it was possible to distinguish the separation between real and virtual objects across their borders as shown in For some scenarios, it was possible to observe some inconsistencies due to errors in the depth data observed with virtual objects placed behind objects with holes, translucid objects, or with the presence of highlights and reflective surfaces and moving objects. This section describes how the solution uses lighting information and PBR rendering to provide a consistent shading appearance with the environment. PBR rendering consists in using physical models to calculate lighting in 3D-rendered scenes, instead of using a visually obtained approximation to model light interaction with surfaces. The idea is that instead of tweaking materials to “look good” under specific lighting, a material can be created that will react correctly according to calibrated lighting scenarios. In practice, this gives a more accurate and realistic-looking result than the Lambert or Blinn–Phong  models.Bidirectional reflectance distribution functions (BRDF)  define hThe “lighting-estimation” module of WebXR provides light intensity, the direction of the main light source, and a cube-map texture of the environment surroundings. These data are obtained from a device’s light sensor and camera.To access lighting data, a light probe was used to query the direction of the main light source as well as environment light information provided as a set of spherical harmonics (SH)  at each SH encode lighting conditions in spherical functions that are then projected into frequency space and represented by 9 coefficients. It is an alternative representation to a full cube map that uses less memory and is effective for scenarios where high-frequency details are not required. The API provides a set of SH coefficients for each RGB colour channel.The three.js library has a PBR rendering implementation based on the model described in . The SH The 3D models used in the framework must contain all the required physical attributes of their surfaces encoded into textures. Using the lighting information, the framework provided a solution for casting AR shadow between 3D objects and the ground planes. The technique described in Planes found were used to render shadows using shadow maps  cast froThe usage of shadow maps presents some limitations: it is not possible to have partial light occlusion from translucid objects, shadows are projected for a finite area, and the number of lights casting shadows must be limited, since computational cost and memory increase exponentially with the number of lights in the scene. Numeric precision limitation may also cause a shimmering effect commonly known as “shadow acne”, that was studied and addressed in .To ensure good performance, shadow maps are typically rendered with a resolution considerably lower than the expected shadow pixel density in the screen space and to improve their perceived quality, a filtering method can be used. We tested 3 different algorithms for shadow map filtering: percentage-closer shadows (PCS) , variancTo evaluate the performance and quality of the shadow-mapping mechanism based on its resolution and filtering algorithm, we tested multiple configurations and measured the performance for each one, and we selected a shadow area of To provide an interaction between virtual and real world objects, depth information was used to create a simplified model of the real environment that was combined with the virtual scene object for the physics simulation.For the physics simulation, the cannon.js rigid body physics engine was used. It was selected based on its performance through the usage of the SPOOK stepper  combinedThe process for the physics simulation is represented in The voxel occupancy grid consisted of a dense voxel volume that could be expanded dynamically as required. Voxel size should be adapted to the precision required. For the implementation, we selected empirically a voxel size of 5 cm. Voxels stored a weight value, which was compared against a threshold value to decide if the voxel was occupied by a real-work object or not.wV is the voxel weight, sU is the update speed, from 0 to 1, and tH is the depth hit result with values 0 or 1.Equation  represenThe depth was projected into a 3D space; each point had to hit a voxel in the grid; if not, the voxel grid needed to be expanded. For each point, all voxels close to the camera had a reduction in occupancy weight and the voxel hit by the point had its weight increased. Voxels outside of the view were not updated. To handle virtual objects and maintain a good computational performance of the physics simulation, simplified convex hulls were generated for each object geometry using the quick-hull algorithm .Information from the probabilistic voxel model and convex hulls were provided to the physics engine, which updated the simulation state and the position of virtual objects before rendering each frame.www.github.com/tentone/enva-xr (accessed on 12 February 2023).The solution presented is distributed openly as a library and focuses on providing an abstraction layer for web AR with fallbacks available for non-AR devices. The code is available on GitHub at The solution takes care of all initialization code required, so that developers can focus on the application development. The “ARApp” object provides access to all the features presented in the document. It checks for the existence of all WebXR features required in the device and manages the rendering of 3D objects and the physics simulation.The features of the system can be toggled trough flags and developers can provide 3D objects to the “scene” attribute of the application instance. A state update of 3D objects can be performed in the “onFrame” method called before rendering each frame.To evaluate the performance of the framework, a simple implementation using only the environment-tracking functionality without any lighting or geometry effect was considered as a baseline, similarly to the solution proposed by .A 3D model of an antique camera available under MIT’s permissive license was used; this model is shown in No optimizations were used during the tests and each group of entities used its own textures and geometries that were replicated in memory. All objects were placed under direct visibility, otherwise the framework would remove them from rendering. The device was placed on a fixed stand during the tests to ensure that the tracking process was under similar conditions.Incremental tests were performed for each of the features proposed with different number of objects; for each scenario, the average, minimum, and maximum values were measured. We tested a baseline approach only with environment tracking , a geomeA significant variation between minimum and maximum frame times could be observed. In the complete system test , we obseFor the most complex scenario, the device ran out of memory for more than 112 entities, caused by the quantity of texture data required for the textures and geometries. In a real scenario, this situation could be easily avoided by compressing and sharing the resources across the multiple entities that can be implemented in the solution.All tests were repeated by measuring only the rendering time of the solution proposed without accounting for the WebXR overhead, considering only the time required to update the data structures and render an image to the screen. The results are presented in In a device using a dedicated depth solution this value is expected to be the major penalty introduced by the platform, easily allowing high-end devices to sustain higher frame rates.To understand better how the WebXR API impacted performance, a new test, independent from the framework, was prepared, evaluating the times for each one of the WebXR API modules necessary to access information used by the framework. Each feature was tested separately, and the results are presented in The results suggested that no specific feature of the API imposed a higher performance limitation as the performance was similar independently of the features in use. A big gap between minimum and maximum values was observed, confirming again the substantial variation in processing time of the API.For comparison of the approach presented, we selected two solutions: DepthLab and MyWebAR. DepthLab uses the AR Core API and can be considered a representative of the state-of-art in mobile augmented reality . MyWebARDepthLab version 1.0.5 and the device presented in Although surface lighting used a similar technique with PBR materials, the proposed solution lacked environment maps, relying instead on a simplified spherical harmonics (SH) model, which, although not as detailed, provided a better consistency, as observed when comparing the results from Shadows were also obtained using different techniques; the proposed approach used shadow maps projected over planar surfaces with soft filtering, whereas the solution by  used harThe results obtained with the DepthLab solution also presented some issues as seen in For the physics interaction, the solution presented combined a voxel occupancy grid approach, which was built and updated as a simplified but dynamic model of the environment with a simplified hull for the virtual objects and which allowed the physics simulation to run even for elements outside of the viewport.The approach presented in DepthLab used a closed triangulated mesh updated in real time from the depth data available in the viewport, which did not cope well with physics elements leaving the screen space or interacting with the surface. In contrast, the proposed solution a updatedMyWebAR is a web-based commercial solution for AR experiences. It supports different types of AR projects based on image tracking, QR codes, and SLAM. For comparison, we created a SLAM project. In SLAM projects, MyWebAR can use WebXR to track the environment similarly to the solution presented.In This paper presented a framework for environment-aware AR application based on the WebXR API providing a consistent and performant package paving the way for the creation of complex AR applications that can better reflect changes in the environment.The novelty and a most significant advantage of the proposed approach resulted from the integration of multiple state-of-the-art AR features into a single performant package  were in a proposal state, and their implementation was subject to changes in future releases of the Chrome web browser.The framework presented is intended to be used by other developers and researchers. It allows users to quickly create a web AR application capable of environment interaction with minimal code required for setup.While the WebXR API specification is still under development, it already offers a significant toolset for environment-aware mixed reality. The performance instability of the WebXR API probably caused by the unpredictability of the SLAM process used to build the environment model, poses some challenges for consistent performance in more complex AR applications, which was also observed in .To further enhance the results of the work presented, real-time reflections could be introduced using environment cube-map information. It was not possible to integrate this feature because the API provided it as a preallocated WebGL texture that we were not able to integrate into the three.js rendering pipeline. An alternative algorithm for occlusion could be used to improve the visual fidelity of the solution; for instance, in Throughout the development of the proposed framework, it was noticed that memory limitations arose quickly, which was expected considering the complexity of the SLAM process used internally by ARCore to keep track of the device position and environment anchors.To reduce the memory used for textures, which represent the largest chunk of memory, GPU-compressed formats should be used whenever possible and are supported by the proposed framework. To ensure compatibility with vendor-specific formats, the approach proposed by  provides


Traditional methods of gearbox fault diagnosis rely heavily on manual experience. To address this problem, our study proposes a gearbox fault diagnosis method based on multidomain information fusion. An experimental platform consisting of a JZQ250 fixed-axis gearbox was built. An acceleration sensor was used to obtain the vibration signal of the gearbox. Singular value decomposition (SVD) was used to preprocess the signal in order to reduce noise, and the processed vibration signal was subjected to short-time Fourier transform to obtain a two-dimensional time–frequency map. A multidomain information fusion convolutional neural network (CNN) model was constructed. Channel 1 was a one-dimensional convolutional neural network (1DCNN) model that input a one-dimensional vibration signal, and channel 2 was a two-dimensional convolutional neural network (2DCNN) model that input short-time Fourier transform (STFT) time–frequency images. The feature vectors extracted using the two channels were then fused into feature vectors for input into the classification model. Finally, support vector machines (SVM) were used to identify and classify the fault types. The model training performance used multiple methods: training set, verification set, loss curve, accuracy curve and t-SNE visualization (t-SNE). Through experimental verification, the method proposed in this paper was compared with FFT-2DCNN, 1DCNN-SVM and 2DCNN-SVM in terms of gearbox fault recognition performance. The model proposed in this paper had the highest fault recognition accuracy (98.08%). Gearboxes are mainly composed of gears, shafts, bearings and casings. With the continuous development of modern science and technology, their application is becoming ever more extensive . They arA large number of scholars have carried out research on gearbox fault diagnosis. Liu et al.  used a cTraditional fault diagnosis is mainly divided into two aspects: feature extraction and pattern recognition . FeatureIn recent years, against the background of artificial intelligence, there has been continuous innovation and development in deep learning technology . This teAlthough the application of deep learning in fault diagnosis has achieved some initial results, most of the research has been based on one-dimensional vibration signal data as the input. This method has a lot of feature information and is complicated and difficult to screen. It has problems such as incomplete feature information, heavy extraction workload and easy signal interference. To a certain extent, it limits the further improvement of gearbox fault diagnosis accuracy and the optimization of diagnosis efficiency .Deep learning techniques have a natural advantage in processing images. It is therefore worth exploring the potential advantages of a bearing fault diagnosis method that converts the vibration signal into two-dimensional image data and then uses the two-dimensional image data as the input into the deep learning model . VisualiCNN is a typical deep learning model. Currently, 1DCNN and 2DCNN are commonly used. The model extracts the signal features layer by layer through convolution, pooling and nonlinear activation function mapping . Compare(1)An experimental platform for gearbox fault diagnosis was built, and a gearbox fault diagnosis method based on multidomain information fusion CNN was proposed. The method was verified as having high robustness and feasibility.(2)The SVD algorithm was used to preprocess and denoise the original signal of the gearbox. In terms of SVD signal reconstruction, a singular value energy difference spectrum was introduced. This method determines the effective order of the reconstruction matrix after singular value decomposition based on the contribution of noise signals and useful signals to singular values.(3)The one-dimensional gearbox vibration signal and the two-dimensional frequency map of STFT time and CNN were combined. CNN multifeature fusion was used to enrich the features of the two different dimensions, which reduced the problem of gearbox information loss during the adaptive extraction process.In order to effectively improve the complexity and incompleteness of manual feature extraction in the fault diagnosis process of railway vehicle gearboxes, the workload of the extraction should be high, the signal should be susceptible to noise interference, the interference caused by the impact of external experimental environment noise on fault recognition should be reduced, and more complete feature information should be saved. This article proposes a gearbox fault diagnosis method based on multidomain information fusion CNN. The contributions of this paper are as follows:This paper is composed as follows: The singular value decomposition (SVD) method is currently used in data dimensionality reduction, image processing, signal processing and other fields . In the For the gearbox vibration signal, In terms of signal reconstruction, the determination of the useful order of singular values is particularly important. If more singular values are selected for signal reconstruction, a part of the noise signal remains in the signal after noise reduction, and the noise reduction will be incomplete. However, if fewer singular values are selected for signal reconstruction, useful signals will be deleted, resulting in incomplete information in the original vibration signal . In thisIn Formula (2), Here, the sequence formed by The short-time Fourier transform (STFT) is also referred to as a windowed Fourier transform. Because the Fourier transform is only suitable for steady-state signal analysis, and as unsteady-state signals are very common in mechanical equipment, the short-time Fourier transform is a method developed to adapt to unsteady-state signal analysis . This meIn Formula (4), Assume that Convolutional neural network (CNN) is a feedforward neural network that has become one of the most commonly used algorithms in the field of deep learning in recent years, particularly in the field of pattern classification . The net(1)Input layer. The CNN input layer can preprocess the input data, such as standardization, normalization, etc.(2)Convolutional layer. The convolutional layer is the core component of CNN. Its largest feature is weight sharing, which can be realized through the convolution kernel. The convolutional layer uses the convolution kernel to locally operate on the input data to extract the corresponding features of this part. As the number of convolutional layers deepens, the required parameters also increase. Deeper features can also be extracted. The convolution operation expression is as follows:(3)Pooling layer. Pooling layers are also called downsampling layers. This layer mainly performs feature extraction and dimensionality reduction during the running of the CNN, which can reduce the amount of calculation required. To a certain extent, it can also reduce the possibility of overfitting. The maximum pooling formula is:(4)Fully connected layer and output layer. After the previous convolutional and pooling rounds, the fully connected layer of the image input is fully connected between the input and output. This mainly summarizes the features extracted by the convolutional layer and the pooling layer to achieve global optimization . The Sof2DCNN is widely used in the field of image recognition and is effective for image feature extraction and classification . It diffSupport vector machine (SVM) is a data analysis method developed on the basis of statistics. Its basic principle is to map the nonlinear problem in the original low-dimensional input space onto the high-dimensional feature space for solution, and it is often used in classification and regression analysis and to solve other problems .In SVM nonlinear data classification, the input data are mapped onto the high-dimensional space primarily via the kernel function. The selection of different kernel functions has an impact on the classification effect. Kernel functions include polynomial, Laplacian and radial basis function kernels. We selected the radial basis function kernel for our study because it has a wider range of applications. The main principle of SVM is shown in Formula (7):i-th component, linearly separable when In Formula (7), the given data set is The overall workflow of the 1DCNN + 2DCNN + SVM model is shown in The significance of multidomain information fusion lies in the integration of several excellent models via scientific methods for the purpose of removing the bottleneck of the generalization ability of a single model of unknown problems. Furthermore, the advantages of multiple models can be combined to achieve the optimal solution to a problem . The modAs shown in In order to verify the actual effect of the method proposed in this paper in gearbox fault diagnosis, this experiment used the JZQ250 fixed-axis gearbox for fault diagnosis research. The experimental platform is shown in (1)An air switch was added between the inverter and the power plug to ensure that the experimental process was carried out under safe conditions;(2)The motor was connected to the frequency converter, and then the gearbox and the motor were connected by a belt. The magnetic powder brake and the gearbox were connected via coupling.(3)A piezoelectric acceleration sensor was installed at the axial position of the high-speed shaft end cover of the gearbox and was connected to a PC via an acquisition card.It can be seen from This was a no-load gearbox experiment, meaning that the magnetic powder brake was closed. In terms of fault diagnosis experiment design, system variability and limited fault coverage will affect the accuracy and reliability of fault diagnosis techniques. As most internal failures in gearboxes occur in the gears, we primarily focused on the gears. The type of gear measured in the experiment is shown in It can be seen from The number of training, verification and test sets are shown in As can be seen from The collection of original vibration signals through gearbox fault diagnosis experimental platforms is usually accompanied by a lot of noise and the aliasing of multiple frequency components, meaning that filtering noise signals and effective feature extraction are very important steps . For thiIt can be seen from STFT was performed on the preprocessed gearbox vibration signal to obtain a two-dimensional time–frequency diagram. Time–frequency diagrams of the randomly intercepted parts of the gearbox signal data in the four states are shown in The computer operating system used in this experiment was Windows 11, the programming language was python3.7 and the deep learning framework was Keras. In order to verify whether the fault accuracy of the multidomain information fusion CNN network model proposed in this paper was as expected, the entire data set was divided into training, verification and test sets. The training and verification set were first used to train the multidomain information fusion CNN network model. Then, the test set was input into the trained model and the result was output to obtain the accuracy of the test set. From the loss value change curves of the training and verification sets in From the change curve of the accuracy rate of the training and verification sets shown in t-SNE (t-distributed stochastic neighbor embedding) is a dimensionality reduction technique. It is applied to represent high-dimensional data in a two-dimensional or three-dimensional low-dimensional space and can be used for the visualization of high-dimensional data . This teIt can be seen from For the final classification, we used the sample feature set output from the fully connected layer (FC layer) as the input sample of the SVM model. The SVM algorithm is essentially a binary classification algorithm, and there are two methods for solving classification problems. The first method is to directly modify the objective function and achieve multiclassification by solving optimization problems. However, this method has high computational complexity and is generally only suitable for use on small sample data. The second method involves combining multiple binary classifiers to construct multiple classifiers, commonly known as the one-to-one method and the one-to-many method. Our study adopted a one-to-one classification method.We identified and classified gearbox fault types by SVM: SVM penalty parameter c = 0.75 and kernel function parameter g = 0.14. The confusion matrix of the test sample classification results of this run is shown in From the sample test results shown in To prevent contingencies in the results, the program was run multiple times. The results from running the program 10 times are shown in Finally, in order to further verify the superiority of the multidomain information fusion CNN network model, we used the FFT-2DCNN, 1DCNN-SVM and 2DCNN-SVM models for comparison and verification. The average value was obtained for each run 10 times, and the final diagnostic results and standard deviations are shown in Under the same experimental conditions, four fault diagnosis methods were compared. The FFT-2DCNN model performs FFT on the original vibration signal of the gearbox to obtain a grayscale image. Then, the grayscale data set obtained by FFT is sent to 2DCNN for fault identification and classification. The 1DCNN-SVM model directly puts the original vibration signal of the gearbox into a one-dimensional convolutional neural network for feature extraction. Then, it uses the support vector machine to identify and classify the faults. The 2DCNN-SVM model puts the original vibration signal of the gearbox into a two-dimensional convolutional neural network for feature extraction and then the support vector machine is used in place of the Softmax layer for fault recognition and classification. The selection of experimental samples for all methods was carried out in the same way as in the experimental design parameters of this study. As can be seen from In order to further prove the superiority and stability of the model. This paper uses the Case Western Reserve University data set for analysis. The experiment uses a 1.5 kW three-phase motor, a torque sensor and a dynamometer. The model of the bearing to be tested is SKF6205-2RSJEM deep groove ball bearing, and the sampling frequency is 48000 Hz. The experimental data set is shown in It can be obtained from Under the same experimental conditions, compare the four fault diagnosis methods mentioned in 5.6.1. The average accuracy and standard deviation are shown in It can be seen from (1)The gearbox fault diagnosis method based on a multidomain information fusion CNN model is feasible and effective. The model combines a 1D gearbox vibration signal, an STFT 2D time–frequency map and a CNN. CNN multifeature fusion is used to enrich the features of two different dimensions, and two-channel random features are pooled and fused into a one-dimensional feature array. The extracted features are fully enhanced and fused to achieve the purpose of intelligent gearbox fault diagnosis. The model also avoids the incomplete expression of feature information caused by feature extraction and the low accuracy of traditional pattern recognition methods.(2)A comparison of the model proposed in this paper with the FFT-2DCNN, 1DCNN-SVM and 2DCNN-SVM models shows that the method proposed in this paper has higher accuracy and stronger generalization ability. In addition, it provides a new conceptualization of a physical model for gearbox fault diagnosis and identification.(3)In research on the fault diagnosis of future rail vehicle gearboxes, multiple sensors can be sampled for data acquisition and multiphysics domain data fusion in order to improve the accuracy of the diagnostic results.This paper proposes and verifies a gearbox fault diagnosis method based on a multidomain information fusion CNN model. A JZQ250 fixed-axis gearbox was used to design and build a fault diagnosis experiment, which verified the effectiveness of the method. An acceleration sensor was used to obtain the original vibration signal of the gearbox, and SVD was used for signal preprocessing and noise reduction. Then, the parallel structure was used to simultaneously perform 1DCNN and 2DCNN for feature extraction. Finally, SVM was used for pattern recognition and classification. The conclusions are as follows:


Establishing an accurate and computationally efficient model for driving risk assessment, considering the influence of vehicle motion state and kinematic characteristics on path planning, is crucial for generating safe, comfortable, and easily trackable obstacle avoidance paths. To address this topic, this paper proposes a novel dual-layered dynamic path-planning method for obstacle avoidance based on the driving safety field (DSF). The contributions of the proposed approach lie in its ability to address the challenges of accurately modeling driving risk, efficient path smoothing and adaptability to vehicle kinematic characteristics, and providing collision-free, curvature-continuous, and adaptable obstacle avoidance paths. In the upper layer, a comprehensive driving safety field is constructed, composed of a potential field generated by static obstacles, a kinetic field generated by dynamic obstacles, a potential field generated by lane boundaries, and a driving field generated by the target position. By analyzing the virtual field forces exerted on the ego vehicle within the comprehensive driving safety field, the resultant force direction is utilized as guidance for the vehicle’s forward motion. This generates an initial obstacle avoidance path that satisfies the vehicle’s kinematic and dynamic constraints. In the lower layer, the problem of path smoothing is transformed into a standard quadratic programming (QP) form. By optimizing discrete waypoints and fitting polynomial curves, a curvature-continuous and smooth path is obtained. Simulation results demonstrate that our proposed path-planning algorithm outperforms the method based on the improved artificial potential field (APF). It not only generates collision-free and curvature-continuous paths but also significantly reduces parameters such as path curvature (reduced by 62.29% to 87.32%), curvature variation rate, and heading angle (reduced by 34.11% to 72.06%). Furthermore, our algorithm dynamically adjusts the starting position of the obstacle avoidance maneuver based on the vehicle’s motion state. As the relative velocity between the ego vehicle and the obstacle vehicle increases, the starting position of the obstacle avoidance path is adjusted accordingly, enabling the proactive avoidance of stationary or moving single and multiple obstacles. The proposed method satisfies the requirements of obstacle avoidance safety, comfort, and stability for intelligent vehicles in complex environments. The development of intelligent driving technology holds significant importance in enhancing driving safety, improving travel efficiency, alleviating traffic congestion, and enhancing driving comfort. Among the various core technologies of intelligent vehicles, local path planning plays a crucial role in converting sequences of driving behavior decisions into executable local motion paths or trajectories of vehicle controllers. Consequently, local path planning serves as a key factor in determining driving quality and ensuring operational vehicle safety, thereby occupying a vital position in intelligent vehicle research . Path plTo generate safe, comfortable, and easily trackable obstacle avoidance paths, the problem of path planning requires the establishment of an accurate and computationally efficient driving risk assessment model. It is essential to consider the influence of the vehicle’s motion state and kinematic characteristics on path planning to enhance the model’s adaptability to dynamic driving scenarios. For the path-planning problem, scholars at home and abroad have carried out extensive research, and the commonly used local path-planning methods can be roughly categorized into five categories: the graph search method, the sampling-based method, the geometric curve method, the artificial potential field method, and the optimal control method . The graIn 1986, Khatib proposed the artificial potential field algorithm and introduced it into the field of robot path planning . HoweverConsidering the similar characteristics between the driving risks posed by traffic factors to driving vehicles and the fields in physics, Wang et al. proposed the concept of the “driving safety field” to quantitatively evaluate the risks posed by various elements in traffic to vehicles on the road ,28. ObjeIn order to devise an obstacle avoidance path that ensures safety, smoothness, and compliance with the vehicle’s motion characteristics, this paper introduces a dynamic obstacle avoidance path-planning method with a two-layered structure, as depicted in The initial path-planning process for dynamic obstacle avoidance, based on the driving safety field, is illustrated in The vehicle body and global coordinate systems are defined in a” at position According to the theory of the driving safety field, the lane line belongs to the second category of stationary objects. The field strength, denoted as When there are obstacles or slower vehicles on the road, the ego vehicle typically chooses to change lanes for obstacle avoidance. In this situation, the potential field generated by the road lane line between the current lane and the target lane will no longer constrain the vehicle. Instead, the potential field created by the lane line will be influenced by the position of the obstacles.In the equation above, the function b is denoted as b to the point b, Static obstacles on the road mainly include stationary vehicles, guardrails, roadblocks, and so on. Based on the theory of the driving safety field, static obstacles are classified as the first category of stationary objects. The potential field generated by static obstacle “c” at position c” is denoted as c” to the coordinates c” is represented by The kinetic field generated by dynamic obstacles is influenced by the characteristics of the objects , their motion state, and road conditions. The field strength, denoted as d” at coordinates The driving field is derived from the ego vehicle’s need to efficiently and safely complete driving tasks, enabling it to reach the target destination. The vehicle’s powertrain system generates driving force, while the steering system produces the steering angle to propel the vehicle forward. The driving field is influenced by various factors, including the ego vehicle’s attributes (type and position), the target location (relevant to driving behavior), the motion state (vehicle velocity), and road conditions. The field strength vector at point In the path-planning process, the vehicle is simplified as a particle, and the motion state of the vehicle during the planning process is described using parameters such as the position of the vehicle’s center of mass, velocity vector, and heading angle. If we disregard the influence of the lateral deviation angle of the center of mass, the velocity direction and heading angle of the vehicle’s center of mass remain consistent. The combined effect of forces in the driving safety field modifies the magnitude of the vehicle’s center of mass velocity and determines its heading for the subsequent stage. Assuming a lane-changing obstacle avoidance process, the longitudinal component of the center of mass velocity, k” represents the k-th stage, and ψ(k) signifies the heading angle of the vehicle’s center of mass, indicating the direction of the resultant force acting on the center of mass due to the driving safety field.In the obstacle avoidance path-planning process, the variable “k + 1)-th stage is estimated based on the position k-th stage using the following formula:To update the vehicle’s position for the next stage, a fixed time interval The process of lane-changing obstacle avoidance entails not only the provision of collision-free paths to ensure safe passage but also the consideration of the vehicle’s unique kinematic and dynamic characteristics. This guarantees that the vehicle can travel smoothly along the planned path, adhering to the specified time constraints.The planned lane-changing obstacle avoidance trajectory requires a continuous curvature, and at the same time, is subject to the constraints of the vehicle’s minimum steering radius The obstacle avoidance path is the target of the vehicle tracking control. If the vehicle can accurately track the target path, according to the principle of Ackermann’s steering geometry, from the vehicle kinematics equations, the target steering angle 2; 2. The steering sensitivity is a commonly employed metric for assessing the precision and stability of the vehicle’s driving direction. The calculation formula is as follows:2/m2. When Dynamic constraints refer to the limitations and restrictions imposed on the vehicle’s motion and behavior due to its physical and mechanical properties. To prevent lateral skidding of the vehicle during the steering process for lane-changing obstacle avoidance and improve the comfort of path-tracking control, certain constraints need to be satisfied for the lateral acceleration Assuming that the lane-changing obstacle avoidance process of the vehicle corresponds to steady-state steering and the lateral deviation angle of the vehicle’s center of mass is minimal, we can approximate the center of mass velocity as the velocity component of the vehicle along the X-axis under constant speed conditions. Consequently, we can determine the lateral acceleration as follows:Based on the analysis above, the front-wheel steering angle of the vehicle is affected by the vehicle velocity Two circles with the same radius are used to cover the minimum outer rectangle of the vehicle contour uniformly to realize the approximate description of the vehicle shape in , as showi and vehicle j, respectively; and Collision detection between two vehicles based on double circles describing the vehicle contour can then be simplified to determine whether there is an overlapping region between the two circles covering the ego vehicle and any circle covering other vehicles around it. Whether there is an overlap between the two circles can be judged based on the distance between the centers of the two circles; therefore, collision detection between vehicles can be carried out based on the following equation:In addition, it is also necessary to ensure that the vehicle always stays within the road boundary while driving along the planned waypoint, which can be judged by whether the two circles covering the vehicle intersect with the lane boundary. In the scenario shown in Let Drawing inspiration from the path-smoothing methods described in ,31, thisThe smoothness of the resulting path increases as the angle between the two-line segments formed by three consecutive waypoints in the smoothed waypoint set becomes larger. This indicates a flatter and smoother path. Therefore, the definition of the objective function for path smoothness is as follows:To ensure the similarity in geometry between the smoothed discrete waypoints and the path geometry planned with the DSF, the cost function for geometric similarity is defined as follows:The integrated objective function for waypoint smoothing is defined as follows:Discrete path point smoothing considers the physical constraints of the vehicle steering system and the positional constraints of the original obstacle avoidance path points.Firstly, considering the vehicle steering system constraints, the vehicle driving along the smoothed path points should satisfy the constraint of the minimum turning radius To improve the computational efficiency in solving the path smoothness issue, the nonlinear constraint of the turning radius shown in Equation (26) needs to be linearized. Assuming that the distance between adjacent path points is approximately equal to the arc length between the two points, the expression F(X) is determined as follows:In the equation mentioned above, the expression for Furthermore, the smoothed waypoints should be positioned in the vicinity of the original waypoints. The expression for the position constraint is given below:In summary, the objective function COST and the two constraint conditions form the mathematical model of the discrete waypoint smoothing problem. By adopting the method described in , this smN+1 waypoints, the obstacle avoidance path can be divided into N intervals. For each interval, two fifth-degree polynomials are used to represent the path. For instance, the representation of the i-th sub-interval polynomial is as follows:To further enhance the smoothness of the obstacle avoidance path and achieve continuous changes in curvature, this paper employs piecewise fitting using fifth-degree polynomials to smooth the path points, thereby generating a smooth path. Assuming that the path generated with the DSF consists of i-th sub-interval is as follows:In the equation mentioned above, the representation of the optimization variables for the To solve the optimization problem and obtain the undetermined parameters To ensure the continuity and smoothness of the final obstacle avoidance path formed by connecting the polynomial curves in each segment, several constraints need to be satisfied during the process of solving for the polynomial curve parameters. These constraints include the position, velocity, and acceleration constraints at the starting points and endpoints, as well as the position constraints for intermediate waypoints (excluding the starting points and endpoints). Furthermore, continuity constraints at the connection points of adjacent polynomial curves must be met. It is worth noting that all of these constraints are linear constraints.In order to verify the performance of the proposed obstacle avoidance path-planning algorithm based on the driving safety field, the verification and comparative analysis were carried out based on the MATLAB 2021a platform, and the model parameters are shown in To verify the feasibility and effectiveness of the path-planning method proposed in this paper for obstacle avoidance, three types of scenarios, namely, static obstacles, dynamic obstacles with low velocity, and multiple obstacles, were established, and the quality of the obstacle avoidance path generated using the proposed method was evaluated by comparing it with a path-planning method for obstacle avoidance based on the improved APF in .Test scenario 1 is a two-lane scenario with a static obstacle, as shown in In order to compare the quality of the obstacle avoidance paths planned using different methods, and to verify the effectiveness of the smoothing method proposed in this paper, the obstacle avoidance paths are planned with the DSF, the APF, and the smoothed APF for the static obstacle scenario, and the curvature of the path, curvature change rate, and heading angle were, respectively, computed for the three types of generated obstacle avoidance paths, and the results of the computation are shown in To quantitatively evaluate the quality of the obstacle avoidance paths, we selected four metrics for comparison: maximum curvature, maximum steering angle, distance to the target position, and minimum distance to obstacles (obstacle vehicles). These metrics are presented in The DSF-based path-planning process assumes that the vehicle’s longitudinal velocity is kept constant, and then the lateral velocity and lateral acceleration can be obtained based on the calculation of the heading angle of the test vehicle as it drives along the planned path; moreover, the paths planned based on the driving safety field implicitly have information in the time dimension. x = 10 m/s, the test vehicle starts to perform lane-changing obstacle avoidance at the position of x = 10 m, and when vx = 18 m/s, the test vehicle starts to perform lane-changing obstacle avoidance at the position of x = 1 m. In order to illustrate the problem more clearly, The driving safety field constructed based on field theory has the characteristics of a vector field in physics, and the force that the test vehicle receives in the driving safety field is not only related to the field strength formed by the traffic elements but is also related to the motion state of the test vehicle. Whether the traditional or improved artificial potential field is used for obstacle avoidance path planning, the motion state of the test vehicle is not taken into account; therefore, faced with the scenario shown in Test scenario 2 is a two-lane scenario with low-velocity obstacles; the test vehicle is driving at 10 m/s, and the low-velocity obstacle is located in the lane where the test vehicle is driving at 1.5 m/s. In order to improve driving efficiency, it is required that the test vehicle is able to complete the obstacle avoidance process by lane changing in a safe and smooth manner. The simulation results are shown in A comparison of path curvature, curvature change rate, and heading angle of obstacle avoidance paths planned using different methods in dynamic obstacle scenarios is shown in Test scenario 3 is a two-lane scenario with multiple obstacles; the test vehicle is driving at a velocity of 10 m/s; and there are three obstacle vehicles on the road ahead, which requires the test vehicle to complete the obstacle avoidance process in a safe and smooth manner. The obstacle avoidance paths in multiple obstacle scenarios are shown in The curves of the curvature and heading angle of the obstacle avoidance path generated using three methods in multiple obstacle scenarios are shown in The evaluation parameters of the obstacle avoidance paths generated using the three methods in the multiple obstacle scenario are shown in In response to the issues of the curvature-discontinuous path and the lack of consideration for the influence of the vehicle’s motion state in the application of improved artificial potential field (APF) to obstacle avoidance path planning, a dynamic path-planning method for obstacle avoidance based on the driving safety field (DSF) is proposed in this work. The method decomposes the path planning for obstacle avoidance into two steps. Firstly, the driving safety field theory is utilized to establish a comprehensive driving safety field for the obstacle avoidance scenario. By employing a virtual kinematic system, a collision-free feasible lane-changing path is generated. Secondly, the obstacle avoidance path is smoothed using quadratic programming, resulting in a dynamically continuous and smooth obstacle avoidance path. Simulation and comparative analyses were conducted in scenarios with static or dynamic single and multiple continuous obstacles. Compared with the improved APF, the path-planning algorithm for obstacle avoidance proposed in this paper generates a collision-free and curvature-continuous path, significantly reducing the maximum curvature, maximum heading angle, and distance to the target position of the obstacle avoidance path. Moreover, the minimum distance to the obstacles is significantly better than that of the comparative methods. Additionally, the proposed method automatically adjusts the starting position of the obstacle avoidance maneuver based on the vehicle’s motion state. As the relative velocity between the ego vehicle and the obstacle vehicle increases, the starting position of the obstacle avoidance path is set farther away from the obstacle vehicle, allowing for sufficient longitudinal space, which improves the safety of the lane-changing process. Therefore, the proposed method satisfies the requirements of obstacle avoidance safety, comfort, and stability for intelligent vehicles in complex environments. In order to enhance its applicability, future work involves expanding this method to scenarios that encompass complex factors such as pedestrians, vehicles, and road conditions. By deploying and testing the algorithm in real vehicles, further improvements in algorithm performance can be achieved.


The development of soft robotic hand exoskeletons for rehabilitation has been well-reported in the literature, whereby the emphasis was placed on the development of soft actuators for flexion and extension. Little attention was focused on developing the glove interface and attachments of actuators to the hand. As these hand exoskeletons are largely developed for personnel with impaired hand function for rehabilitation, it may be tedious to aid the patients in donning and doffing the glove, given that patients usually have stiff fingers exhibiting high muscle tone. To address this issue, a hybrid securing actuator was developed and powered pneumatically to allow for rapid securing and release of a body segment. As a proof of concept, the actuator was further adapted into a self-securing glove mechanism and assembled into a complete self-securing soft robotic hand exoskeleton with the attachment of bidirectional actuators. Our validation tests show that the self-wearing soft robotic hand exoskeleton can easily conform and secure onto the human hand and assist with manipulation tasks. The advent of soft robotics research in recent years is a response to the increasing need for automation, efficiency, and human-robot collaboration, especially in healthcare . A majorA prevailing feature of the developed hand exoskeletons in literature is the incorporation of actuators mounted on gloves that the user wears. The actuators are the driving force in aiding the flexion and extension of the impaired fingers, while the glove acts as an interface between the hand and the actuator, holding the actuators in place to act upon the user’s hand. In most hand exoskeleton designs, much emphasis is placed on the actuator’s functionality, whereas soft actuators are characterized in great detail concerning their bending characteristics and force output . On the The concept of self-securing and self-adjustment has been explored in several application domains, such as adaptive clothing for the disabled , aerospaThe securing actuator developed in this study is a disjointed circular ring in its inactivated state, which passively encircles a body, tethering itself to the body . ActivatThe securing actuator is constructed by encasing a flexible polyvinyl chloride (PVC) endoskeleton, shaped as a disjointed ring, within a fabric air bladder, hence its hybrid nature. The PVC endoskeleton supports the securing actuator mechanically and defines the shape taken by the actuator, either the curled or uncurled configuration. On the other hand, the fabric air bladder component is pneumatically powered and can take two forms. In its inactivated state (without air pressure within its chambers), it is flaccid and compliant. Upon activation by air pressure, it stiffens, straightens into a straight beam-like structure, and resists deformation. Switching between curled or uncurled states of the actuator involves the interplay of two opposite forces—the curled mechanical state of the supporting endoskeleton in the absence of external forces, much like a coiled spring, and the opposing straightening force provided by a pneumatically-powered air bladder upon activation. Without external forces, the actuator assumes its disjointed ring or C-shape. This is the non-activated or ‘secured’ state . The fabFabrication of the securing actuator involves attaching the PVC endoskeleton ring atop the fabric air bladder component and sealing it with another layer of fabric. Thermoplastic Polyurethane (TPU)-coated polyester fabric  was used to fabricate the air bladder component to attach the exoskeleton to the air bladder. TPU melts and forms an adhesive under heat and pressure. The TPU coating is on one side of the fabric, allowing the air bladder to be sealed by placing two TPU-coated sides against each other and applying a heat press. The supplied heat and pressure melt the TPU and cause the two TPU-coated surfaces to adhere to another one, forming the seal . The airThe process of fabrication is outlined in The securing actuator was characterized in terms of two defined quantities: The locking force and opening width. The locking force was defined as the force needed to overcome the securing forces conferred by the endoskeleton to hold the enveloped body segment in place. The opening width was the linear displacement that can be inserted and secured.The locking force was measured using an Instron machine  with a mounting platform and 50 N tension load cell . The steThe opening width of the actuator was determined visually using a camera and two black tape markings attached at each end of the actuator, as is shown in In addition to the opening width, a finite element (FE) model was developed to gain quantitative insights into the forces involved. This is because in the opening width characterization, the loading conditions of the actuator during inflammation of the air bladder are complex, and the force of opening of the actuator consists of the embedded endoskeleton ring. The FE model only considers a half-ring structure representing the endoskeleton, with applied external loading representing the forces exerted by the air bladder. Only half of the endoskeleton ring is modeled by setting symmetric boundary conditions on the middle cross-section because the mechanical conditions are symmetrical to the midline.F represents the pneumatic force on the end of the endoskeleton ring and pb represents the pneumatic pressure on the external surface of the endoskeleton. Material parameters were obtained directly from a simple tension test conducted on the PVC material of the endoskeleton, revealing Young’s modulus of 0.89 GPa. The forces’ direction (as defined by θ) is approximately based on the observation in the opening width experiment. A normalized force (F*) is defined as shown in Eq. max is the maximum force applied on the ring, and the value of F* was increased linearly from 0 to 1. The actuator was applied as a self-securing glove interface within a soft robotic hand exoskeleton. Currently, targeted users of rehabilitative hand exoskeletons have various hand impairments, such as stroke, and the main difficulty is getting their hands inserted within the exoskeleton for it to work. The main idea behind the developed self-securing glove is that the user can don the hand exoskeleton by simply placing their hands onto the glove in its uncurled ‘released’ state and then securing their hands by activating the ‘secured’ and curled state. This can help reduce setup time and caretaker assistance needed for donning and doffing hand exoskeleton for rehabilitation.Two new actuator types were developed for the self-securing glove, adapted from the basic design of the securing actuator for securing the fingers and the palm. They are termed the finger-securing mechanism and palm-securing mechanism, respectively. Subsequently, the self-securing glove is combined with assistive bidirectional finger actuators (developed in previous work ) to yielThe finger-securing mechanism comprises three combined actuators to hold each finger’s phalanges. The air bladder component’s design was adapted and extended to supply pressure to all three actuators-components. Instead of a single rectangular piece, a three-branched pattern was patterned and cut using the TPU-coated polyester fabric. Each brand contains a single securing actuator for the respective phalanx . MaskingThe palm-securing mechanism is larger than the finger-securing mechanism, designed to secure the sides of the palm. The fabrication of this component is identical to that of the single securing actuator except that two endoskeleton rings, similar in dimension to that used in the finger-securing mechanism, were used, one for each side of the palm. Two layers of rectangular-shaped TPU-coated polyester fabric sheet were cut out . The airFirst, five finger-securing mechanisms were assembled in the directions of the fingers of a hand, and the palm-securing mechanism was placed at the sides of the palm . These sFor hand impairment without muscular tone, to don the self-securing hand exoskeleton , the extAn alternative method of donning was devised for hand impairment with high muscle tone . First, To investigate if the addition of the self-securing glove within the hand exoskeleton would affect its mechanical behavior and outcome of rehabilitation activities performed with it for patients, a visual analysis was carried out to compare the self-securing hand exoskeleton with a previously developed manual, hand-worn soft robotic hand exoskeleton. In particular, the flexed position of the user’s hand within the exoskeleton on activation of the assistive flexion component of the bidirectional actuators was compared when either hand exoskeletons were used.A gripping test was made to measure the gripping force of the assistive glove. The grip force applied by the assistive glove was measured with a universal testing machine (Handy Tester JSV H10000) to obtain normal and frictional grip forces. The assistive glove was pressurized to 120 Kpa to bend and grasp a cylinder that is 80 mm in diameter in a vertical orientation for frictional grip force . The cylTen measurements for the force exerted on the load cell’s claw were collected for the locking force characterization test. The average result with the standard deviations is presented in The air bladder’s pressure was increased in a stepwise manner, in steps of 10 k Pa, from 0 KPa to 150 kPa. The averaged opening width profile, as shown in The results for the flexion angles during the cyclical bidirectional actuator activations are shown in Several authors used grip tests to measure the gripping force of their respective works . The griIn this paper, the concept for an endoskeleton-supported hybrid securing actuator was developed to secure wearable devices to the human body. It acts as an interface between the wearable component, such as an actuator or sensor, and the human body segment. The core mechanism involves the mechanical behavior and interaction of two key components-a flexible endoskeleton strip and a pneumatic air bladder. The endoskeleton strip gives shape and defines the configuration of the actuator. At the same time, the pneumatic bladder provides the external forces necessary to bring the actuator from one configuration to the other. As a proof of concept, we have further applied the securing actuator as a self-securing glove within a soft robotic hand exoskeleton to make it more wearable for personnel with impaired hands or hands with a high muscular tone, such as in the case of stroke.Characterization of the securing actuator was conducted in terms of two quantities—the locking force of the actuator when it is in the default ‘secured’ and curled state; and the opening width, defined as the distance the actuator uncurls upon varying levels of pressurization. From our results, it was observed that the highest force required to overcome the forces was 1.6N, which was considerable in securing small items to the human body, such as the phalanges of the hand. As for the opening width, our experiments revealed that the maximal opening width obtained from the porotype is 48mm, which is sufficient for using it on fingers. Notably, to gain more insights into the forces involved in the opening width experiment, we designed a finite element model and defined the forces of opening the actuator. Our results show a good match between the experiment and model, demonstrating that such modeling techniques may be utilized to optimize actuator design in future studies.The actuator was further developed into a self-wearing glove for a soft robotic hand exoskeleton. Using the basic actuator, different types of securing mechanisms were developed for the fingers and the palm. The fully assembled self-wearing glove mechanism quickly secures or releases the user’s hand from the hand exoskeleton. In this test, the evolution of bending angle during flexion-extension cycles for the self-securing hand exoskeleton was similar to that of the manually-worn exoskeleton . This showed that in addition to force transmission of the finger actuators, the kinematics of the subject’s hand under actuation were also unaffected after integrating the self-securing glove.The current work was limited to developing self-securing actuators fitting the fingers and the hand sizes. A plausible extension to this work will be to scale the securing actuators to different sizes and extend its application to include apparel, such as wearing a surgical gown, a practical application with which the concept of self-securing can very much help. Also, fatigue testing of the securing actuator and the self-securing glove mechanism would be considered for future work as this is important in determining the life cycle and durability of the actuator in operation.