Music can serve as a potent tool for conveying emotions and regulating learners’ moods, while the systematic application of emotional assessment can help to improve teaching efficiency. However, existing music emotion analysis methods based on Artificial Intelligence (AI) rely primarily on pre-marked content, such as lyrics and fail to adequately account for music signals’ perception, transmission, and recognition. To address this limitation, this study first employs sound-level segmentation, data frame processing, and threshold determination to enable intelligent segmentation and recognition of notes. Next, based on the extracted audio features, a Radial Basis Function (RBF) model is utilized to construct a music emotion classifier. Finally, correlation feedback was used to label the classification results further and train the classifier. The study compares the music emotion classification method commonly used in Chinese music education with the Hevner emotion model. It identifies four emotion categories: Quiet, Happy, Sad, and Excited, to classify performers’ emotions. The testing results demonstrate that audio feature recognition time is a mere 0.004 min, with an accuracy rate of over 95%. Furthermore, classifying performers’ emotions based on audio features is consistent with conventional human cognition. Among tnetworks . For exanetworks .Music classes can allow students to express themselves; for example, in the knowledge teaching of “Reforming Rivers and Mountains to Wait to born later”, teachers can let students learn folk music and stimulate the emotional experience of folk music. In the study of “Homesick”, students can feel the creator’s feeling of homesickness. However, song implication and emotional expression are difficult to spread effectively in online learning. The essence of music lies in its possible emotional expression . CurrentOnline music instruction is poised to offer a captivating alternative to traditional classroom pedagogy, affording students unprecedented opportunities to acquire musical skills irrespective of temporal, spatial, and financial constraints. Nonetheless, a lingering challenge faced by music educators lies in their ability to promptly discern fluctuations in students’ emotional states during song interpretation, underscoring the need to explore Internet-based technologies tailored to online music instruction. By doing so, we can effectively bolster the quality of music literacy, artistic assessment, and overall teaching standards, ultimately driving the evolution of a new and personalized online instructional model.This article firstly designs a music feature recognition system based on IoT technology, which realizes the rapid, accurate and intelligent segmentation and recognition of notes. Then, the identified music signals are classified by RBF neural network. Generally speaking, the Internet of Things is an extension and expansion of the network based on the Internet. It combines various information-sensing devices with the Internet to form a vast network which can realize the interconnection of people, machines and things at any time and anywhere. In addition, RBF  is a single hidden layer feedforward neural network that uses the radial basis function as the hidden layer neuron activation function. In contrast, the output layer is a linear combination of the output of the hidden layer neuron. Its main contributions are as follows:Segmentation is carried out based on the changing trend of physical characteristics of musical notes, where the first segmentation is realized by the start-stop time and pause of notes and the second segmentation is completed according to the voice segment, noise segment and vocal segment of music, thus completing the division of a frequency spectrum into independent note tones.The proposed method can fuse multi-source information and improve the accuracy of multi-classification. When classifying music emotions, the emotional tags marked by user groups represent the user’s subjective experience, which means the user’s emotional perception of music after deduction. The emotional recognition results are closer to the user’s personal feelings.This article integrates the human-computer interaction technology of the IoT and artificial intelligence algorithm, which makes different people form an excellent teaching loop in online music education and provides a new idea for realizing the interactive teaching method of music and intelligent mobile terminals.Currently, the most common method of multimodal music feature recognition is the combination of lyrics and audio. Among the existing methods, the machine learning method using audio bottom features is blind and unknown. The cost of comparing the combination of audio bottom features in a wide range is too high. Meanwhile, the traditional audio feature recognition method cannot solve the problem of speech recognition in a noisy environment in online education. IoT technology realizes the real-time transmission of music signals through wireless networks, which can quickly collect, process and analyze them.In recent years, the accuracy of using deep learning methods to identify musical emotions has been dramatically improved . Most ofFrom the above analysis, it can also be seen that musical emotion analysis is inseparable from feature extraction and quantification. Especially in online education, the music features of the music works performed by students are not as good as those published. Meanwhile, when the users guide the search to express teaching emotion, the feedback results will develop in a direction favourable to users’ requirements. Thus, in light of these limitations, collaboration between humans and machines becomes essential to compensate for the shortcomings of computer-based understanding. Integration of Internet of Things (IoT) technology with deep learning represents a promising avenue for bridging the gap between users’ high-level semantic concepts and the underlying characteristics of music, thereby significantly enhancing the accuracy of emotional interpretation and ultimately improving the effectiveness of online music instruction.The overall structure block diagram of the proposed system is shown in The module includes a music acquisition sub-module and a voice coding sub-module. The music acquisition sub-module comprises sound sensors installed in different positions responsible for acquiring the original music signals.via network communication to enable the division of music melody based on the start and end times of individual notes. This approach is advantageous in improving the efficiency and accuracy of music melody classification. Once an error occurs, the accuracy of the segmentation result will be reduced. In this article, the note segmentation recognition method of audio feature technology is used to cut the note, and the specific implementation process is shown in In the realm of human-computer interaction for the Internet of Things, the perception layer can effectively capture online audio recordings of students’ music interpretation. This audio can then be transmitted (1) Firstly, the selected musical notes are segmented for the first time according to the sound level, so as to reduce the workflow for the cutting of the later notes.(2) The first segmented notes are subjected to data frame framing, spectrum mapping and feature extraction according to the 12-average mapping principle.(3) Finally, the extracted note features are determined by the step length of note cutting and the threshold between each note. After the key data of note cutting is determined, the notes of each feature are cut twice, and the intelligent music note segmentation and recognition operation is completed.For musical notes, frequency, amplitude, intensity, pitch, duration and temperament are the main physical characteristics of musical notes. Each physical characteristic shows different features of musical notes, and these essential characteristics play a significant role in composing chords, rhythms and melodies of middle and high levels of music . The relThe step size of notes is determined according to the tone level, melody and sound intensity of melody notes. Firstly, the energy value of each frame in the whole speech signal should be taken as the research object and considered as a whole. Calculate the maximum and minimum values of short-time energy of all note frames, randomly divided into z equal parts, where z is randomly determined according to the melody length. According to the length of the melody. The step of a note is the quotient of the absolute value of the difference between the maximum and minimum value of the note data frame with z. The calculation formula is shown in When satisfied The music feature analysis adopts a dynamic time-warping (DTW) algorithm . It idenAssume that the grid points that the path passes through, in turn, are The final cumulative distance of the path is as follows:The minimum cumulative distance is the result of music signal feature recognition.Music emotion classification is the first step of emotion analysis. The Thayer and Hevner emotion models are commonly used in music . China’sBased on the note features extracted from “Audio feature segmentation”, music emotion analysis needs to consider the basic music information, rhythm changes and structural forms and divide the music features into note features and high-level features. Pitch, duration and intensity, as the essential components of music, are acoustic clues for music emotion cognition. The pitch change of music is represented by the mean square deviation of pitch, as shown in Duration is the author’s duration, and the expression method is shown in Dynamics is an important means for music to express emotions; different musical dynamics can cause different emotional experiences for listeners. The average strength and strength variation characteristics can be characterised by the average dynamics represented by the Melody reflects the organisational form of music in time and space, and it expresses the change of pitch trend. For this reason, melody direction is expressed, such as Rhythm is a phenomenon of alternating regular intensity in music. Different festivals give music different tension. According to the tension change of rhythmic movement, the density of pronunciation points directly reflects its basic state. The greater the density, the more muscular the tension. Sign the change of rhythm density, as shown in (1) After extracting the music features that can express emotion, the music emotion classifier is constructed. In this algorithm, the three-layer radial basis function neural network is used as the intelligent classifier, which has the best approximation and global approximation and is superior to the traditional BP network in function fitting. The algorithm flow is as follows: firstly, a record is randomly selected from the library, and similar records in the library are searched; In the results found by users, matching records are marked and fed back to the system, and the system marks them in the database; The system uses these marked records to train the classifier until the searched false emotion fragments are within a certain error range. In the process of correctly constructing the classifier, the emotional annotation of music in the library is realised. The construction algorithm of the classifier is as follo(1). The feature vector (2) A fragment (3) The trained classifier is used to classify the last marked fragments in the library and return them to the user. The user judges these results, draws them in the library if they are fragments with similar emotions, and jumps to Step 4 for execution. Otherwise, return to Step 2.(4) Return to Step 1, and circularly execute Step 1 to Step 4 until all student inputs are classified.corpus of the seed audio. By searching this corpus, the joint representation of any kind of audio can be obtained without repeated calculation, which provides support for subsequent research.This test adopts AMG1 608 data set  which coIn order to ensure the rationality and scientificity of this experiment, this article compares the methods used in literature  and diviFrom the analysis of The music signals of two different scenes in an online learning segment are shown in From In order to verify the effect of the classification model, a comparative experiment was conducted between the proposed classification model and the BP neural network-music-emotion classification model used in In The confusion matrix of models under different emotions is shown in It is evident from the initial experimental findings that the proposed method realises the rapid segmentation of musical note features because the technique in this article can segment musical notes according to the changing trend of their physical features. For a musical melody, the hierarchical filtering segmentation method is adopted to complete the operation, which effectively extracts the features of musical notes, shortens the recognition time, and provides a foundation for “student” emotional expression in online teaching. Music emotion recognition is the basis of “student” emotion recognition in online learning. With the continuous expansion of the online teaching scale and the daily explosive growth of massive data, online education platform needs to classify music in time. It can accurately recommend new music to users in need. The proposed musical emotional expression can obtain the emotional tendency of many songs. Technically, it can fuse multi-source information and improve the accuracy of multi-classification. In this article, when classifying music emotion, the emotional tags marked by user groups represent the users’ subjective experience, which is the users’ emotional perception of music after listening to music. The emotional recognition results are closer to the users’ personal feelings.Although the current interactive music platform Wolfie combines artificial intelligence technology with a music database to support the “student” musical performances , the leaIn this article, a music feature recognition system based on IoT technology is designed. The feature extraction analysis is carried out by analysing the physical characteristics of music notes, and the music signal feature recognition is realised, where musical emotional expression is then identified according to the results. Regarding feature extraction results, the time of audio feature recognition using this method is only 0.004 min, and the accuracy rate is higher than 95%. In addition, different languages have certain influences on learner pronunciation, which leads to a small degree of confusion between happiness and excitement, but a large degree of confusion with sad and quiet lyrics, In practical testing, it was observed that happiness was less likely to be confused with excitement but was more prone to be mistaken for sad and melancholic lyrics, which could potentially impede a teacher’s ability to effectively manage student emotions. To address this issue, future research will employ a fine-grained emotion analysis model to investigate the intersection between positive emotions, and to further refine “student” emotional classification for improved accuracy.10.7717/peerj-cs.1472/supp-1Supplemental Information 1Click here for additional data file.


Recently, in various fields, research into the path tracking of autonomous vehicles and automated guided vehicles has been conducted to improve worker safety, convenience, and work efficiency. For path tracking of various systems applied to autonomous driving technology, it is necessary to recognize the surrounding environment, determine technology accordingly, and develop control methods. Various sensors and artificial-intelligence-based perception methods have limitations in that they must learn a large amount of data. Therefore, a particle-filter-based path tracking algorithm using a monocular camera was used for the recognition of target RGB. The path tracking errors were calculated and a linear-quadratic-regulator-based desired steering angle were derived. The autonomous trucks were steered and driven using a pulse-width-modulation-based steering and driving motor. Based on an autonomous truck with a single steering and driving module, it was verified that the path tracking could be used in three evaluation scenarios. To compare the LQR-based path tracking control performance proposed in this paper, an elliptical path tracking scenario using a conventional sliding mode control with robust control performance was performed. The results show that the RMS of the lateral preview error of the SMC was approximately 18% larger than that of the LQR-based method. Recently, in various fields, such as logistics, agriculture, and transportation, autonomous driving technology has been used to increase the safety, convenience, and efficiency of workers. In particular, in logistics or factory industrial sites, robots and automated guided vehicles (AGVs) with autonomous driving technology are used to move and classify heavy goods. The AGV is responsible for loading goods, tracking workers, or driving to the target point. At work sites, early infrastructure was composed of magnetic, barcode, beacon, and color bands, and the surrounding environment was perceived using various sensors, such as cameras, lidar, and radar. Based on the surrounding environment information obtained through sensors, various control methods, such as artificial intelligence and optimization control, are used to determine and control the target path of the autonomous driving system. Systems with autonomous driving technology used in various fields require a control method to firmly recognize and decide the surrounding environment and a given path, and to track the path. To track the path of systems to which this autonomous driving technology is applied, various recognition, decision, and control methods are being researched and developed.Tang et al., proposed a path tracking controller that combines a kinematic model predictive controller and a proportional–integral–derivative (PID) controller to reduce the uncertainty of the model and compensate for the side slip in the high-speed driving situations of autonomous vehicles . Lee et For autonomous driving, paths and obstacles should be detected by measuring the surrounding environment and system information based on various sensors. A number of studies have used artificial intelligence methods to reduce the noise of sensors and detect paths. In addition, adaptive methods are used to reduce the uncertainty between the mathematical model and the actual system, such as autonomous vehicles and AGV.A particle filter generates weighted particles to approximate posterior probability density function. It can be employed for nonlinear, non-Gaussian noise systems and may not require mathematical models. Therefore, it is used to design controllers by estimating system parameters or states ,11,12. HLiang et al., proposed a decoupling and a side-slip phase-plane-based control method to improve the lateral stability of four-wheel independent steering and autonomous vehicles under high-speed conditions . Song etMethods of using yaw rates and adjusting weights were used to control the steering angle in path tracking. Most studies have controlled based on the lateral preview error and yaw angle error to converge the tracking path of an autonomous vehicle to the reference path. Normal MPC may have adverse affects on the control performance owing to the uncertainty between an actual system and a mathematical model. In addition, it has a large computational load. Therefore, methodologies that are robust and can reduce the computational load have been proposed for applications in real systems.In previous studies, it was confirmed that research into autonomous vehicles and AGVs is being conducted to increase worker safety, convenience, and work efficiency. To track the path of various systems by applying autonomous driving technology, the surrounding environment should be perceived and the technology must be decided and controlled accordingly. For the path tracking of an autonomous driving system, various sensors and artificial intelligence methods are used to detect paths and obstacles, and adaptive theory or artificial-intelligence-based path tracking control methods are used. However, artificial intelligence requires a large amount of training data and image processing for path and obstacles detection. It was recently confirmed that particle filters can be used in object tracking and path planning methods as an advantage that can be applied to nonlinear systems without mathematical models. Therefore, in this paper, we propose a particle filter that does not require image processing for path tracking using RGB information from the obtained image. In addition, we propose a path tracking control algorithm using a simplified error equation and monocular-camera-based path error derived from the particle filter.The remainder of this paper is organized as follows. In The particle filter algorithm for tracking the RGB of the target path comprises particle generation, particle update, likelihood calculation, and particle resampling. The RGB values are acquired from images within a monocular camera, and the positions and normally distributed velocity, with a mean of zero and a variance of 1, are randomly generated within the designed region of interest (ROI). The generated particles predict the movement position and velocity of the particles in the current state through the update process, as shown in Equation (1).As shown in Equation (1), the state variable is estimated at the current time (In Equation (1), Multinomial resampling methodology was used for the particle resampling process, and the maximum value of log likelihood was subtracted from the log likelihood derived from Equation (4) to make it all negative. The value of the difference was applied to the exponent of the natural constant to normalize the maximum value of likelihood to 1.The weight To derive the path error between the target path and current camera position based on the derived clustered point, it is necessary to convert the pixels in the camera image into actual distances. To convert to actual distance, 15 cm rulers were placed at 20 cm intervals, and the actual distance according to the pixels were measured through an experiment. As shown in In this study, we considered a clustered point as a tracking object and simplified it using a path tracking algorithm. Therefore, the lateral preview error and yaw angle error were derived based on pixels, as shown in Equations (9) and (10).To derive the desired steering angle, the error equation was derived as Equations (11) and (12) based on the Ackerman geometry model.In Equation (11), the yaw rate is considered to be zero, and in Equation (12), A is the system matrix, B is the input matrix, and the input is the desired steering angle. Equation (13) is a feedback control input for minimizing the error states, which can be substituted into Equation (12) and represented as Equation (14):Equation (15) is the cost function of the LQR, and the feedback gain matrix is derived as shown in Equation (17) by deriving The current steering angle is required to track the desired steering angle derived from the LQR. The current steering angle was derived using a map-based variable resistor (VR), as shown in Equations (19) and (20). The VR used had a voltage of 0 to 5 V, and by converting it into an analog signal of 0 to 1023, the current steering angle according to the voltage of VR within the designed minimum and maximum steering angle range was derived.The steering angle error is defined as shown in Equation (21) using the desired steering angle derived based on the LQR and the current steering angle derived based on the map. The current steering angle is driven based on the rotational direction and PWM of the pulse-signal-based step motor, as shown in Equations (22) and (23), to track the desired steering angle.When the steering error is positive, the PWM is applied to the step motor until the steering error converges to zero along with a counterclockwise pulse signal. Conversely, when the steering angle error has a negative sign, a clockwise pulse signal and PWM are applied to the step motor until the steering error converges to zero.When the particles deviate from the target path, the error between the RGB data of the clustered point and target RGB data increases. Driving and braking were determined using the Euclidean norm of the RGB data of the clustered point and the error of the target RGB data, as shown in Equation (24). As shown in Equation (25), the PWM-based in-wheel motor was driven with a driving flag of zero when it was located within the designed error threshold range; when the error exceeded the threshold, the driving flag was changed to 1 and a braking signal was applied to the in-wheel motor. In Equations (22), (23) and (25), The PC implemented an RGB data-based particle filter algorithm obtained from a monocular camera and measured the voltage of VR to derive the steering angle error for controlling the step motor and error of the RGB data for controlling the in-wheel motor using MATLAB software. The PWM and braking signals were applied to the in-wheel motor through MCU-1 according to the determined driving flag based on the error of RGB data, and the steering rotation direction flag derived according to the steering error was transmitted to MCU-2. MCU-2 was driven based on pulse signal rotation direction switching and PWM-based operation of the step motor according to the steering rotation direction flag received through MCU-1.The next section provides the performance evaluation results under various evaluation scenarios .The parameters such as target RGB, position, and velocity standard deviations of particles and the number of particles are sensitive parameters and were determined through trial and error methods. To check the effect of cycle loop time according to the number of particles , a total of 450 frames for the experiment were designed as total experiment frames. The next subsections highlight the performance evaluation results for the ellipse, S-curved, and lane-change scenarios. The LQR and sliding mode control (SMC)-based evaluation and comparison results are provided.The ellipse path tracking scenario had two 5 [m] straight sections and two semicircles with a diameter of 4 [m], as shown in The autonomous truck was driven for approximately 90.8 [s] as the real time taken for the ellipse path scenario, and 0.1745 [s] per sampling instance. The total length of the path of the scenario is approximately 22.5 [m], and when the total driving time is divided, it can be confirmed that the average velocity was approximately 0.248 [m/s].This section presents the S-curved path tracking scenario with two semicircles with a diameter of 5 [m], as shown in In The autonomous truck was driven for approximately 63.97 [s], which was the real time taken for the S-curved path scenario, and 0.1784 [s] per sampling instance. The total path of the scenario was approximately 15.7 [m], and when the total driving time was divided, it was observed that the average velocity was approximately 0.245 [m/s].This section describes the lane-change path tracking scenario in which a lane with a width of 3.5 [m] is changed by 5 [m], as shown in It was driven for approximately 45.89 [s] as the real time taken for the lane-change path scenario and 0.1767 [s] per sampling instance. The total path of the scenario is approximately 12.1 [m], and when the total driving time is divided, it can be seen that the average velocity is approximately 0.264 [m/s].To compare the LQR-based path tracking control performance proposed in this study with another controller, a conventional SMC of a robust controller was used to conduct performance evaluation under the elliptical path tracking scenario. The desired steering angle was derived by designing a sliding surface and a cost function based on the Lyapunov function using the derived path error-based simplified error dynamics. The conventional SMC has a chattering phenomenon if an unreasonable smoothing method is applied, so a sigmoid function was applied in this study. The SMC parameters were derived using a trial and error method to ensure reasonable path tracking performance and compare to the LQR-based evaluation results .Recently, research and commercialization for systems with autonomous driving technology has been conducted to improve worker safety, convenience, and efficiency in various fields, such as logistics warehouses, smart farms, and smart factories. The path tracking of these systems recognizes the surrounding environment using various methods, such as artificial intelligence based on various sensors or image processing. However, these methods require a lot of training data, the initial infrastructure configuration has a high cost, and the number of required sensors may increase. Using many sensors can increase failure probability and energy consumption. Particle filters have recently used in-object tracking and path planning methods as an advantage that can be applied to nonlinear systems without mathematical models. We focused on develop a path tracking algorithm using only target RGB without image processing and a large amount of learning data using a monocular camera based on a particle filter.In this study, we proposed a path tracking algorithm for autonomous trucks with a single steering and driving module. The target path was tracked using a monocular camera and the target RGB data based on a particle filter. Pixel-based path errors were derived by clustering the particles generated at the points most similar to the target RGB of the path in the ROI. The LQR-based desired steering angle was derived using the path tracking errors, and the map-based current steering angle was derived through voltage measurements using VR. To track the desired steering angle, the PWM and pulse signal were applied to the step motor so that the calculated current steering angle converges to the desired steering angle.The proposed path tracking method is an attempt to develop particle-filter-based target RGB tracking using a monocular camera.An autonomous truck with a single steering and driving module was developed, and its path tracking performance was evaluated.The following is a summary of the major contributions of this study:In this study, a constant PWM was applied to the in-wheel motor to drive the autonomous trucks at an average velocity of approximately 0.25 [m/s]. The performance evaluation was conducted based on the developed autonomous truck under four scenarios: ellipse, S-curved, and lane-change paths, and an SMC-based elliptical path. As can be seen in the performance evaluation, the chattering of the current steering angle can be confirmed by the hardware delay while generating a pulse signal to the step motor for the steering rotation flag generated based on the particle filter, and this limitation will be overcome through algorithm and hardware optimization in the future.The design parameters of the particle filter and LQR are sensitive and were derived through trial and error. It was confirmed that the cycle loop time also increased as the number of particles increased. In addition, to compare the LQR-based path tracking control performance proposed in this study, an elliptical path tracking scenario using conventional SMC was performed. As a result, the RMS of the lateral preview error of the SMC was approximately 18% larger than that of the LQR-based scenario. The RMS of the steering angle error between LQR and SMC was 1%, confirming that the steering angle error was similar. SMC parameters require changing in various curvature scenarios because the desired steering angle is derived according to the boundary value of disturbance, regardless of the magnitude of the path error. However, LQR has the advantage of being applicable in various scenarios if appropriate parameters are used because the desired steering angle is derived according to the magnitude of the path error. In addition, it has relatively few design parameters compared to SMC, and has the advantage of being able to compute optimal control inputs. However, LQR requires a mathematical model and has limitations in that uncertainty exists in the process of converting pixels of a camera image into real distance.Therefore, in the near future, we will focus on improving the path tracking algorithm that use only pixels from camera images by applying model-free control methods or various adaptive theories that do not require mathematical models. In addition, adding an object tracking mode indoors and outdoors using various sensors, such as GPS for outdoor driving, and IMU and lidar for obstacle detection, is considered for future work which will fuse various sensors to advance more robust and safe path tracking algorithms by recognizing the surrounding environments and evaluating the performance in path scenarios with various curvatures. Based on the algorithm proposed in this study, it is expected that a target path and the surrounding environment can be recognized in various fields using autonomous driving technologies for various purposes.


Accurate terrain mapping information is very important for foot landing planning and motion control in foot robots. Therefore, a terrain mapping method suitable for an indoor structured environment is proposed in this paper. Firstly, by constructing a terrain mapping framework and adding the estimation of the robot’s pose, the algorithm converts the distance sensor measurement results into terrain height information and maps them into the voxel grid, and effectively reducing the influence of pose uncertainty in a robot system. Secondly, the height information mapped into the voxel grid is downsampled to reduce information redundancy. Finally, a preemptive random sample consistency (preemptive RANSAC) algorithm is used to divide the plane from the height information of the environment and merge the voxel grid in the extracted plane to realize the adaptive resolution 2D voxel terrain mapping (ARVTM) in the structured environment. Experiments show that the proposed mapping algorithm reduces the error of terrain mapping by 62.7% and increases the speed of terrain mapping by 25.1%. The algorithm can effectively identify and extract plane features in a structured environment, reducing the complexity of terrain mapping information, and improving the speed of terrain mapping. Due to the structural characteristics of foot robots, the utilization of terrain information is crucial for foot landing planning and motion control. Foot robots are mainly utilized in industrial settings and structured terrains, like tall buildings. Therefore, establishing precise terrain mapping in these environments is essential for enhancing the motion efficiency of foot robots .For legged robots, perception sensors are crucial for estimating terrain during movement . However(1)Through the terrain mapping framework, the environmental point cloud undergoes transformation into height information, which is subsequently mapped onto a two-dimensional voxel grid. This height information is then downsampled to streamline the data structure and reduce redundancy.(2)The utilization of the preemptive RANSAC algorithm for plane extraction from the terrain height information within the voxel grid enables the estimation of parameters such as height and depth in structured environments.(3)To accommodate the characteristics of various planes, an adaptive resolution strategy is employed to adjust and merge the resolution of the voxel mesh within the extracted planes. The voxel meshes from different planes are then seamlessly integrated. A specialized data structure is utilized to efficiently store the final terrain mapping information.(4)An environment awareness system is constructed based on the Unitree Go1 platform. This system is deployed on an NUC10 embedded computer, and terrain mapping tests and evaluations are conducted in both simulation and real-world scenarios.To accurately and efficiently map indoor structured terrain and obtain terrain parameters, this paper introduces a novel adaptive two-dimensional voxel terrain mapping method. The impact of sensor noise and pose estimation uncertainty is mitigated through the mapping coordinate frame. Additionally, the random sampling consistency algorithm  is incorThe paper is organized as follows: Currently, the perception of the surrounding environment by mobile robots predominantly relies on depth cameras or Lidar sensors to model their environment. These sensors capture point cloud data from the environment, which, when combined with the robot’s pose information, is used to generate raster maps ,11,12, eElevation maps designed for legged robots usually involve mapping the height of the environment to create a flat grid of sheets. Fankhauser and colleagues  proposedIn voxelized occupancy diagrams, a two-dimensional height map corresponds to a three-dimensional occupancy grid diagram. Each voxel is assigned a binary label to store information about space occupancy, and the occupancy information of each label is updated through a Bayesian filter. These mapping computations typically demand significant memory resources but can be optimized by employing efficient memory representation methods like octrees ,24. AddiIn addressing the feature detection requirements for structured terrains like stairwells, existing research generally adopts methods to segment the stair plane from point cloud data, such as the RANSAC method for stair plane extraction . In the In comparison to these methods, our approach utilizes the preemptive RANSAC algorithm to extract planes in structured terrains based on the terrain mapping framework. Additionally, we use an adaptive voxel mesh to adjust the grid resolution in recognized planes to splice voxel meshes with different resolutions, ultimately completing the terrain mapping process. This method not only eliminates robot positioning drift and sensor noise but also effectively identifies and extracts structured planes, acquires terrain parameters, and ensures the accuracy and consistency of terrain mapping.In this paper, the terrain mapping algorithm primarily encompasses the construction of a terrain mapping coordinate frame, the design of a two-dimensional voxel mapping grid, the extraction of planes from height values, and the subsequent adaptive resolution adjustment. The specific algorithmic flow is depicted in Z axis of the world coordinate system, while the azimuth of the X and Y axes may change.With the robot as the center, the entire terrain mapping system comprises four coordinate systems. As illustrated in q denoting the rotation angle between the two frames. Considering the link between the sensor, the robot body, and the environmental point cloud information collected by the distance sensor, we transform this information into a terrain mapping framework using the robot’s own position and orientation:To process the height information within each terrain grid, we establish a two-dimensional voxel grid representation in which each voxel stores the height information of the terrain.(1)Height information and position information of the point cloud can be mapped from each point cloud, and the side length of the voxel grid is set to (2)Encoding the two-dimensional voxel grid, the spatial index (3)The terrain mapping height value obtained by point cloud transformation in each grid is traversed, the point cloud height mapping value in each grid is maximized, and the other point cloud height mapping values are removed.i, and In the original two-dimensional voxel terrain mapping system, the local normal vector (1)The height values in the three-voxel grid are randomly selected from the voxel grid. If the three-voxel grid is not collinear, the corresponding plane (2)Calculate the difference between the height in the remaining grid and the height of the plane, respectively, (3)d is selected. If An appropriate height threshold (4)N in this plane is the largest, it is identified as the best plane.Cycle the above three steps and iterate for K times. When In this paper, the preemptive RANSAC algorithm is used to extract the stair plane and estimate the stair parameters directly from the height information mapped into the voxel grid. The local normal vector and the height limit are added to the extraction process of the staircase plane to realize the extraction of a multi-level plane. The specific plane extraction process is as follows:According to process Assuming that When the probability of accepting the endovoxel grid is As shown in Formula (9), the inner and outer voxel grid is divided into:The number of iterations N is a crucial parameter that dictates when the random sampling consistency algorithm should conclude. Typically, termination depends on the estimated value N can be set as The termination threshold To extract multilevel planes, we have introduced height and local normal vector constraints. The height constraint involves dividing height values in the voxel grid into height ranges, extracting planes within each range, and iteratively adjusting the height values. The local normal vector constraint uses the terrain mapping surface’s normal vector to determine whether the local normal vector of the point cloud corresponding to the height value in the voxel grid is parallel or perpendicular to the terrain mapping surface’s normal vector. If the result is otherwise, the voxel grid containing the height value is excluded. The specific algorithm flow is illustrated in The increasing value of the height limit can be adjusted according to the environmental conditions, thus avoiding missing detection. After the optimal plane is extracted, the Angle between the local height normal vector Determine whether the included Angle In the preemptive RANSAC algorithm, plane extraction is mainly restricted by the conditions, and the goal is to extract the most suitable plane rather than the largest plane. The local normal vector restriction mainly excludes non-parallel and non-vertical planes from the ground plane to improve the accuracy and efficiency of plane extraction. Height restriction is mainly achieved by segmenting the height range in the voxel grid to achieve structured plane extraction, which can prevent the interference of environmental point clouds and other level planes, and make the staircase plane extraction more accurate.After extracting the plane in the structured environment, as described in This section validates our approach through a series of experiments conducted on both simulated and real robots, in comparison with Elevation Mapping and Voxblox. The experiments primarily assess the performance of the proposed mapping algorithm from three dimensions: terrain fitting accuracy, terrain mapping error, and map updating time.We simulated a structured environment in the Gazebo simulation environment, comprising elements like the ground plane, stairs, walls, and more. The parameters of these scene elements, such as the height and depth of the stairs, were adjustable to closely match real structured environments. The robot used in the simulation is based on the Unitree Go1 model, and it uses simulated Intel RealSense depth camera data, which aligns with our real-world experimental robot setup. The experimental environment is depicted in Adhering to international building codes, the height and depth of the stairs in the simulation experiment were configured to be 15 cm and 32 cm, respectively. The point cloud data in the simulated environment was derived from the camera’s point cloud information which was collected within one meter in front of the robot. During the experiment, the robot’s motion control and terrain mapping algorithm nodes were initiated separately. In the Gazebo simulation environment, the robot model’s motion was controlled using keyboard control nodes, allowing real-time retrieval of terrain mapping results.x-axis and y-axis denote the direction and height of the terrain, respectively. These three mapping algorithms were evaluated using the same map resolution of 0.001 m. As observed in To quantitatively compare the accuracy of various terrain mapping algorithms, this paper introduces a mapping error formula. This formula assesses mapping accuracy by contrasting the height value in the mapped voxel grid with the deviation in terrain height data from the environmental model. Let us assume there are n mapped voxel grids, each containing a height value To evaluate the algorithm’s mapping capabilities in various structured terrains, the experiment involved conducting error comparison experiments in different stair-like environments. As depicted in The experimental results shown in To quantitatively compare the terrain mapping speed of three different terrain mapping algorithms, we conducted an experiment using the same structured terrain point cloud data while monitoring the robot’s pose changes. The environmental data collected pertained to a structured stairwell with dimensions of 5 m in length and 3 m in width. Each of the three terrain mapping algorithms was employed to construct a complete map.The calculation times for maps with different resolutions in the same environment are presented in The real-world experiments were conducted using a custom-built four-legged robotic system equipped with a depth camera and a Lidar sensor. The robotic platform used was a Unitree GO1 EDU quadruped robot, featuring 12 servo motor joints. The rear of the robot was outfitted with a 16-line laser radar and an IMU attitude sensor for precise position and attitude estimation. The front of the robot was equipped with an Intel D435 depth camera, which captures point cloud data from the environment for distance measurements. The overall hardware configuration is depicted in The experiments involved comparing the performance of various terrain mapping methods in the same scene initially. Subsequently, a more representative scene, encompassing ground, stairs, and other structured terrains, was chosen to evaluate the effectiveness of the algorithm proposed in this paper. Furthermore, a preliminary experiment was conducted in a semi-outdoor setting.Furthermore, due to the inclusion of the plane extraction process, the algorithm’s capacity for recognizing terrains in outdoor sports scenes remains somewhat limited. As demonstrated in In this paper, we propose an efficient adaptive two-dimensional voxel terrain mapping for structured environments, which includes three key parts. Specifically, in the extraction process of environmental terrain height information, considering the impact of distance sensor noise and robot pose estimation uncertainty on height information estimation, the height information mapped to the voxel grid is downsampled to simplify the data structure and reduce the redundancy of information. This allows the robotic system to obtain the height information of the terrain more accurately, and effectively reduces the complexity of the data. Through plane extraction of terrain height information in a voxel grid using preemptive RANSAC, the estimation of height and depth of structured environment is realized. The resolution of the voxel mesh in the extracted plane is adjusted and merged by adopting the strategy of adaptive resolution, and the voxel mesh in different planes is spliced. To efficiently store the final terrain mapping information, custom data structures are also used. In this way, the details of different planes can be flexibly processed, and the information of the overall terrain can be effectively preserved. At the same time, the algorithm presented in this paper also demonstrates its limitations in unstructured terrain, which will be further addressed in subsequent research.


This article provides a critical analysis of the application of the advanced languagemodel, GPT-4, in generating surgical operative notes, with a focus on its use inophthalmology as presented by Waisberg et al. The discussion underscores theinherent complexity and specificity of operative notes, the issue of accountability, andthe potential data protection issues associated with the use of AI in healthcare. Theletter emphasizes the need for a more comprehensive understanding of thecomplexities involved in applying AI in healthcare and calls for a more nuanced andresponsible approach to the integration of AI in surgical documentation. Dear Editor,I am writing to express my concerns regarding the article "GPT-4 and Ophthalmology Operative Notes" by Waisberg et al. . The autWhile the potential of AI in healthcare is undeniable, the application of GPT-4 in this context is problematic for several reasons. Firstly, the authors seem to overlook the inherent complexity and specificity of operative notes. These notes are not merely generic descriptions of a surgical procedure; they are detailed accounts of a specific operation performed on a specific patient, taking into account the unique characteristics and circumstances of the case. A language model like GPT-4, no matter how advanced, cannot replicate the nuanced understanding and clinical judgment of a human surgeon.Secondly, the authors fail to address the issue of accountability. Operative notes are legal documents that carry significant weight in the context of patient care and medico-legal disputes. The responsibility for the accuracy and completeness of these notes lies with the operating surgeon. Delegating this responsibility to an AI model raises serious ethical and legal questions.Thirdly, the authors' assertion that GPT-4 can generate a "detailed note, with all the required components of a good ophthalmic note" is misleading. The example provided in the article is a generic description of a cataract surgery, lacking the specific details that would be expected in a real operative note. Furthermore, the authors do not provide any evidence of GPT-4's ability to accurately capture the unique aspects of individual cases, such as intraoperative complications or deviations from the standard procedure. Lastly, the authors' claim that GPT-4 can "drastically improve the time-consuming process of writing ophthalmic surgical notes" is unsubstantiated. While AI can certainly assist in automating certain tasks, the writing of operative notes requires a level of clinical understanding and decision-making that cannot be automated. The time saved by using GPT-4 to generate a generic operative note would likely be offset by the time required for the surgeon to review and correct the note.Moreover, the application of AI in healthcare raises significant data protection issues that the authors fail to address. Operative notes contain sensitive patient information, and the use of AI models like GPT-4 in processing this data poses potential risks to patient privacy. The authors do not discuss how these risks would be mitigated, nor do they consider the implications of data protection regulations such as the General Data Protection Regulation (GDPR) in the EU. This oversight highlights the need for a more comprehensive exploration of the complexities involved in applying AI in healthcare.In conclusion, while the application of AI in healthcare holds great promise, it is essential to approach this field with a clear understanding of the limitations of AI and the unique complexities of medical practice. The use of GPT-4 to generate surgical operative notes, as proposed by Waisberg et al., is in my view a misguided approach that overlooks these crucial considerations.Sincerely,Aaron Lawson McLean


In application, training data and test data collected via indoor positioning algorithms usually do not come from the same ideal conditions. Changes in various environmental conditions and signal drift can cause different probability distributions between the data sets. Existing positioning algorithms cannot guarantee stable accuracy when facing these issues, resulting in dramatic reduction and the infeasibility of the positioning accuracy of indoor location algorithms. Considering these restrictions, domain adaptation technology in transfer learning has proven to be a promising solution in past research in terms of solving the inconsistent probability distribution problems. However, most localization algorithms based on transfer learning do not perform well because they only learn a shallow representation feature, which can only slightly reduce the domain discrepancy. Based on the deep network and its strong feature extraction ability, it can learn more transferable features for domain adaptation and achieve better domain adaptation effects. A Deep Joint Mean Distribution Adaptation Network (DJMDAN) is proposed to align the global domain and relevant subdomain distributions of activations in multiple domain-specific layers across domains to achieve domain adaptation. The test results demonstrate that the performance of the proposed method outperforms the comparison algorithm in indoor positioning applications. With the dawn of the era of digital intelligence, the importance of accurately acquiring location information between devices and users to optimize device–person interaction has become paramount. Consequently, indoor localization has risen to be one of the most critical technological components, finding extensive applications across various areas such as smart energy management, smart home systems, and points of interest identification ,2,3. It Many localization methods based on Wi-Fi signals use Received Signal Strength Indication at the link layer as a data source. RSSI can only describe the overall superimposed result of the distorted signals received from different paths. Channel State Information can distinguish the specific conditions of signals propagating through different paths, so it is more stable than the RSSI on a timescale but has strong specificity over space . AlthougTo solve the domain shift problem, transfer learning has been enrolled in many solutions and proven to provide strong robustness and high accuracy. But in real applications, this method still does not perform well because they only learn shallow representation features to find a transformation form in a low-dimensional manifold to make the subspaces of the source and target domains similar after mapping, which can only slightly reduce the domain discrepancy ,13,14,15In this work, we propose a Deep Joint Mean Distribution Adaptation Network (DJMDAN) to align the global domain and relevant subdomain distributions of activations in multiple domain-specific layers across domains for domain adaptation. Our work can roughly achieve the same distribution after domain adaptation and retain the fine-grained information between different categories at the same time. The experiment results show that the proposed DJMDAN method can effectively mitigate the interference in the data distribution caused by domain shift and improve localization accuracy in dynamic environments.The main contributions of this work can be summarized as follows. First, we present an improved domain adaptation method, which performs global and local adaptation of the data domain at the same time under the premise of avoiding the loss of fine-grained information of the data as much as possible. Second, the deep TL method is introduced into indoor localization to reduce the overhead caused by resampling fingerprint points and solve the problem caused by the instability of CSI data due to time and environmental changes. Third, we implement a localization system based on the proposed method and prove the accuracy and robustness in complex and changeable indoor environments by evaluating the performance of this system in a real environment.The rest of this paper is structured as follows. Localization in dynamic environments can be considered a domain adaptation problem. When conducting positioning, usually, all features of the entire sample will be input into the model. This can cause feature values sensitive to environmental changes to vary significantly in distribution, thus rendering the model trained in the source domain inapplicable in the target domain. Therefore, to solve this problem, some researchers have proposed methods based on feature selection. The main idea of this kind of method is to reduce the impact of environmental changes on positioning by selecting features with strong robustness . In the early stage of positioning transfer learning, the team of Sinno Jialin Pan  proposedTo overcome the loss caused by shallow feature adaptation, researchers tried to apply deep learning in transfer learning and utilize the strong feature extraction ability of deep networks to achieve more transferable local features from data to optimize domain adaptation performance , and theThe method of deep domain adaptation mainly depends on matching and minimizing the difference between two domains by aligning the first-order or second-order moment of domain features. Therefore, the network can learn the domain-invariant features. In this process, solutions relying on characteristics of Maximum Mean discrepancy (MMD) ,25,26,27In fact, most deep transfer learning methods are used in the field of image, and there are few related studies applied in the field of indoor localization. The deep transfer learning method is introduced into indoor localization , and theMost of the existing deep domain adaptation algorithms only consider the global domain adaptation to solve the domain shift issue, that is, aligning the global data without considering the relationship between the two subdomains. Although this method mitigates the interference of domain shift on data distribution to a certain extent, the mapped data are often prone to confusion, making the data unable to be accurately classified. FitLoc  completeTo solve this problem, the proposed DJMDAN can adapt to the overall distribution difference between the two domains’ data and retain the fine-grained information between different categories of data at the same time. It effectively solves the confused alignment issue while ensuring the domain adaptation effect to achieve greater environmental adaptability and higher accuracy of indoor localization.The DJMDAN uses a deep neural network to improve domain adaptation performance by learning and updating the network by comparing the Multi-Kernel Maximum Mean Discrepancy (MK-MMD)  and LocaAs listed in Most of the existing domain adaptation solutions in deep transfer learning localization focus on adapting to the distribution differences of both domains from a global perspective. Although these methods effectively mitigate domain shift affection, the local geometric characteristics of the data are also ignored. They do not consider the relationship between the same category of data in the two domains, resulting in the loss of fine-grained information in the mapped data. It leads to the confused alignment problem of two domains, finally affecting the accuracy of the locating algorithm. To solve this problem, DJMDAN takes the MK-MMD and LMMD to achieve the adaptation to both global probability distribution and local probability distribution of two domains on the premise of avoiding fine-grained information loss as far as possible.The MMD metric was often used as a measurement method in previous deep transfer learning methods based on statistical moment matching. However, MMD only uses a single kernel function to calculate the distribution difference between the two domains after mapping, so it cannot dynamically obtain the optimal kernel function according to the difference of the output data when calculating the distribution difference of the multi-layer network output data. As a result, the accuracy and adaptability of previous depth transfer methods are limited. To this end, MK-MMD is introduced as a measure of the difference between marginal probability distributions. MK-MMD originated from Multi-Kernel Learning (MKL) , which iSince the kernel function of MK-MMD is composed of multiple different basis kernel functions, its kernel function Domain adaptation modules are considered to be added after each of the multiple fully connected layers to calculate the difference in the marginal probability distribution of the output, which can better reduce the difference in marginal distribution.We assume that the output of two domains after the The above adaptation method based on marginal probability distribution only solves the problem of the overall distribution alignment of the two domains from the perspective of global changes and ignores the fine-grained information between data categories in the mapping process, which leads to the loss of information in the mapped data. To further improve the final output accuracy, the difference in conditional probability distribution between the two domains still needs to be adapted by aligning the distribution of the subset data belonging to the same category in both domains. Therefore, we used LMMD rather than MMD as the measurement method to calculate the distribution difference in the subset data of different categories, respectively. The weighted sum of them is the final conditional distribution probability difference, as is shown in (5).In the above equation, To better mitigate the difference in the conditional distribution, it is also necessary to calculate the conditional probability distribution difference in the multi-layer domain adaptation layer. Assuming that the outputs of two domains in the To verify the localization effect of the DJMDAN algorithm in a dynamic environment, we conducted comparative experiments with existing localization algorithms based on transfer learning and traditional machine learning algorithms in real scenarios.2, and the actual photo and plan graph are shown in The length and width of the experimental area is about 7 × 7 MA commercial router, TP-Link WDR7660, with three transmit antennas, was selected as the signal transmission device. The signal-receiving device was a Lenovo ThinkPad X201i laptop computer with Ubuntu 12.0.4 system and an Intel 5300 network card. The whole localization scene was divided into 32 grid-fingerprint points with 1 M spacing. The placements of devices and fingerprint points are shown in Two criteria are mainly considered here to better evaluate the performance of localization algorithms.Average Error Distance (AED): The arithmetic mean of the random error of all measurements taken in the experiments; the calculation formula is as follows.Cumulative Distribution Functions (CDF): The probability distribution of the random variable X. In indoor localization, it is often used to reflect the proportion of the sample number in the overall sample number when the localization error value is less than a certain standard, and its expression is as follows:To evaluate the DJDMAN algorithm, we compare several algorithms mainly used in this field, such as KNN, TCA, 1D-CNN, DAN and DeepCORAL (which only uses the covariance of the fully connected layer output two domain data for the domain adaptation depth transfer method) .Among them, 1DCNN is a deep learning algorithm without a domain adaptation module. Since the deep transfer learning method does not have a fixed basic network structure, the 1DCNN with three convolution-pooling layers and three fully connected layers is used as the basic network. 1DCNN is added as a benchmark algorithm to explore the influence of the domain adaptation method on the indoor localization effect. DAN is a depth transfer method that only uses the marginal probability distribution difference of data output by a fully connected layer for calculation, while DeepCORAL only uses the covariance.For traditional machine learning methods, such as KNN and TCA, the K is set to 1. For deep learning methods, all 1D convolutional neural networks are used as their basic networks, and their parameters are shown in The DJMDAN uses four basic kernels, which are the linear kernel, polynomial kernel, Gaussian kernel and Laplacian kernel. As pointed out in ,42, the To verify the effectiveness of DJMDAN with the change of time conditions, we collected CSI data within 30 days in the experimental environment for comparison. The data on the first day were collected as the fingerprint database data, that is, the source domain data in transfer learning. The data collected in the remaining days were used as the test data to represent the target domain data in transfer learning. From the results, due to the strong feature extraction ability of deep networks, all deep learning methods maintain lower localization error in the early stage of the experiment. However, with the passage of time conditions, the domain shift level of CSI signals collected at different times gradually increases, and the localization error of the algorithm with a one-dimensional convolutional neural network begins to be gradually higher than that of other transfer learning methods and even worse than that of traditional machine learning methods. Although the localization error of the algorithm based on deep transfer learning increases under the condition of time change, its localization accuracy is still better than algorithms based on traditional transfer learning methods.All deep learning methods use the same basic network, so they are consistent in terms of model scale. Among them, the proposed DJMDAN achieves a more accurate localization effect than two classical deep domain adaptation algorithms in the experiment. This is because DAN only considers the marginal probability distribution difference of two domains as the loss function when performing deep domain adaptation, which ignores the important factor of the conditional probability distribution difference. However, DeepCORAL only uses the covariance of two domains as the loss function, and the domain adaptation ability is also limited.The DJMDAN performs domain adaptation operation on the multi-layer fully connected layer from the perspective of the joint probability distribution difference, which is more effective in mitigating the domain shift phenomenon of the two domains. As a result, better localization results are achieved.The cumulative distribution function diagram can better show the overall distribution of localization errors. As shown in To examine the effectiveness of DJMDAN under other environmental conditions, we conducted experiments on three cases, which are door and window switches in the laboratory, the position of indoor lockers, and the condition changes of both changes at the same time.As shown in The test error in According to the experimental results, it can be seen that the proposed DJNDAN can obtain a better domain adaptation effect than the traditional transfer learning methods and achieves greater indoor locating accuracy because of the abstract features extracted using a deep one-dimensional convolutional neural network. In addition, DJMDAN considers the overall domain adaptation and the local perspective of the data simultaneously so that the fine-grained information of each category is retained in the domain adaptation process. Therefore, DJMDAN can maintain better robustness and obtain higher localization accuracy to resist environmental changes.In this paper, a deep domain adaptation localization approach for Wi-Fi CSI signals using transfer learning has been proposed. It can maintain data domain adaptation, both global and local, by adapting the marginal and conditional probability distribution of two-domain data. The proposed method is an update based on DAN, and experiments have shown that our method outperforms models of the same scale that do not use transfer learning. It also surpasses DAN and other transfer learning methods based on MMD metrics and methods using CORAL metrics. Experimental results show that it can effectively solve the domain shift problem and maintain a relatively ideal localization accuracy in complex and changing indoor environments.Moving forward, there are two potential areas for further exploration and improvement. Firstly, the current domain adaptation operation is still based on the overall perspective of data, and we could consider introducing some metric functions based on the internal geometric characteristics of the data for domain adaptation work. Secondly, we are currently using a relatively basic one-dimensional convolutional neural network structure. It would be beneficial to select deeper network models that are more suitable for CSI signals as the basic model for deep domain adaptation localization algorithms.


Diffuse reflectance spectroscopy (DRS) has proven to be a powerful, reliable, and non-invasive optical method for characterizing a specimen. Nevertheless, these methods are based on a rudimentary interpretation of the spectral response and can be irrelevant to understanding 3D structures. In this work, we proposed adding optical modalities into a customized handheld probe head in order to increase the number of parameters in DRS acquired from the light/matter interaction. It consists of (1) placing the sample in a reflectance manual rotation stage to collect spectral backscattered angularly resolved light and (2) illuminating it with two sequential linear polarization orientations. We demonstrate that this innovative approach leads to a compact instrument, capable of performing fast polarization-resolved spectroscopic analysis. Due to the significant amount of data available with this technique in a short time, we observe sensitive quantitative discrimination between two types of biological tissue provided by a raw rabbit leg. We believe that this technique can pave the way for rapid meat quality check or biomedical diagnosis of pathological tissues in situ at an early stage. Technological assistance for medical diagnosis became an active field of research in the last decade and consisted in investigating tissues and cells in a non-invasive way to enhance the survival rates for cancer patients . ParticuFor decades, numerous spectroscopic works have studied the scattering and absorption properties in in vivo tissues ,3. PractIn this work, we develop a multiparametric DRS setup in the near-IR (NIR) for collecting an optimal set of information from the light backscattering from the tissue surface. More specifically, we have developed an innovative approach based on (1) the acquisition of the polarization-resolved spectra based on the geometrical scattering angle that has never been measured for biological tissues, and (2) an easy-to-handle 3D representation of the variation of the DRS spectra through four distinct physical parameters without requiring any deep knowledge from the operator or computational performances. In this approach, we demonstrate our proficiency in measuring the spectral changes, sensitivity to the molecular content of different tissues in the function of the scattering angle, and the polarization of the light for capturing the geometrical local orientation available from a raw rabbit leg. Here, the near-IR is used in order to opt for a low light dose in the therapeutic window and investigating without damaging the living organisms . FinallyThe scattering sample is a starch granule extracted from raw potatoes. Four potatoes were peeled, shredded into small pieces, and mixed in a container with lukewarm distilled water. The melted liquid was extracted separately from the residual potato pieces through a filter several times. After one hour of decantation, the wet powder was separated from the water and was left to stand for 24 h to obtain a dry, purified starch powder.The biological specimen of interest for this work was a rabbit leg, known to present strong optical contrast, from the muscle to the tendon regions. The tendon tissue was harvested from a raw rabbit leg purchased from a butcher shop and was cleaned from other connective tissue. The muscular tissue was rinsed several times using an ethanol solution. The thickness of the muscle section is estimated to be around 1 mm down to 0.5 mm. The sections were fixed between a microscope slide and a coverslip, then imaged. In the final application, a raw piece of rabbit leg was directly placed in the sample manual rotation holder.The single-fiber reflectance system is composed of a 1 W tungsten-halogen white-light source , a portable NIR spectrometer between 900 and 1600 nm  with an integration time of 3.8 ms–10 s, and a single-fiber inox optical probe enclosing fused multimode fibers  able to both illuminate and collect the light. The head-probe is formed by five optical bundles, containing around 100 active fibers each, and distributed between two anti-symmetrical pairs of distal bundles for emission and a single central bundle for collection as presented In this work, only one distal bundle is connected to the lamp and the central bundle sends the light to the spectrometer as presented in r), the linear dichroism (LD), the degree of linear polarization (DOLP), and the grating factor (G) through the entire spectral range defined asThe fibers have a numerical aperture (NA) of 0.39 allowing for a collection of approximately 80° in scattering angle. In front of the central collection bundle of the probe head, a custom-built rotational mechanical polarization module is placed. It is composed of a linear NIR polarizer film  in a manual metallic holder. A stick is attached to the holder to manually rotate the polarizer between the exact 0° and 90° angles. Thus, the collected scattered light is projected into two orthogonal polarization states, giving the spectral intensities noted r  dedicated to wide-field phase contrast and polarization-resolved microscope. The imaging device is a Sony Tri-CCD 736 × 574 pixels camera , and the images are captured and stored using Sarfusoft software .All the spectra are recorded following the block diagram The characterization of starch granules under stress or thermal conditions is an emerging research topic for investigating food quality . In ordeThe extracted starch granules are used in this work to illustrate the capability of our technique to measure the polarization optical response of molecules in a turbid medium. The sample is presented under normal and polarization-resolved white source illumination, i.e., crossed polarization, as seen in As expected, the granules under polarized light in As a next step, the starch granules are placed under white source illumination through the optical probe-head providing the polarization spectra obtained from the calibration procedure detailed using Equation , leadingr and LD are shown in the bottom row of LD scattering pattern for chiral molecules exhibits two antisymmetric amplitudes related to forward emission, and is consistent with the Kramers–Kronig dispersion relation [r function is almost negative and decreases drastically at a high scattering angle, caused by the specular reflection on the plane surface. The LD function is symmetrical with two identical absolute magnitudes in the positive and negative signs. Usually, this pattern is well-observed for the visible wavelength range and LD becomes negligible in the NIR. However, the curve is still Kramers–Kronig-consistent in the function of the angle due to the conservation of the averaged macroscopic light emission that implies no change in the complex dielectric function. In order to simplify, DOLP and G are not presented for this sample, are assumed to be almost inconsequential for such molecules, and exhibit a preferential circular polarized light [The raw data are reported in the top row of relation . The r fed light . In summIn this work, the application of the optical technique reported in the The raw rabbit leg used in this study is shown in To correlate the optical fingerprint through the different parameters provided by this instrument with the tissue type under illumination, we selected three ROIs for each area. In the blue plot, we can see that the single muscular tissues exhibit a stronger specular behavior compared to the connective/muscular tissues. This could be explained by the Mie theory which mentions an emission diagram linked to the type with a relative refractive index and the size of the scatters . Thus, tMore precisely, two typical areas of the specimen have been selected for imaging under widefield microscopy without and with polarized light, as shown in As for the previous starch sample, we propose in r and DOLP, but they differ between both regions. The reason for this lies in the mathematical definition of these two parameters, which only differ by a factor of two in the intensity ratio [LD is becoming a zero parameter for both tissues but exhibits differences at higher scattering angles. Indeed, since LD is sensitive to averaged molecular order, at this NIR wavelength range, the two bulky tissues appear to be too similar, and cannot produce any significant dichroism difference in the forward direction. The r and G parameters exhibit the highest difference spelled out by their extreme sensitivity to the macroscopic biological organization. Since incoherent light interacts more efficiently with weak-ordered structures such as raw thick tissues, it provides enough quantitative information for differentiating both tissues [Comparing the four parameters presented in the ty ratio . LD is b tissues .To look more closely at the statistical dispersion of the parameter amplitudes and understand the influence of the 3D illumination location, The statistical distribution of the averaged magnitudes of the four parameters from LD and DOLP in which the discrimination is greater. This is important because any weak modification in the light amplitude would be hidden in a small signal-to-noise ratio (SNR) for a wide scattering angle range. Thus, we can conclude that for such measurement, the most optimal contrast between different biological tissues may be obtained from the collected light at more than 45° related to the illumination direction.From In this work, we have implemented a multiparametric approach for characterizing raw biological specimens in a remote in situ mode. The instrument prototype is based on a collection of spectra through a customized optical fiber, providing direct, easily handled, and low-cost measurements. We have proposed to measure the optical fingerprint of well-known anisotropic chiral molecules placed on a microscope slide. This preparation suffers from inhibiting the 3D organization contributions but offers the advantage of presenting a dataset in adequation with the literature. In parallel, we have investigated the optical fingerprint of a raw rabbit leg for quantifying the difference between two types of tissues in the angular and polarization-resolved process. In summary, we showed that the Lambertian-type reflection from the connective/muscular tissues is produced by multiple scattering with numerous different molecular species, which can be considered as a bulk region. Thus, this is expressed through small changes of the associated four parameters over the scattering angles. On the contrary, tendon tissues are composed of fewer different species and strongly oriented fibers; thus, the specular reflection expresses a higher angular variation of the multiparametric approach. Furthermore, having access to all these parameters simultaneously allowed for a simpler analysis of how the illuminated sample is organized and discriminating different tissues in situ without preparation of the sample. In future work, we propose an upgrade of the technique to speed up the measurement and simplify the multimodal use of the instrument. First, the collected linear polarizer will be placed in a small motorized rotation stage, synchronized with the acquisition of the spectra by polarization states. In parallel, the sample holder could be synchronized with a stepper motor driver. Second, we propose automating the spectra acquisition without moving parts for measuring clinical tissue to put forward a scoring approach and classification in medical diagnosis.


In the paper, the effect of spontaneous Brillouin scattering (SpBS) is analyzed as a noise source in distributed acoustic sensors (DAS). The intensity of the SpBS wave fluctuates over time, and these fluctuations increase the noise power in DAS. Based on experimental data, the probability density function (PDF) of the spectrally selected SpBS Stokes wave intensity is negative exponential, which corresponds to the known theoretical conception. Based on this statement, an estimation of the average noise power induced by the SpBS wave is given. This noise power equals the square of the average power of the SpBS Stokes wave, which in turn is approximately 18 dB lower than the Rayleigh backscattering power. The noise composition in DAS is determined for two configurations, the first for the initial backscattering spectrum and the second for the spectrum in which the SpBS Stokes and anti-Stokes waves are rejected. It is established that in the analyzed particular case, the SpBS noise power is dominant and exceeds the powers of the thermal, shot, and phase noises in DAS. Accordingly, by rejecting the SpBS waves at the photodetector input, it is possible to reduce the noise power in DAS. In our case, this rejection is carried out by an asymmetric Mach-Zehnder interferometer (MZI). The rejection of the SpBS wave is most relevant for broadband photodetectors, which are associated with the use of short probing pulses to achieve short gauge lengths in DAS. Phase-sensitive optical time domain reflectometer (phase-OTDR) is a promising and modern method for the simultaneous detection of spatially distributed dynamic events. The phase-OTDR sensing element is usually a widely available single-mode optical fiber. Fiber optic cable is placed in an elastic medium along an extended object and perceives vibrations with high sensitivity provided by the interferometric measurement principle. For this reason, phase-OTDR is commonly referred to as a distributed vibration sensor (DVS) or a distributed acoustic sensor (DAS). In the last decade, DAS has been widely used in various fields, from perimeter or pipeline security ,2 and trThe DAS is usually characterized by strain sensitivity (or strain resolution), which determines the minimum possible detectable strain of a fiber under test (FUT) in the presence of noise ,13,14. Ament. In , the usequencies  resultedpolation ,17,18 alThe strain sensitivity of DAS is limited by various types of noise arising in the receiving part of the device; however, despite the progress in increasing the sensitivity, the issue of noise composition has not yet been given enough attention. Identifying the dominant noise source will allow appropriate measures to be taken to increase the DAS strain sensitivity. To date, the main sources of noise in DAS are considered to be noise caused by phase fluctuations of the laser source , as well as the shot and thermal noise of the receiving circuit. In this paper, it is shown that there is another fundamental process that leads to an increase in noise power in DAS, namely noise due to spontaneous Brillouin scattering (SpBS). Identifying the dominant noise source will allow appropriate measures to be taken to increase the DAS strain sensitivity.The phase noise in DAS is investigated in . Due to Other noise sources limiting the DAS strain sensitivity are determined by the design specifics. In the case of a direct detection scheme, additional noises are shot noise and thermal noise of the photodetector and preamplifier ,28,29,30Here, we will focus on the conventional DAS architecture without an optical preamplifier and with direct detection, which has proven to be reliable and highly sensitive . We willSpBS is initiated by thermally excited spontaneous phonons in fiber ,36,37,38The intensity of the SpBS Stokes wave grows exponentially from the initial SpBS wave over the length of the probing pulse. For the pulsed probe signal under steady-state conditions where the pulse width itten as ,34:(1)Ptfficient ,37, thisThe noise sources in DAS were analyzed using the experimental setup shown in The Koheras BASIC X15 laser module from NKT photonics  with 1.3 kHz linewidth was used as a coherent light source. The light was intensity modulated by a semiconductor optical amplifier (SOA) from Finisar . Further pulse was amplified by an erbium-doped fiber amplifier (EDFA) and fed into a 1 km long FUT  through an optical dense wavelength division multiplexing (DWDM) filter with a bandwidth of 100 GHz and an optical circulator. The backscattered light that passed through the same circulator was filtered by a temperature-tunable Mach-Zehnder interferometer-based (MZI) filter with a 10 mm optical path difference corresponding to a free spectral range of about 20 GHz. Through temperature adjustment, either Rayleigh or both Brillouin waves (Stokes and anti-Stokes) can be rejected. The contrast of the MSI filter was estimated at about 27 dB. In order to select Stokes or anti-Stokes waves separately, the MZI filter was additionally supplemented with a tunable 50 GHz channel spacing filter from DiCon Fiberoptics . The filter had a bandwidth of 170 pm at the 3 dB level. After filtering, the backscattered light was further attenuated by a variable optical attenuator (VOA) and detected by a photodetector (PD) with a bandwidth of 50 MHz. The detected signal was digitized by a 12-bit analog-to-digital converter (ADC) at a sampling rate of 100 Ms/s.The SpBS process can be considered quantum mechanically as an interaction between an incident light photon and an acoustic phonon. In the case when a new phonon is created, the energy of the original light photon is downshifted, resulting in a photon with a reduced frequency, which is called the Stokes component of SpBS; in the case when an acoustic phonon is absorbed by a photon of the incident light, a photon with increased energy and increased frequency appears, which is called the anti-Stokes component of SpBS.In the pure SpBS approximation, the Stokes and anti-Stokes wave intensities depend on the absolute temperature atistics . HoweverThe theoretical consideration of the interaction between the pump wave and Stokes wave is made in ,36,37,38ctrum is :(3)ΔωFWHThe total intensity of the Stokes wave in this approximation (spectrum :(4)IS(0)In the case when The expressions (2) and (4) also describe the spectral density and total intensity of the SpBS Stokes wave when the pump wave is pulsed, provided that the steady-state condition is met, i.e., the pulse width is much longer than the phonon lifetime This value can be compared with the reflectivity of Rayleigh backscattering  with thefraction , where NThe SpBS process is initiated by fluctuating thermal noise, and thus, the intensity of the Stokes wave also fluctuates in time ,36,39,40icted in ,39,40. TThe observed Brillouin OTDR trace fluctuates rapidly in time at each point along the fiber; these intensity fluctuations are incoherently added to the Rayleigh OTDR trace and can be completely considered noise.The property of the exponential PDF is that its variance is equal to the square of the mean value. Thus, the average Stokes-induced intensity noise power is determined by the square of the mean Stokes intensity:The anti-Stokes wave is not correlated with the Stokes wave and has approximately the same power as Stokes in the case of low gain This total intensity noise power due to Brillouin scattering power can be compared with other noise sources mentioned in the introduction. According to ,25, the ulse DAS  with comIt is important to note that although the ratio of Rayleigh and SpBS reflectivities is approximately equal to 18 dB (6), (7) the ratio of the width of the rectangular probe pulse and coherence time, i.e., the first ratio in the right-hand side of (11), can be low, and the two powers of noise intensity fluctuations In order to experimentally compare different noise sources, we measured the averaged noise power of intensity fluctuations at the DAS output, shown in Taking into account the above considerations, the terms of the polynomial with coefficients a, b, and c can be interpreted as follows: the term 2] since the units of measurement of the fluctuating value are optical watts [W]. It can be seen that thermal noise components shown by diamonds (blue and red) coincide in both cases. The same is true for the shot noise components shown by asterisks (blue and red). The square components shown by circles (blue and red) are different because the blue curve includes noise caused by Brillouin scattering, while the red curve does not. An estimate of the noise caused by Brillouin scattering can be obtained by subtracting the red curve from the blue curve; the result is shown in Thus, Brillouin scattering rejection in front of the photodetector helps to reduce DAS noise. This statement is most relevant in the case of using short probing pulses , where a broadband PD with a bandwidth of the order of 30 MHz is required.Based on the data obtained, it is possible to estimate the expected increase in strain sensitivity using the results presented in . The strIn addition to spectral filtering, the relative effect of Brillouin scattering-induced noise can be reduced by using single-mode optical fiber with ultra-weak fiber Bragg gratings . With a The DAS noise composition has been experimentally studied for a specific case of a short (1000 m) FUT. This case is typical for engineering geology. It is shown that along with the known noise sources, namely thermal, shot, and phase noise, Brillouin backscattering-induced noise makes a significant contribution. The noise level is theoretically estimated. It is shown experimentally that the intensity of the Brillouin waves fluctuates, and the probability distribution of these fluctuations is described by a negative exponential PDF. The resulting average noise power of these fluctuations is equal to the square of the average Brillouin intensity, which is about 18 dB lower than the Rayleigh backscattering power of Stokes and anti-Stokes waves. In our case, the noise from Brillouin scattering is dominant. The reduction of the overall noise level by rejecting the Brillouin components of the scattered field using an asymmetric MZI is demonstrated. Such filtering is relevant for small gauge lengths and would increase the DAS strain sensitivity.