A common problem for healthcare providers is accurately tracking patients’ adherence to medication and providing real-time feedback on the management of their medication regimen. This is a particular problem for eye drop medications, as the current commercially available monitors focus on measuring adherence to pills, and not to eye drops. This work presents an intelligent bottle sleeve that slides onto a prescription eye drop medication bottle. The intelligent sleeve is capable of detecting eye drop use, measuring fluid level, and sending use information to a healthcare team to facilitate intervention. The electronics embedded into the sleeve measure fluid level, dropper orientation, the state of the dropper top (on/off), and rates of angular motion during an application. The sleeve was tested with ten patients (age ≥65) and successfully identified and timestamped 94% of use events. On-board processing enabled event detection and the measurement of fluid levels at a 0.4 mL resolution. These data were communicated to the healthcare team using Bluetooth and Wi-Fi in real-time, enabling rapid feedback to the subject. The healthcare team can therefore monitor a log of medication use behavior to make informed decisions on treatment or support for the patient. Glaucoma is the leading cause of irreversible blindness worldwide and the third leading cause of irreversible blindness in the U.S. ,2. AlthoThere are currently no standard methods for tracking medication adherence ,17; howeAdherence monitoring systems designed specifically for liquid medication are lacking. One of the few approaches in the literature or market place for tracking eye drop use, a very common liquid medication, was developed in Europe, where eye drops are transferred from the prescription bottle into a custom container with sensors to quantify use . These sensors include an inertial measurement unit (IMU) for position and shaking detection, Global Positioning System (GPS) for location tracking, and wireless capabilities to communicate with a smartphone. However, this approach requires the use of the company’s own custom eye drop medication bottles to latch to a system that contains the embedded sensors. As a result, this product is not currently approved for use in the United States because this system alters the dispensation of the medication. To receive approval, the Nemera system would require a full set of Food and Drug Administration (FDA) drug trials. To mitigate the extensive time and costs associated with FDA drug trials, a monitoring system capable of detecting and quantifying eye drop medication use without altering the original medication packaging must be developed. To address this gap, this work describes the design and fabrication of a portable sensor system that surrounds an approved eye drop medication package for the adherence monitoring of liquid medication. The sensors reliably detect medication use, measure the level of the fluid-based medication, and transmit that information to the healthcare team rapidly. The ability to accurately monitor medication frequency of use and the amount of medication in the package will give clinicians important information about patient adherence in real-time, enabling the opportunity to provide timely feedback to improve adherence.This work used an integrated approach, combining embedded sensors, real-time state classification, and communication to and from the embedded system to quantify adherence and fluid level for glaucoma medication . A custoThe sleeve was designed as a low power, portable Internet of Things (IoT) system for day-scale use . The lowTo allow for easy size adjustments for different bottle shapes and sizes, the mechanical components of the sleeve were 3D printed. The sleeve  consistsTwo reed switches and magnets embedded in the 3D printed cap detected the state of the bottle top. An inertial measurement unit  with a 3-axis accelerometer, gyroscope, and magnetometer provided information about the bottle orientation . To compTwo printed circuit boards (PCBs), each with the same area as the footprint of the bottle, were fabricated for the electronics . The topThe sleeve works in tandem with a commercially available Bluetooth-enabled device for data storage and communication of events and measurements. The sleeve system operates in the following manner: (1) power on, (2) wait in an idle state, polling for a BLE connection every 40 milliseconds until 30 s have passed, and (3) time out and wait 30 s before trying again. When the reed switches data readings indicate that the cap is off, an internal timer is activated and the processor polls for sensor data. The sensor data are used to determine if a use event has occurred. Additionally, the system queries the IMU every 30 min to determine if the bottle is stationary and upright, in which case, the fluid level is measured using the capacitive sensor. When the sleeve is connected to the Bluetooth enabled device, data are sent via BLE to a custom application on the communication device that logs and timestamps the received data. For the initial testing, an iOS-specific application was developed in Swift 3.0 that both receives and redirects data to a web server. The application was developed on top of the Core Bluetooth Module library, which allows for easy auto connect in background mode, freeing the user from direct interaction with the app. If the bottle is not connected to the peripheral device, all data are stored in the internal memory until a connection can be made.An offline rule-based algorithm was used to verify that features from the sensor data could be used to determine use events. The three chosen rules for the offline event detection algorithm were: (1) the cap state was off, (2) the orientation (based off the IMU data) was between 100° and 260° with respect to the vertical axis along the height of the bottle, and (3) there was a positive increase in the capacitance data for at least four consecutive data points (signifying force application).For the system to be effective, it is crucial to reliably determine when medication has been used by the patient online. To identify a use event automatically, sensor data were used to identify and train a classification algorithm to run, in real-time, on the microprocessor. Sensor data gathered when the sleeve assembly was dispensing liquid were then used to create features using descriptive statistics  polling for a BLE connection, (2) in ‘idle’ state waiting for cap removal, and (3) in ‘active’ state running all sensor components. The measured power consumption was used to estimate the time until recharging was required. Walk around the room with the instrumented system in a pocket/purse for one minute.Remove the bottle cap, dispense eye drops , then place the cap back on the bottle. Repeat five times.Remove the bottle cap, place the bottle back on the table without dispensing any fluid, then place the cap back on the bottle. Repeat five times.Shake the bottle for five seconds (with the cap still on). Repeat five times.Remove the bottle cap, simulate the motion used to dispense eye drops, but do not dispense fluid, then place the cap back on the bottle. Repeat five times.Remove the bottle cap, dispense eye drops in the same fashion as Step 3, but with the participant reclining, then place the cap back on the bottle. Repeat five times.To test the bottle electronics and detection algorithms, ten participants aged 65 or above were recruited to perform a series of tasks in a controlled experimental environment. All protocols were approved by the Institutional Review Board at the University of Michigan (IRB # HUM00162357). After the study coordinator obtained written informed consent, each participant received a bottle and performed specific tasks to simulate how the bottle might be used in a real-world scenario. A Wi-Fi enabled iPod Touch was placed in the center of the room to collect and store data transmitted from the system, and video of the participants was collected as they performed six tasks: The ten subjects each used the eyedropper ten times (fluid was dispensed five times each in tasks 2 and 4), resulting in 100 use events. Additionally, the cap was also removed from the bottle without dispensing medication a total of 100 times (tasks 3 and 5). Data were synchronized with the video recordings and labeled by a trained observer. The labeled data were used to train the detection algorithms and evaluate how features from the sensors contributed to correct event detection , dispensed medication hourly, and logged a time stamp of use for comparison with the algorithm’s results. The eyedropper was programmed to send capacitance data every 30 min when the bottle maintained an upright position, which was verified through the accelerometer. An iPod Touch received data from the sleeve, and the participant came into range of the receiver every three hours. At the end of the day, data from the iPod Touch were exported to an Excel Spreadsheet, where the results from the on-board detection algorithm were compared to the participants’ log. p-values, rather than each of the individual comparisons, were reported. A non-parametric test for each pairwise comparison of the experimental volume levels was used for the experimental characterization of the capacitance sensor. The Mann–Whitney U-test was selected because it is appropriate for small sample sizes (five replicated experiments). Multiple comparisons were not used as each pairwise-comparison was assessed separately. The range of comparison Precision, Recall (true positive rate), and F-measure (F1 Score) metrics were used to compare the multiple machine learning algorithms and machine learning with the rule-based algorithm. These metrics are calculated as follows:C representing the capacitance reading and V representing the fluid volume. The regression fit the data well with an R2 value of 0.998. Results from the high-resolution fluid test . However, fluid changes of 0.4 mL resulted in statistically significant differences . The error bars on the graph represent the sample standard deviation for the five repetitions at each fluid level. The results of the fluid level testing of the capacitive sensor are presented in uid test  indicateThe results of the range test indicate that, on average, BLE was able to communicate up to 30 feet before the connection between the instrumented bottle and iPod Touch was lost. z-axis acceleration from a representative subject trial is presented in z-axis vertically along the height of the sleeve in the downward direction. For example, when the bottle was in the upright position, the z-axis acceleration was approximately 9.8 m/s2. A trained observer reviewed the video and classified the six tasks. There were clear differences between the ‘everyday actions’ such as walking or shaking, and the use events. Sensor data collected during the simulated eye drop use events (no fluid dispensed) showed similar features compared to the real use events, but there were clear differences in the capacitance data. x-axis and z-axis accelerometer data flipped and there were minor spikes in the gyroscope data.Data from each sensor were successfully logged for the ten participants. Features from the sensor measurements demonstrated clear differences between the six tasks performed in the protocol. The The comparison of machine learning algorithms when classifying use events is shown in Offline and online use event algorithms were built from the subject trial data. The online event detection algorithm parsed data into windows based on the status of the cap, created features from the sensor data, and passed those data into a machine learning procedure, while the offline event algorithm was built in MATLAB by observing and testing different rule” cut-offs with the trial data. The random forest (online) and rule-based (offline) algorithms were evaluated using the subject trial data as the testing set and the results are provided in The online random forest algorithm had a true positive rate (TPR) of 97% compared to 92% from the offline algorithm. The online algorithm also had a false positive rate (FPR) of 11% compared to 23% from the offline algorithm. The false positives were from the simulated use events flagged as real use events of the medication. The false positives were also from incorrectly flagged cap removals without medication dispensing events. These incorrectly identified events occurred when a participant squeezed the bottle to remove the cap with the eyedropper nearly horizontal. These events were flagged as use events because they executed behavior similar to the start of a use event when the participant begins moving the medication toward their eye. The cap removal detection used with both algorithms was accurate and filtered out all events that did not require cap removal. With the machine learning algorithm running on the eye dropper’s printed circuit board (PCB), the instrumented eye dropper classified all usage events from the day scale trials correctly. Out of the seven times that the researcher dispensed medication, the bottle successfully recognized that the cap was off, ran the machine learning algorithm, and timestamped seven use events. The reed switches reliably detected the cap state, and the features and thresholds determined for the classifier were accurate enough to classify a use event at 100% accuracy, given proper use by a lab researcher. The researcher also observed every time a capacitance reading was recorded. When the bottle was standing still at an upright position, capacitance readings were recorded every 30 min. During one 30-min period when the researcher traveled with the bottle, the capacitance was not recorded. Fluid level measurements were only made when the bottle was in an upright position. Embedded sensing and electronics have the potential to greatly improve medication adherence and the consistency of medical interventions. Most healthcare providers currently do not have real-time information about when or how much medication their patients are using. Devices like glucose monitors or digital pedometers can provide meaningful feedback to patients. However, integrated systems that pass health monitoring data to or from both patients and clinicians in real-time are lacking. Additionally, systems to monitor medication adherence in real-time are currently only used in research because of their high cost and cumbersome design. Embedded systems to measure eye drop use, a particularly unique type of medication, and transmit the information both to patients and providers has great potential to improve medication adherence and outcomes. Patient access to information about when and how much medication they are using could motivate higher levels of medication adherence. Information from the bottle and supplementary reminders from the clinicians could also be used to reduce the amount of time that the patient is without medication. In addition to providing feedback to an individual patient, the proposed system for monitoring glaucoma medication has the potential to facilitate research that is necessary to quantify and improve patient adherence. Clinical studies to investigate the impact of improving adherence on biological outcomes require hundreds of patients. Current state-of-the-art systems can cost as much as $200 per month per medication for monitoring fees . For a study of 1000 subjects over the course of a year, this would result in an estimated cost of over $2,000,000. For this preliminary study, we made ten instruments at a total cost of ~$500, and we estimate that monthly monitoring fees will accrue at a rate of approximately $10/month for small data use plans. Additionally, the price per unit will drop when hundreds or thousands of sleeve systems are fabricated, creating a price point that would likely reduce the financial barriers that currently limit access to adherence monitoring. A lower price point would facilitate both large clinical studies and potentially ubiquitous medication monitoring for all patients with glaucoma, or at least for those in whom adherence is poor and/or the disease is worsening.For glaucoma medication, in particular, creating a system to augment existing FDA approved medicine bottles with embedded electronics is a necessary design feature, and there are currently no devices in the literature or market place of this type. The system presented here integrates sensors to monitor use and measure fluid level in a streamlined package, capable of real-time use classification and fluid level measurement. This information can also be shared in real-time and does not require patient input other than taking their medication as they normally would. To assess the accuracy and reliability of the system, a series of trials with the instrumented bottle were conducted with subject participants 65 years of age and over. For the online detection, a random forest algorithm was chosen over other optimal classification algorithms based on the results of cross validation. Random forest uses a combination of trees and averaging to determine the optimal threshold for features to achieve the highest accuracy . The macThe online random forest algorithm also outperformed the offline rule-based method investigated in this work. This can be seen by comparing the F1 score, which is the weighted average of the Precision and Recall values. Since F1 considers both Precision and Recall, a higher F1 score means that the model has lower false positives and false negatives if the Precision and Recall values are also both higher. The online algorithm obtained an F1 score of 0.93 and the offline obtained a score of 0.86, so the online algorithm had less false positives and false negatives. The machine learning algorithm was able to out-perform the rule-based algorithm for a couple of reasons. First, the machine learning algorithm looked at the summation of the absolute value of all changes of data in order to capture force variations due to the capacitance sensor. By looking at the absolute value, the algorithm captured the minor spikes in data that occurred when the user squeezed the bottle to dispense fluid see . Second,To qualitatively assess the streamlined packaging, the researchers asked the participants if they had any difficulties or concerns with the sleeve. Three of the ten participants mentioned that the sleeve made the eyedropper more difficult to squeeze. After reviewing the video recordings from these participants, it was clear that this difficulty was a result of hand placement near the bottom of the eye dropper. The sleeve added length to the bottle, resulting in hand placement lower than what the subject might normally expect. More time to familiarize themselves with the sleeve and the addition of colored marks or features to indicate correct hand placement would address this observation. Three participants also mentioned that the cap cover made the top of the bottle easier to remove. The cover made the cap larger, and the magnets allowed for an easier location to grip and twist off the cap.Currently our system can monitor adherence for ~17 h, enough time to operate during the day on a single charge. To improve power efficiency of the system going forward, a few changes can be made. First, the sensors that were chosen require more power than necessary. As can be seen from the results of the information gain evaluation, the implemented on-board detection algorithm could accurately classify use events without using features from the on-board gyroscope or magnetometer. Removing the gyroscope and magnetometer as well as selecting more power-efficient accelerometers would improve the system power consumption. Second, as seen in the power budget, the majority of power consumption was due to BLE. Since the purpose of this version of the system was to reliably communicate sensor data with a web server, BLE was optimized with the maximum settings in order to establish a steady connection. However, the trade-off of reliable communication is an increase in power consumption, which can be avoided by changing parameters such as the advertising and connection intervals. Since the bottle spends the majority of its time polling for a BLE connection, by reducing the advertising interval, the overall power consumption will decrease . By decrImproved manufacturability of future devices will be key to the creation of a low-cost system for persistent monitoring. For example, the reed switches were reliable and convenient, but require manual assembly because they need to be soldered to wires, glued into place, and wired down into the PCB. A different sensor such as a dedicated magnetometer could do the same job and would improve the manufacturability of the system. Finally, 3D printing of the sleeve is convenient because it can be easily adjusted for different sized bottles; however, 3D printing makes the manufacturing more expensive and limits the materials that can be used. Other manufacturing methods such as injection molding can be used to manufacture the larger parts such as the silicone-like outer sleeve, with 3D printing used to make inserts that adjust to different sizes.Accurately tracking patients’ adherence to medication is an issue that has not been well addressed, especially when it comes to eye drop based medications. Utilizing a select group of off-the-shelf and custom-built sensors combined with a Bluetooth enabled device, we developed a sensor system capable of detecting eye drop use, measuring fluid level, and sending use information to a healthcare team to facilitate intervention. A 3D printed sleeve that fits around a prescription eye dropper enables customization for various bottle shapes and sizes. Subject testing with ten patients aged ≥65 demonstrated that the system could successfully identify and timestamp 97% of use events and was capable of measuring fluid level at a 0.4 mL resolution. While developing this application, we found other uses for the sensors, specifically the capacitance sensor. Though eye drops are a very common liquid medication, accounting for 2.3% of Medicare Part B expenditures or $2.4 billion annually , there aThe portable sensor system developed is not a simple, one-off device with the sole purpose of monitoring eye-drop use. Rather, the core hardware and communication system can incorporate additional sensors to capture data on individual patient environments and activities. Future designs of this system can evaluate adding or replacing sensors that will enable physicians or researchers to understand the environment and movements of the patient in more detail. This can help provide a richer understanding of the patient environment and behavior, and can tailor the adherence process to each individual patient.


The interferometry synthetic aperture radar (InSAR) deceptive jamming method utilizing two synergetic transponders can generate a false three-dimension (3D) scene in a single baseline InSAR image. However, such deceptive capability could be reduced by the multibaseline InSAR system. To obtain effective deception on multibaseline InSAR, a novel deceptive scene generation method jointly employing multiple transponders is proposed. It only demands that each transponder is modulated with a complex coefficient when generating a false point. The complex modulation coefficient can be offline calculated according to the deceptive point coordinate by solving a matrix. Besides, the complex modulation coefficient can be combined with the deceptive scene template, and thus a large 3D deceptive scene is able to be created quickly in the multibaseline InSAR image by using the fast two-dimension (2D) SAR deceptive scene generation algorithm. As long as the number of transponders is not less than the number of antennas of the multibaseline InSAR system, this proposed method is effective. The effectiveness of the proposed method is validated by computer simulations. As a good supplement of the light imaging, synthetic aperture radar (SAR) can acquire two-dimension 2D) high-resolution images sustainably under all-weather and all-time conditions . InterfeD high-reHowever, the existing many traditional SAR interfering techniques such as noisy barrage jamming, shift-frequency jamming, intermittent sampling repeater jamming and scattered-wave jamming are hardly used to jam InSAR ,12,13,14Propose a new 3D deceptive scene generation method for jointly utilizing multiple transponders against multibaseline InSAR.Introduce an algorithm to acquire the complex coefficients applied to each transponder.Comparative experiments on the performance of the proposed jamming method for jointly employing three transponders with the existing jamming methods using two transponders against double-baseline InSAR.To generate a 3D deceptive scene in the multibaseline InSAR image, motivated by the idea of jamming multichannel SAR by using multiple transponders from ,21,22, wThe rest of the paper is organized as follows. In In this section, the geometrical model of single-pass multibaseline InSAR is first given, and then the basic principle of InSAR deceptive jamming is introduced.x-axis at a height of y-axis in the x–o–y plane with the same azimuth coordinate. As shown in From the above description, we know that the coordinate of the antenna Supposing that the transponder Defining In addition, define Considering the synthetic aperture duration and the baseline are relatively short comparing with the imaging slant-range, namely Correspondingly, As is well known, to achieve 2D high-resolution, SAR usually emits the linear frequency-modulation (LFM) signal. Therefore, the baseband signal that the SAR transmits can be expressed asTherefore, the signal intercepted by the transponder To generate a deceptive target Then, the echo from the transponder received by the antenna After I/Q demodulationLast, after range Doppler (RD) imaging and image co-registration, the imaging result of the jamming signals generated by the transponder Equation (14) reveals that the phase of the final deceptive jamming result is only related to the shortest length This section first analyzed the limitation of the traditional deceptive jamming generated by a single transponder and then introduced an effective jamming method for jointly employing two transponders against the single baseline InSAR. In the end, the reason why double transponders fail to deceive multibaseline InSAR was given.From the above analysis, we learnt that the final phase of the imaging result for the antenna Similarity, the imaging output of the false target in the image domain for antenna Accordingly, the phase difference of the false target Obviously, the phase difference In Considering the azimuth position of the transponder has no difference in the final phase of the false target in the image domain, for simplification, two transponders Thus, to obtain the desired deceptive jamming target, the phase difference between Obviously, there are arrays of solutions Solving Equation (21), the complex modulation coefficients In Apparently, the phase difference of the false target between Based on the previous analysis, we could conclude that to obtain the desired phase satisfying the height of the deceptive target, the number of transponders should not be less than the number of antennas of the multibaseline InSAR system. Thus, in this section, we proposed a new deceptive jamming method against multibaseline InSAR based on multiple transponders.M transponders working synergistically, and to generate the false target Reconsidering Equation (14), it gives the imaging result of the false target Accordingly, if the phase difference of the imaging output of the jamming signals between two co-registration images for the antenna To obtain the desired height of the false target, the Equation (24) should satisfy the phase corresponding to the real target’s height for all Rewrite Equation (25) in matrix as By solving Equation (26), the complex modulation coefficients can be obtained. By applying these complex modulation coefficients in the transponders, the desired height of the false target can be obtained.Besides, in practical electronic war (EW), the jamming systems perform better when the number of the transponders is more than the number of antennas of InSAR systems ,26. TherFrom the above analysis, we know that the new deceptive jamming method does not require the transponder to change too much compared to the traditional SAR deceptive jamming method and only to be modulated with a complex modulation coefficient when generating the jamming signals of a deceptive target. We could also find that the complex modulation coefficient could be equivalent to the reference complex scattering coefficient of the false target. Thus, we could combine the complex modulation coefficient with the reference complex scattering coefficient of the false target to form a new scattering coefficient. In this way, the jamming system using the proposed jamming method to generate a 3D deceptive scene is just like using the traditional SAR deception jamming method to generate a 2D deceptive scene.Assume that the deceptive scene template is a To generate a 3D deceptive scene, the existing fast deceptive jamming method against SAR can be used. Here, we used the algorithm proposed in . In thisIn this section, to verify the effectiveness of the proposed method, simulations were provided based on a double-baseline InSAR system with three receiving antennas The program of the simulation experiments was written with the Matlab language and carried out in the Matlab 2016 environment. In the simulation, first, we simulated the jamming signals based on the proposed method and planted them in the background signals; and then, used the range-Doppler (RD) algorithm to obtain the SAR image of each antenna and operate image coregistration; next, after image coregistration, we extracted the phase interferogram corresponding to the baseline and carried out the phase unwrapping operation. Last, according to the absolute interferogram after unwrapped, we obtained the height of the false target. Besides, the jamming signal power to clutter signal power ratio (JCR) was 5 dB in the simulation experiments.In this section, we verified the effectiveness of the proposed method based on whether the coordinates of the generated false targets match the set values. Therefore, we set five false point targets in the image scene, denoting as However, when only two transponders were utilized to generate jamming, the created false point targets shown in Based on above analysis, we could also conclude that the number of transponders should not be less than the number of antennas of multibaseline InSAR systems when the transponders were utilized to generate deceptive jamming.In this section, a 3D deceptive scene was generated based on our proposed method. The 3D deceptive scene template was created using peaks function in Matlab, and it was an area of 100 m × 100 m in the ground range and azimuth directions. The elevation map is shown in To quantitatively verify the effectiveness of the proposed method in generating the 3D deceptive scene, we chose three points The simulation results verified the effectiveness of the proposed method. In fact, the applicability of the proposed method was also feasible in practice. First, with the rapid development of the electronic reconnaissance techniques ,28, the In this paper, a novel 3D deceptive scene generation method based on multiple transponders for interfering multibaseline InSAR was proposed, where the number of transponders should not be less than the number of antennas of the multibaseline InSAR systems. It derived from the traditional SAR deceptive jamming and only required each transponder to be modulated with the complex coefficients related to the deceptive scene template when generating the jamming signals. By applying these coefficients in each transponder, the phase of the composite jamming signals in each InSAR antenna satisfied the desired DEM. Moreover, this method was also computationally efficient for a large 3D deceptive scene generation, because these coefficients could be calculated offline and combined with the reference complex scattering coefficient of the deceptive template. Therefore, by using the existing SAR 2D deceptive jamming fast algorithms, the 3D deceptive scene could be created rapidly. This makes the proposed jamming method suitable for EW environments that require high real-time conditions. Simulation results show that the proposed method could effectively generate 3D deceptive scenes in multibaseline InSAR image.Besides, in this paper, we only gave the method for generating the jamming signals against the single polarimetric InSAR. However, the proposed method is also appropriate for jamming the fully polarimetric InSAR. As is well known, the fully polarimetric InSAR increases the observation information on target scattering characteristics and can help to provide more physical features of the target. While using our proposed method to generate the jamming signals, the scattering coefficients of the false targets are fully utilized. Therefore, for the fully polarimetric InSAR, by performing polarization modulation for the jamming signals, the synergetic transponders are able to generate the false targets with high fidelity.


The symmetric positive definite (SPD) matrix has attracted much attention in classification problems because of its remarkable performance, which is due to the underlying structure of the Riemannian manifold with non-negative curvature as well as the use of non-linear geometric metrics, which have a stronger ability to distinguish SPD matrices and reduce information loss compared to the Euclidean metric. In this paper, we propose a spectral-based SPD matrix signal detection method with deep learning that uses time-frequency spectra to construct SPD matrices and then exploits a deep SPD matrix learning network to detect the target signal. Using this approach, the signal detection problem is transformed into a binary classification problem on a manifold to judge whether the input sample has target signal or not. Two matrix models are applied, namely, an SPD matrix based on spectral covariance and an SPD matrix based on spectral transformation. A simulated-signal dataset and a semi-physical simulated-signal dataset are used to demonstrate that the spectral-based SPD matrix signal detection method with deep learning has a gain of 1.7–3.3 dB under appropriate conditions. The results show that our proposed method achieves better detection performances than its state-of-the-art spectral counterparts that use convolutional neural networks. The essence of signal detection is a binary classification problem to judge whether an input sample contains a target signal. It is the main content of signal detection to find the characteristics of the target signal and distinguish the input sample as only containing interference or existing target signal. Feature-based signal detection is a hotspot in the signal processing field. There are many classic approaches that have helped drive the field, but their shortcomings are also obvious. The instantaneous method , which uIn recent years, the manifold of SPD (symmetric positive definite) matrices has attracted much attention due to its powerful statistical representations. In medical imaging, such manifolds are used in diffusion tensor magnetic resonance imaging , and in To make full use of the Riemannian geometry characteristics of SPD matrix manifolds, a series of studies have been carried out in the fields of nonlinear metric and matrix learning. To address the problem that SPD matrix manifolds are not applicable to the Euclidean metric, many non-Euclidean metrics are introduced, such as the affine-invariant metric  and the Regarding deep learning, ref.  proposedIn this work, we propose a spectral-based SPD matrix signal detection method based on a deep neural network. This method utilizes an SPD matrix deep learning network and two matrix models: an SPD matrix of spectral covariance and an SPD matrix of spectral transformation. This method transforms the problem of signal detection into a binary classification problem on a nonlinear spatial matrix manifold. When nonlinear metrics are applied to signal detection, the differences between the SPD matrices can be found well. Under low SCR conditions, our method outperforms the spectral signal detection method based on a convolutional neural network.The remainder of this paper is organized as follows: In The signal detection problem can be transformed into a spectral dichotomy problem, making the methods applicable to image classification also applicable to signal detection. The spectrogram saved in the form of 2-D image is a typical Euclidean structure data . The strFor an image sample with N pixel points, after it is expanded into a vector, it can be regarded as a point in the N-dimensional Euclidean space, and each original pixel point can be regarded as a dimension value of that point. At this point, the difference between different samples is transformed into the distance of different points in N-dimensional Euclidean space. The idea of full connection layer algorithm in deep learning is based on it.Based on the above characteristics, the convolutional neural network is well established. Since AlexNet  first apGoogLeNet is a type of convolutional neural network proposed by Google in 2014 that won first place in the ILSVRC2014 competition. GoogLeNet has an improved sparse network design, called Inception. Inception has steadily been improved across multiple versions . Here, we introduce the Inception V1 version used by GoogLeNet.The Inception module of GoogLeNet originates from a simplified model of the neurons in the human brain. Its structure is shown in ResNet was proposed in 2015 and won first place at ILSVRC2015. Its innovation was the inclusion of a residual block structure that solves the vanishing gradient problem that occurs as the number of neural network layers increases.ResNet networks currently have evolved through five versions: ResNet18, ResNet34, ResNet50, ResNet101 and ResNet152. In this study, we only use ResNet50.The residual block structure of ResNet50 is shown in The innovations introduced by GoogLeNet and ResNet improved the connection mode of each neural network layer, providing a strong inhibitory effect on the learning degradation caused by deepening the network.Just as the statistical distribution of image data follows the principle of Euclidean space, the statistical distribution of SPD matrices also follows the principle of Riemannian geometry. The problem of signal detection based on SPD matrix manifold can also be transformed into a dichotomy problem about SPD matrices, but solving the problem of Riemannian geometry by utilizing Euclidean metric will lead to unsatisfactory results. In The following eigen-decomposition can be carried out on a square matrix A:The eigenvalues and eigenvectors of the square matrix reflect the matrix properties. Eigenvalue decomposition is not applicable to a non-square matrix Then, the non-Euclidean properties of the SPD matrix manifold are used to classify the set of Based on the above ideas, we propose the threshold method for constructing an SPD matrix:The steps to transform the spectrum into the SPD matrix are as follows: first, the original time domain sequence is transformed into the energy distribution based on time axis and frequency axis in the form of two-dimensional matrix using the short-time Fourier transform, and the modulus of such matrix is retained to generate sample The authors of  proposedThe covariance matrix constructed by this method follows the properties of the SPD matrix. Inspired by it, we propose the SPD matrix construction method for spectral transformation, which is called spectral covariance in this paper. The steps of this method are as follows:The steps to transform the spectrum into the SPD matrix are as follows: first, the original time-domain sequence is transformed into a time-frequency joint energy distribution in the form of a two-dimensional matrix by short-time Fourier transform, and their modulus matrix is retained. Then, the modulus matrix generated in the previous step is transformed into The covariance method has some disadvantages. First, it describes the correlations between samples; however, the use of correlation means that constructing an SPD matrix requires multiple raw samples. Thus, this constructor is not suitable for addressing situations with insufficient data. Second, averaging calculation performed in the covariance method may result in the loss of some features. These deficiencies reduce the performance of the SPD matrix learning method based on a deep neural network.Similar to classical convolutional neural networks, SPDnet also designs convolutional-liked layers and rectified linear units-like layers, named bilinear mapping (BiMap) layers and eigenvalue rectification (ReEig) layers, respectively. BiMap layers aim to make the input SPD matrix manifolds more compact and more discriminative and ReEig layers aim to rectify the upper input SPD matrices with a non-linear function. After BiMap layers and ReEig layers, the input SPD matrices will be converted into SPD matrices that follow a more compact and discriminative Riemannian manifold distribution. The formula for a BiMap layer is as follows:The formula for a ReEig Layer isThe LogEig layer is exploited to perform Riemannian computing on the resulting SPD matrices. It utilizes the log-Euclidean Riemannian metric to reduce the Riemannian manifold of SPD matrices to a flat space with the matrix logarithm operation The main difficulties in training SPDnet lie both in backpropagation through the structured Riemannian functions and in manifold-constrained optimization. For more details, please refer to ,21.In the framework of a deep neural network, we compared three signal detection methods: the signal detection method based on time-frequency spectrum images, the SPD matrix signal detection method based on spectral covariance and the SPD matrix signal detection method based on spectral transformation. The data were derived from a simulation signal dataset with K-distribution clutter as interference.K-distribution is the most recognized model of real sea clutter radiation. It takes the correlation between pulses into account, well fits the amplitude distribution of sea clutter, and well explains the scattering mechanism of sea clutter. K-distribution model is composed of the texture component with slow change following the Gamma distribution and the speckle component with fast change following the Rayleigh distribution. The probability density function of K-distribution isWe use a simulated target signal in our experiment. The driving vector of the target signal isThe normalized doppler frequency is defined as follows:The dataset is divided into two categories: clutter only and signal interfered by clutter. The clutter only consists of 11,000 samples. The signal interfered by clutter consists of a series of samples with different SCR distribution ranges from −5 dB to −30 dB at intervals of 1 dB, each with 5000 samples. We exploit GoogLeNet and ResNet50 to detect the target signal in time-frequency spectra and SPDnet to detect the target signal in SPD matrices. For GoogLeNet and ResNet50, we used transfer learning to adjust the fully connected layers to fit our detection problem, and SPDnet is retrained. The data used for transfer learning and retraining are clutter samples and signal samples with SCRs of −5, −10, −15, −20, −25 and −30 dB, respectively. Each training set includes 1000 clutter samples and 1000 signal samples with the same SCR. We used 4000 signal samples for calculating detection probability at each SCR level that had no overlap with the samples used for training. The hyperparameters involved in training are as follows: the batch size is 20, the learning rate is 0.001, and the weight decay is 0.0005. For SPDnet we used an epoch of 500; the epochs for GoogLeNet and ResNet50 were 2000. Under these values, the accuracy of the models is guaranteed to reach saturation during the training processes. We trained SPDnet on an i7-8565U CPU, while GoogLeNet and ResNet50 were trained on an Nvidia GTX 960 M GPU. The detection probability results are depicted in A comparison of the two SPD matrix detection networks showed that when the false alarm probability of the spectral transformation SPD matrix network is greater than Hyperparameters affect the learning performances of deep networks. In this section, we study the influence of two hyperparameters, the learning rate and the weight attenuation coefficient, on the performance of the spectral-based SPD matrix signal detection method with deep learning and study the relationship between the number of network layers and the spectral-based SPD matrix signal detection method with deep learning. For simplicity, we exploit only the spectral transformation SPD matrix network as the research target. The false alarm probability in all the experimental combinations involved in the detection probability comparison is less than Weight decay is a method to prevent overfitting of deep network models. By gradually reducing the weight to a smaller value, the deep network model overfitting problem can be reduced. The overfitting problem occurs when the network’s performance is good in training but poor in testing. In our study, one of the hyperparameters is the magnitude of weight decay. From For a deep neural network, differences in the number of layers can affect model performance. We conducted a comparative experiment to investigate the ideal number of layers; the results are shown in By comparing the results of the detection probability curves, when the number of iterations is sufficient, the influence of the learning rate on the detection probability is greater than its influence on the false alarm probability.To further compare the detection performance of the different models, we utilize the lake IPIX (Ice Multiparameter Imaging X-Band) radar data file collected by McMaster University in Ontario on February 23, 1998 as the clutter . The refIn this paper, we proposed a spectral-based SPD matrix signal detection method based on a deep network. Using the SPD matrix of spectral transformation and combined with the deep SPD matrix learning network, the proposed method can detect a target signal even under low SCR. The advantages of this method are that it exploits the nonlinear spatial property of the SPD matrix space as a Riemannian manifold, increases the discriminability of different matrices, and uses a nonlinear metric to achieve the dichotomy. Simultaneously, the adoption of deep learning enhances the method’s ability to aggregate the SPD matrices and adjust the parameters to adapt to signal detection problems. We used a simulation signal dataset based on K-distribution clutter and a semi-physical simulation signal dataset based on real sea clutter to test our proposed method. The results showed that our method outperforms the signal detection method based on a convolutional neural network. And the gains realized based on the simulation dataset and the semi-physical simulation dataset are 3.3 dB and 1.7 dB, respectively. We also studied the performance impacts of the hyperparameters on our method. Because our method is based on time-frequency spectra transformation, it can be applied to both signal processing and image detection tasks.


The method computes sOThe ability of photoacoustic imaging to measure functional tissue properties, such as blood oxygenation sO Furthermore, characteristic changes in local sO2. State-of-the-art methods to obtain this value are limited as they are either invasive , lack practicability and accuracy , or only yield a rough global estimate from peripheral vasculature (e.g. pulse-oximetry6). None of these techniques yield real-time, spatially-resolved sOTissue blood oxygen saturation sO7, including model-based inversion techniques12 and data-driven approaches19. Due to limitations in terms of applicability, repeatability, or ease-of-use, none of the advanced methods are routinely used in the field of multispectral PAI. Instead, the most commonly applied technique for estimating sO21, which assumes a linear combination of relevant chromophores . LSD is based on the assumption that the domain gap can be addressed by performing training on initial pressure data by using normalized pixel-wise pOur work introduces a data-driven approach for tackling the problem of sOlearned spectral decoloring (LSD) method as depicted in Fig. This section will first introduce the principle of the 8 has shown that it is theoretically feasible to compute pixel-wise sOc: e: a priori. It is assumed that the extraction of 26 it was shown that deep learning algorithms can help to mitigate these issues. One of the core assumptions of learned spectral decoloring (LSD) is that the optimal set of constraints for the inversion can be learned from the wavelength-dependent changes of n is the number of recorded wavelengths. Assuming that the Grüneisen parameter mentclass2pt{minimDuring training, the algorithm is given tuples domain gap and thereby enables the LSD method to be applied to real data with unknown calibration of the PA measurement device. Real data needs to be reconstructed from raw time series pressure data in order to form a spatial image of signal amplitudes. A reconstruction algorithm can introduce artifacts to the image, for example, due to limited view geometries27. However, in this work, the biases introduced by the reconstruction are assumed to be independent of the wavelength, allowing the algorithm to be directly trained on The amplitude information of the recorded spectra is crucial for quantitative photoacoustic imaging which aims to obtain absolute concentrations of chromophores in tissue. However, by sacrificing the amplitude information, we also eliminate the need to calibrate the in silico training data to the acquisition device and the specific target domain. This drawback is tolerable, as we are only interested in the relative ratios of HbOSeveral data sets were used in this work for training and validation. These validation data comprised (a) an in silico data set, (b) in vitro data set from a blood flow phantom, and in vivo data sets from (c) an open porcine brain and (d) forearms of healthy human volunteers. The purpose of the in silico data sets was to evaluate the sOSynthetic data. A total of three simulated data sets were generated to train the models and test the method in different scenarios , defined as vessel structures where the signal at the isosbestic point of 800 nm was higher than a noise equivalent threshold  and setting a threshold of CNR Blood flow phantom data. This data set consisted of three measurements of a blood flow phantom setup, including reference blood oxygenation measurements provided by partial oxygen pressure . The blood samples were first chemically oxygenated and then chemically deoxygenated during the measurement process, theoretically going from 100% blood oxygenation to 0% blood oxygenation over the measurement time, with continuous reference measurements being taken by pO41. For the evaluation of the method on this data set, ten consecutive frames of the same wavelength were averaged to account for laser pulse energy fluctuations. The tube structure was automatically segmented by only taking pixels in which the signal at 800 nm was greater than Porcine brain data. This data set consists of a multispectral image series that was taken from a previous animal experiment in which a porcine brain was imaged during open brain surgery42. The used images were acquired as part of a pilot study to see the hemodynamic responses of the brain during spreading depolarization. The specific data used for this study corresponded to a baseline measurement done to establish the capability of PAI to distinguish different hemodynamic states which were induced by using different levels of respiratory oxygen. During the entire length of the imaging procedure, 38 minutes, the animal was supplied with different levels of respiratory oxygen (rO43). Further details on the experimental design and the general image acquisition can be found in previous publications of the experiments44. Human forearm data. This data set consists of multispectral images of the left and right forearms of three healthy human volunteers. The images were taken at three distinct positions a distance of approximately 2, 4, and 6 cm from the radiocarpal joint, leading to a total of 18 image sequences. The person operating the handheld PAI device attempted to capture either the arteria radialis or the arteria ulnaris in the imaging plane. These vessels could be identified by their pulsating motion which is induced by the heartbeat. The data sets were recorded using the MSOT Acuity Echo PAI device . The images were taken at a wavelength range from 700–950 nm in intervals of 10 nm for at least 30 s. The ten subsequent multispectral sequences with the least amount of movement were averaged to account for laser intensity fluctuations and to increase the robustness against motion artifacts. Finally, the images were scaled to a size of 256 For each of the training data sets, a separate fully connected feed-forward neural network was trained to account for domain-specific differences. The feed-forward architecture was chosen because we decided to use a well-understood baseline method for this initial study and a single-pixel approach does not warrant the use of convolutional neural networks. The number of input features was set to the number of wavelengths in the respective multispectral sequence (26 for the forearm and generic data set and 17 for the flow phantom data set), the model contained four hidden layers and the size of these hidden layers was set to be twice the size of the input vector see Fig. .Figure 3The models were trained for 100 epochs, where one epoch contained 500 batches of size 33. The unmixing method was implemented in Python 3.7., using the minimize function of the scipy python package that implements the SLSQP  algorithm for finding the best fit. The unmixing was done exclusively for Hb and HbOAs the LU technique constitutes the state-of-the-art method in functional parameter estimation for photoacoustic imaging, it was used as a reference method to compare the proposed LSD method to. It was performed using literature absorption spectra of pure Hb and HbOThe healthy human volunteer experiments were carried out in accordance with relevant guidelines and regulations and were approved by the ethics committee of the medical faculty of Heidelberg University under reference number S-451/2020. The study is registered with the German Clinical Trials Register under reference number DRKS00023205. All porcine experiments were carried out in accordance with relevant guidelines and regulations, including the ARRIVE guidelines, and protocols were approved by the institutional animal care and use committee in Karlsruhe, Baden-Wuerttemberg, Germany (Protocol No. 35-9185.81/G-174/16).Informed consent was obtained from all volunteers.This section presents the results of the LSD method on the simulated in silico data sets, on the in vitro flow phantom data sets, and on the in vivo porcine brain and human forearm data. A separate LSD regressor was trained for each of the respective in silico training data sets. The key findings of the experiments are summarized in Fig. Figure The median relative sOabsolute sOThe median For a comparative analysis of the flow phantom data, three techniques of sOAll three of these measurements showed a monotonous decrease in the blood oxygenation level over the time frame of the experiment. The pOIn addition, Fig. In the porcine brain image series, LSD was observed to increase the dynamic range of the predictions while maintaining the same tendency . Median oxygenation values were computed and tracked over time in a manually placed ROI in Fig. Figure LSD, which is able to account for the spectral coloring effects in multispectral photoacoustic imaging when estimating the blood oxygen saturation of tissue. Other recently published deep learning-based methods have only reported results on simulated data17, or preliminary results on simple phantom setups19. This shortcoming in the field may be attributed to the domain gap between simulated data and real measurements. LSD is based on the assumption that the consideration of spatial relations in PA measurements in simulations amplifies the domain gap between simulation and reality. Therefore, it is trained on single-pixel pThis paper introduces the machine learning-based method In vivo measurements of the forearms of healthy human volunteers suggest that LSD is capable of yielding physiologically plausible sO45. On the other hand, since single-pixel spectra can be highly ambiguous when considering a too large solution space, thus we strike a careful balance between generalizability and applicability of the method by designing a suitable data set for a specific application. In addition to this trade-off, another severe limitation of the method is that the inversion model is wavelength-dependent, which means that a trained model can only estimate sOThe LSD method is capable of performing estimations in realistic settings, despite being trained on simulated data, demonstrating the ability of the approach to bridge the domain gap between real and simulated images. In fact, to our knowledge, no prior work has successfully applied convolutional neural network-based approaches to sOIn summary, LSD provides encouraging advantages over conventional LU techniques for sOSupplementary Information 1.


Obstructive sleep apnea syndrome is a reduction of the airflow during sleep which not only produces a reduction in sleep quality but also has major health consequences. The prevalence in the obese pediatric population can surpass 50%, and polysomnography is the current gold standard method for its diagnosis. Unfortunately, it is expensive, disturbing and time-consuming for experienced professionals. The objective is to develop a patient-friendly screening tool for the obese pediatric population to identify those children at higher risk of suffering from this syndrome. Three supervised learning classifier algorithms  common in the field of machine learning were trained and tested on two very different datasets where oxygen saturation raw signal was recorded. The first dataset was the Childhood Adenotonsillectomy Trial (CHAT) consisting of 453 individuals, with ages between 5 and 9 years old and one-third of the patients being obese. Cross-validation was performed on the second dataset from an obesity assessment consult at the Pediatric Department of the Hospital General Universitario of Valencia. A total of 27 patients were recruited between 5 and 17 years old; 42% were girls and 63% were obese. The performance of each algorithm was evaluated based on key performance indicators . The logistic regression algorithm outperformed  the support vector machine and the AdaBoost algorithm when trained with the CHAT datasets. Cross-validation tests, using the Hospital General de Valencia (HG) dataset, confirmed the higher performance of the logistic regression algorithm in comparison with the others. In addition, only a minor loss of performance  was observed despite the differences between the datasets. The proposed minimally invasive screening tool has shown promising performance when it comes to identifying children at risk of suffering obstructive sleep apnea syndrome. Moreover, it is ideal to be implemented in an outpatient consult in primary and secondary care. The obstructive sleep apnea syndrome (OSAS) is a health problem characterized by recurrent episodes of reduction of airflow. These can be partial (hypopnea) or complete (apnea). According to the American Academy of Sleep Medicine (AASM), apnea is defined as an airflow reduction of at least 90% lasting for at least 10 s, while a hypopnea is a 30% reduction for at least 10 s or at least a 3% reduction in oxygen saturation SpO2) in comparison with the pre-event baseline or associated with arousal  in compa. Its preFull night polysomnography (PSG) has been recognized by numerous professionals as the gold standard procedure to diagnose OSAS in children and adults ,6,7,8. GOut of all the listed alternative methods, the authors are especially interested in those which can be performed in an outpatient context. This is especially relevant for screening and follow-up strategies in the obese pediatric population. The first of them is the RP, which reduces the invasiveness of the procedure by reducing the number of signals being recorded. Usually, the oronasal flow, chest and abdomen movements, heart rate and blood oxygen are the key parameters measured. Different tests have demonstrated that RP and home-based RP (HRP) are suitable for diagnosing OSAS in children and adolescents ,11. The Despite the evident differences between all of these methods, they still require trained personnel to interpret the results. In some cases, this can be time-consuming even if a signal processing software is available with the hardware. Besides the increased physician workload that can be associated with such testing, the results are strongly dependent on the experience of the operator, and strong variabilities in the diagnosis might occur between experts.In the past two decades, multiple solutions have been proposed to reduce or mitigate such dependencies in adults and pediatric populations see . GeneralFor the pediatric population, there seem to coexist ML- and DL-based solutions with similar performance between them. The threshold values used to classify severity have a greater impact on performance in this cohort than in adults.Unfortunately, the development of this type of solution is not accessible to everyone and in some cases might require a continuous update. In this context, the main objective of this paper is to develop a patient-friendly screening tool oriented to the pediatric population and feasible to be implemented on an outpatient basis. Moreover, this solution should be easy to run, minimizing the interaction of the physicians throughout the whole process and keeping them updated without the need for domain-specific skills. This will allow the popularizing of this type of diagnostic test, which can be especially relevant for specific pediatric cohorts like the obese population.Two datasets were used in this study. The first one was the Childhood Adenotonsillectomy Trial (CHAT), obtained from National Sleep Research Resource (NSRR), ,35,36,37Out of all the variables available from this dataset, only those related to oxygen saturation (SpO2) were considered in the present study. The main reason is that apneas and hypopneas are both defined by this parameter. This signal was recorded in the CHAT with a pulse-oximeter, Nonin Model 8000J or comparable, at a sampling rate of 10 Hz or higher. The extracted features from the SpO2 signal are listed and defined in The second dataset consisted of 27 subjects, between 5 and 17 years of age and recruited from those attending an obesity assessment consult at the Pediatric Department of Hospital General of Valencia. All were Caucasian, 42% were girls and 63% were obese (defined as BMI percentile at or above 95%). All participants underwent an HRP using the Philips Alice Respironics device with a minimum of 6 h of time in bed (TIB) and 3 h of sleep time . The recFinally, to allow direct comparison of the variables from both datasets, a standardization process was performed. This consisted of calculating the Z-score value for each of them.Three popular ML algorithms specially oriented to supervised learning were used in this study. The first of them is the logistic regression (LR) model, which uses a weighted least square algorithm to predict the regression line that best fits the data points by minimizing the weighted sum of the square distances to the fitted regression line. It is simple and easy to implement and can relate one dependent variable with one or several independent variables. The second is the support vector machine (SVM), which tries to model the input variables by finding the separating boundary  to reach the classification of the input variables . The thiThe CHAT dataset was used to train and test each of the classifiers according to the extracted features from the SpO2 raw signal. A 15 k-fold approach was used. Balanced datasets were used. On the other hand, cross-validation was performed using the Hospital General de Valencia (HG) dataset, which is only composed of an obese population ranging from 9 to 17 years of age. The same features as those in the CHAT dataset were calculated from the raw SpO2 signal using MATLAB software.Balanced datasets were used to evaluate the performance of each model based on the following parameters: recall or sensitivity, precision or positive predictive value (PPV), specificity, negative predictive value (NPV), accuracy, F1 score and area under the curve (AUC). The recall or sensitivity and precision or PPV are two very important parameters for assessing the performance in identifying unhealthy patients . The former is the portion of real positive cases that are correctly predicted positive, while the latter denotes the portion of predicted positive cases that are really positive. Mathematically, they are defined according to Equations (1) and (2), respectively.The specificity and the negative predictive value (NPV) are respectively homologous to the two previously described parameters but assess the performance of the model regarding its capability to identify the negative values or healthy subjects. They are mathematically defined by Equations (3) and (4).Accuracy and F1 score are two parameters frequently used to assess overall performance. Accuracy is the portion of correctly identified cases, independently of being positive or negative, from the total number of samples (see Equation (5)). The F1 score is the harmonic mean between PPV and sensitivity. Out of the three different types of averages that can be calculated , the harmonic average is the most conservative of them all; in other words, it is the one that yields the lowest value. It is defined by Equation (6).Finally, the area under the curve (AUC) parameter was also considered. This value is calculated after performing a receiver operating characteristic curve (ROC) analysis. It can also be assessed graphically by plotting sensitivity against 1 – specificity and determining the area under the resulting curve. A higher AUC indicates that the model has a better capability to distinguish between healthy and unhealthy subjects. p-value ≤0.05 was considered to be significant.Statistical analyses were performed on both datasets using Python ,41,43,44To build the model based on the extracted features, the first step was to perform a cross-correlation matrix to determine the relationship between each pair of features as well as with AHI. The authors arbitrarily defined a strong correlation if values were greater than 0.9. In this case, dimensionality reduction was applied. This is crucial for obtaining an effective algorithm by avoiding the incorporation of repeated information into the model. This dimensionality reduction was achieved in LR using the L1 penalty term and by the mutual information measure (MI) in the SVM and AdaBoost procedures. In other words, features with lower MI were eliminated.The selection of the best predictive model was done using a nested cross-validation method for adjusting the model parameters and estimating the error.Initially, the distribution of AHI for each dataset was plotted see . As expep-values were lower than 0.05  ,4. The cAlternative methods have been proposed to make these tests more patient-friendly and to reduce the workload and the diagnostic discrepancies between experts. One of the most promising due to its ease of use and low cost is the NOS, which mainly focuses on measuring the SpO2 signal, coupled with an ML- or DL-based classifier. One major limitation of developing automated interpreting systems is choosing the most clinically meaningful biosignal. Multiple studies have tested different signal combinations to improve the performance of the classifier in detecting the OSAS severity ,25,26,34Unfortunately, it is very common that these tools present a strong dataset dependency reflected by a drastic reduction of their performance when applied to a second independent dataset. Consequently, this makes it complex to compare the performance between tools and to widely implement them. DL-based solutions can overcome these limitations to an extent by using unsupervised learning algorithms to perform pattern recognition and data interpretation. However, the need for big datasets as well as domain-specific skills and software to develop and maintain these classifiers makes this approach more expensive, which limits its implementation outside clinical research where the budget of healthcare providers is already tight. In this context, the authors have focused on the development of an affordable screening tool that can be easily implementable in an obesity assessment outpatient consult to identify those asymptomatic children and adolescents who are at higher risk of suffering OSA. This is especially relevant not only for prescribing personalized physical activity strategies to fight against obesity but also for following up. This tool is composed of a commercially available device for measuring transcutaneous oxygen levels in the blood (SpO2) and an ML-based classifier for data interpretation based on the extracted features. Out of the three tested ML algorithms  the LR has shown better performance on the CHAT test set and after cross-validation in a second independent dataset . The LR outperforms the other two regarding specificity and PPV and underperforms regarding sensitivity, while all three have similar accuracy and AUC see . This peCross-validation of this tool was performed with a second independent dataset . Interestingly, only a minor loss of performance was observed in the LR-based solution despite the significant differences between both datasets . It is also relevant to point out that while the readings in CHAT were hospital-based, those in HG were home-based. Even though this loss is observed in all five indicators, specificity and PPV are the indicators that present greater reductions; this is in contrast with accuracy, recall and AUC, which only suffer a slight decrease see . It is wDespite the need to increase the sample size of the HG dataset, these results are promising and indicate that it is worth continuing the development and optimization of the developed tool, which is expected to benefit clinicians as well as the pediatric population and their families. The former will benefit by having a tool that does not increase their workload; is easy to use and maintain; is cheap, robust and reliable; and reduces diagnosis variability between experts. The latter will benefit from a minimally disturbing tool capable of being utilized at home in an unattended manner.In the recent past, alternative methods based on SpO2 measurements have been proposed to diagnose OSAS in symptomatic adults and the pediatric population. However, none of them have targeted the asymptomatic obese population even though OSAS prevalence can reach up to over 50%.The proposed tool has shown promising results even when applied to a second independent dataset where the tests were performed on a very distinct cohort and in a home-based setting instead of a hospital-based one. Additionally, major differences were also present regarding the SpO2 measuring device. The performance yield suggests that the methodology employed can be implementable abroad. Clinicians and, in particular, the pediatric population will benefit from a tool like the one developed in this manuscript.


A traditional successive cancellation (SC) decoding algorithm produces error propagation in the decoding process. In order to improve the SC decoding performance, it is important to solve the error propagation. In this paper, we propose a new algorithm combining reinforcement learning and SC flip (SCF) decoding of polar codes, which is called a Q-learning-assisted SCF (QLSCF) decoding algorithm. The proposed QLSCF decoding algorithm uses reinforcement learning technology to select candidate bits for the SC flipping decoding. We establish a reinforcement learning model for selecting candidate bits, and the agent selects candidate bits to decode the information sequence. In our scheme, the decoding delay caused by the metric ordering can be removed during the decoding process. Simulation results demonstrate that the decoding delay of the proposed algorithm is reduced compared with the SCF decoding algorithm, based on critical set without loss of performance. At present, the fifth generation (5G) mobile wireless communication system has begun to be commercially used. In 5G mobile wireless communication, polar codes have been used to transmit signal or synchronize data in control channel. In , Arıkan However, the complexity of the SCL algorithm is high. To reduce the complexity of the SCL algorithm, Orion et al. proposed the successive cancellation flip (SCF) decoding algorithm, which is a combination of the bit-flipping decoding and the SC decoding . The SCFTo further reduce the decoding complexity, Zhang et al. proposed a critical set, which has a high probability to include the first error bit in the initial SC decoding algorithm based on search tree . Condo eHowever, the SCF decoding algorithm has a high complexity at low signal-to-noise ratio (SNR). To overcome this shortcoming, Lv et al. in  proposedRecently, machine learning (ML) techniques have been widely applied to many industries and research domains. In communication systems, ML-assisted communication systems have been widely studied.For a deep learning technique, some authors in ,20,21,22For reinforcement learning, some authors in ,24,25 usFor the SCF decoding algorithm, it is critical to accurately locate candidate flipping bits and reduce decoding delay. In order to accurately locate the candidate bits, Wang et al. proposed a deep learning-assisted SCF decoding algorithm, using a long short-term memory (LSTM) network and reinforcement learning to find the error bits . At present, little work has been devoted to applications of reinforcement learning on the SCF decoding of polar codes. The reinforcement learning algorithm can be used in the SC bit-flipping decoding, because the selection of candidate flipping bits can be seen as the decision process. The selection process of candidate bits is modeled as MDP. Compared with the traditional method, the reinforcement learning algorithm can quickly obtain the appropriate strategy. Therefore, the reinforcement learning is used to optimize the SCF decoding of polar codes in our design.In this paper, we propose a Q-learning-assisted SCF (QLSCF) algorithm which uses the reinforcement learning to reduce decoding delay of the SCF decoding. First, we provide a set of candidate bits as the action space for reinforcement learning, and verify the completeness of the action space. Secondly, we design a one-bit SCF decoding algorithm assisted by reinforcement learning. Finally, the performance of the proposed QLSCF algorithm is compared with that of the traditional decoding algorithms, and the results are discussed. The structure of the paper is as follows. Polar codes  are charThe output data of decoder is denoted as The hard decision function can be defined as Equation (3).To improve the decoding performance of the SC algorithm, the SCL algorithm and the SCF algorithm were provided ,4. The Ste paths . The SCLlgorithm .The SCF decoding algorithm contains a standard SC decoding and several additional flipping decoding attempts . The SCFFirst, the SCF decoder performs a standard SC decoding to obtain For the SCF algorithm based on the critical set (SCF-CS), the structure of the critical set is very important. First, the algorithm establishes the binary tree corresponding to polar codes. Secondly, the algorithm decomposes the binary tree into several subblocks whose code rate is 1. Finally, the algorithm adds the first bit of these subblocks to the critical set. The SCF-CS decoding algorithm can effectively reduce the decoding complexity compared with the SCF algorithm . AlthougReinforcement learning is used to solve the problems of how to make agents use learning strategies to maximize rewards or achieve specific goals during the interaction with the environment . The comn of MDP . The fouIf the parameters in MDP are known, dynamic programming can be used to calculate the best strategy. If not, the optimal strategy can be obtained through repeated interaction with the environment. In this paper, the Q-learning algorithm in reinforcement learning is used to obtain the optimal selection strategy during the SCF decoding. The advantage of the Q-learning algorithm is that the function The Q-learning is a value-based algorithm in reinforcement learning. There is a Q-table usually denoted by  state s . TherefoIn the proposed QLSCF algorithm, the selection of candidate bits is offline trained. Once the learning has been completed, the candidate bits obtained from Q-Table are already sorted. The decoding delay caused by the metric ordering can be removed during the decoding process. Therefore, the proposed QLSCF algorithm can reduce the decoding complexity and the decoding delay.In this section, we propose a new algorithm which uses the Q-learning-assisted SCF decoding to reduce decoding delay. In order to realize this algorithm, we give the model design of the Q-learning-assisted SCF decoding algorithm.The Q-learning algorithm can be used in the SCF decoding, because the selection procedure of candidate bits is modeled as the MDP process. Selection of candidate bits can be seen as a decision process in general. For the SCF decoding, the decision of the Q-learning is how to select one bit from candidate bits to make the decoded codeword correct.The state space In the above model, the state corresponds to SNR during the decoding, and the action of the state transition corresponds to a candidate bit. The reward function is determined by whether the decoded codeword is correct. The candidate bits set can be obtained from the Q-table. The QLSCF decoding algorithm is modeled by a single-step MDP with a Q-learning algorithm, as shown in Discrete SNR values are obtained by systematic sampling method from continuous SNR, ranging between 0.5 dB and 2.5 dB. Nine sampled values of SNR are used as states in state space The action space Algorithm 1 Optimized design of action spaceInput:The number of trials: Information bits length: Discrete SNRs: Output: The action space: Initialization:ForinWhileUpdating the received LLR: //The Ifiselse:End ifEnd whileForinIf Move End ifEnd forEnd forWe assume that the action space The reward Reinforcement learning needs the reward to reflect the effect of action execution. The simple reward function returns to “1” when the flipping decoding is successful, and if the flipping decoding fails, the reward function returns to “−1”. This would imply that an optimal policy is maximum decoding accuracy. This reward function cannot reflect the priority of the candidate bits, because every action may make flipping decoding successful. We introduce the LLR into the reward function and The exploration strategySince each action may make the bit-flipping decoding obtain the correct decoded codeword, we hope that the agent can explore full action space in the early stage of learning. Therefore, the Algorithm 2 Q-learning algorithm for the QLSCF algorithmWhileInitial observation WhileInitial information of polar codes: IfElse:End ifEnd whileEnd while// The env functionFunctionIfElse:End ifEnd functionThe Q-learning algorithm for the QLSCF decoding algorithm is described in Algorithm 2, as well as learning parameters being listed in The processed Q-table is obtained by sorting the Q-values. In the processed Q-table, each row corresponds to an SNR value. The elements in each row represent actions, in which the actions are sorted according to the Q-value from largest to smallest. This process of the processed Q-table does not participate in the SCF decoding. Therefore, this process does not increase the decoding delay of the proposed QLSCF algorithm.The detailed description of the proposed functions in Algorithm 2 is as follows.The QLSCF algorithm uses the reinforcement learning algorithm to select the candidate bits. In the SCF algorithm, the selection of the candidate bits is a decision process. Therefore, an optimal selection strategy needs to be obtained. Reinforcement learning technology can solve decision problems in complex environment and obtain optimal selection strategy through a large amount of learning. The selection of the candidate bits can be seen as the MDP process in the complex environment, so Q-learning is used to obtain an optimal selection strategy. For the conventional SCF algorithms, the values of LLR and the set of candidate bits are obtained after the SC decoding, and the values of LLR are sorted to obtain the order of the candidate bits. For the proposed QLSCF algorithm, the order of candidate bits is already obtained in the reinforcement learning. Hence, the proposed QLSCF algorithm only needs to use the sorted set of candidate bits during the decoding process, and the additional sorting process is not needed.Algorithm 3 Q-learning-assisted Successive Cancellation Flip Decoding AlgorithmInput:The received LLR: Information bits length: SNR: The processed Q-table: Output: The decoded vector: Initialization:Return// The QLSCF functionFunctionFortoIfEnd ifEnd forReturnEnd functionThe decoding process of the proposed QLSCF decoding algorithm is shown in The detailed description of the proposed functions in Algorithm 3 is as follows.In this Section, the proposed QLSCF decoding algorithm is compared with other decoding algorithms in terms of the FER performance and the decoding delay. The experimental results validate the advantage of the proposed QLSCF algorithm. All decoding algorithms are simulated with a 0.5 code rate. The maximum number of flips is equal to the size of the critical set. Error detection performance of CRC is also considered. For different decoding algorithms using the same CRC length, their error detection probabilities of CRC are the same. The longer CRC length is, the higher the error detection probability is. However, the long CRC length has a big impact on code rate of polar codes. Therefore, we make a tradeoff between the error detection probability and code rate. The CRC length for each decoding algorithm is set to 16. In the training process of reinforcement learning, the transmitted codeword is known. The binary phase shift keying (BPSK) modulation and the additive white Gaussian noise (AWGN) channel are used in this experiment.The proposed QLSCF decoding algorithm is compared with traditional decoding algorithms, including the SC decoding algorithm , the OAS and l=4 . The CASThe comparison of the candidate bits set for different algorithms is shown in In order to analyze the decoding complexity, we count the average number of additional decoding attempts for different algorithms, which are shown in When SNR is less than 1.25 dB, the average additional decoding attempts of the proposed QLSCF algorithm are less than that of both the original SCF and the SCF-CS decoding algorithm in The additional decoding attempts and the acquisition of the priority for the candidate bits cause an increase of decoding delay in the SCF decoding. In this subsection, the delay caused by sorting candidate bits are analyzed, so does the overall decoding delay. The decoding delay is affected by the number of time-steps during decoding process . The greFor the delay caused by the sorting, the absolute values of the LLR for information bits need to be sorted in the original SCF algorithm, so the decoding delay caused by the sorting is considered as The comparisons of decoding delay caused by sorting are shown in For the overall decoding delay, the delay caused by the initial SC decoding and the delay caused by additional decoding attempts need to be considered. The number of decoding time-steps caused by the initial SC decoding process is equal to 2N−2 . AdditioIn summary, the number of total decoding time-steps is equal to The number of additional decoding attempts is different for different decoding algorithms. In order to intuitively illustrate the comparisons of the QLSCF decoding, the original SCF decoding and the SCF-CS decoding algorithm, the total decoding delay is obtained by counting the average decoding time-steps. The decoding time-steps of the proposed QLSCF algorithm are compared with that of other algorithms in At low SNR (less than 1.5 dB), the average decoding delay of the proposed QLSCF algorithm is less than that of both the original SCF and the SCF-CS decoding algorithm in In At high SNR, the decoding delay of the proposed QLSCF algorithm is the same as that of the SCF-CS decoding algorithm and less than that of the original SCF decoding algorithm. The above results imply that we can apply the Q-learning algorithm to an SCF decoding algorithm to reduce decoding delay without loss of performance.The SCF decoding algorithm based on LSTM network  uses reiA Q-learning-assisted successive cancellation flip decoding algorithm has been presented, which uses the reinforcement learning assisted SCF decoding to reduce decoding delay. A reinforcement learning model for the SC bit-flipping decoding is proposed, and the SCF decoding algorithm is executed by using reinforcement learning algorithm to select candidate bits. The proposed QLSCF algorithm selects the candidate bits to decode the codeword through the learned Q-table rather than the metric sorting. For other code lengths, the proposed algorithm is also applicable. It only needs to construct the action space under the corresponding code length to learn in reinforcement learning model. The proposed QLSCF decoding algorithm reduces decoding delay without loss of performance at low SNR. Future research directions involve reinforcement learning model of the multi-bit SCF decoding and further optimization of the proposed algorithm.