This strategy can be captured by current optimization-based computational frameworks. In contrast, broad-variance learners emerge only in the unstable environment. Local analyses of the moment-by-moment fluctuations, naïve to the overall outcome, reveal an initial period of memoryless learning, well characterized by a continuous gamma process starting out exponentially distributed whereby all future events are equally probable, with high signal (mean) to noise (variance) ratio. The empirically derived continuous Gamma process smoothly converges to predictive Gaussian signatures comparable to those observed for the error-corrective mode that is captured by current optimization-driven computational models. We coin this initially seemingly purposeless stage exploratory. Globally, we examine a posteriori the fluctuations in distributions’ shapes over the empirically estimated stochastic signatures. We then confirm that the exploratory mode of those learners, free of expectation, random and memoryless, but with high signal, precedes the acquisition of the error-correction mode boasting smooth transition from exponential to symmetric distributions’ shapes. This early naïve phase of the learning process has been overlooked by current models driven by expected, predictive information and error-based learning. Our work demonstrates that  learning is a highly dynamic and stochastic process, unfolding at different time scales, and evolving distinct learning strategies on demand.The brain integrates streams of sensory input and builds accurate predictions, while arriving at stable percepts under disparate time scales. This stochastic process bears different unfolding dynamics for different people, yet statistical learning (SL) currently averages out, as noise, individual fluctuations in data streams registered from the brain as the person learns. We here adopt a new analytical approach that instead of averaging out fluctuations in continuous electroencephalographic (EEG)-based data streams, takes these gross data as the important signals. Our new approach reassesses how individuals dynamically learn predictive information in stable and unstable environments. We find neural correlates for two types of learners in a visuomotor task: narrow-variance learners, who retain explicit knowledge of the regularity embedded in the stimuli. They seem to use an  The type of highly dynamic, spontaneous, exploratory learning that is at first driven by surprise and curiosity, has no initial goal or desired target. At this early stage of learning, all future events are equally probable to the cognitive system. The learning is merely a wondering process, “what happens if I do this?”, perhaps a guess, “if I do this, then this (consequence) will ensue, otherwise, this other (consequence) will happen…”. The current work offers evidence to suggest that this endogenous and dynamic type of learning in early life may scaffold how we learn in general. That is, that before realizing that certain regularities are present in the environment we collect information spontaneously, without relying on prior knowledge, committing to some stimuli salient feature, or using referencing goals. This stage, that has so far been overlooked, is not well described by traditional models of error correction learning. These models rely on expectation and surprise minimization. However, there are situations whereby the system does not yet have referencing information to generate a prediction error or expected prediction error code.At the start of life, human babies gradually become aware of their bodies in motion and as they understand it, they come to own the consequences of impending movements that make up all their purposeful actions. Seemingly a priori imposed theoretical means under assumptions of normality and stationarity in the data registered during the learning process, we miss the opportunity to know what possible information lies in the gross data.Research about learning, whether in the perceptual, the motor, or the cognitive domain, is primarily based on error-correction schemas . These svs. exploitation in reinforcement learning (RL) (The exploratory code discussed above is not to be confused with the exploration mode that is commonly addressed in models of exploration ing (RL) . Within ing (RL) . Indeed,We here focus precisely on how the system comes to self-discover the task-goal or purpose by firstly opening information channels welcoming surprise. More specifically, we isolate the spontaneous exploratory mode of learning. This mode without expectations, or referencing signals, leads to the self-discovery of the goal or objective. To that end, we focus on the cognitive processes known as implicit or statistical learning (SL). While we recognize other influential computational frameworks such as active inference and Bayesian RL contribute to our understanding of learning in general , 2017, SImplicit SL describes the ability of the brain to extract (largely beneath awareness) regularities from the environment . Such caIn this work, we reevaluate SL from the standpoint of sensory-motor systems. We reasoned that the motor percept that emerges from the sensations of our own endogenously generated biorhythmic motions could serve to support the type of SL that other perceptual systems would experience to gain behavioral control. More specifically, we propose to reframe the SL problem using recent advances in developmental research of neuromotor control  that foca posteriori, the fluctuations in those stochastic signatures that we empirically estimated, to gain insight into the overall dynamics of the SL process that took place. For example, we track the evolution of the empirically estimated probability distributions’ shapes.To uncover the continuous dynamics of SL, we consider multiple time scales  within tWe analyze fluctuations of a continuous EEG signal, recorded during the visual search task. While we leverage the precise time stamping of the events in the data acquisition system and the use of stable and unstable implicit-learning environments , we empiThis study involving human participants was reviewed and approved by the Institutional Review Board of Tel Aviv University. The participants provided their written informed consent to participate in this study. Behavioral and ERP analyses of these data were previously published . Here weData from 70 participants  was analyzed in this study: 24 in the random group, 23 in the correlated, and 23 in the mixed groups. There were no differences in age or gender between the three experimental groups. Two participants (one in the mixed group and one in the correlated group) were removed from the analyses due to incomplete data: their EEG recording started late, missing the first few trials. As we focus here on continuous data analyses of the full learning experience, these two subjects were excluded.All participants gave informed consent following the procedures of a protocol approved by the Ethics Committee at the Tel Aviv University. The EEG signal was recorded during the visual search task. This task was followed by an explicit memory test during which EEG was not recorded. A more detailed account of the procedure can be found in Stimuli in the visual search task and the explicit memory test were white T’s and L’s . All stiParticipants searched for a rotated T (target) among heterogeneously rotated L’s (distractors) while keeping their eyes on the fixation cross. Each trial began with the presentation of a fixation cross for 2,100, 2,200, or 2,300 ms (randomly jittered) followed by an array of one of two possible targets (left or right rotated T) among seven distractors. Participants were instructed to press a response key corresponding to the appropriate target as fast as possible -i.e., the goal of the task was to be accurate as fast as possible. Each participant was randomly assigned to one of three groups, with the degree of regularity in the task varying along a gradient. At one extreme the participants searched for the target within a highly predictable environment where predefined spatial configurations of target and distractors (layouts) were repeated from trial to trial (correlated group). Presumably, the embedded regularity can be easily and systematically confirmed by the system. At the other extreme, participants experienced the least amount of regularity, as from trial to trial, the layouts of the display were generated randomly (random Group). For the third group, consistent and random layouts were mixed throughout the task (mixed group). Any regularity cumulatively built from random guesses and confirmations, thus creating the ground for self-emergence of the overall goal or purpose of the task. This task is ideal to investigate the dynamic progression of SL. The gradient of predictability enables to examine, moment by moment, stochastic variations in learning between environments that differ in the reliability of predicting and confirming a guessed regularity. Depending on the group, the visual search contained the consistent mapping condition (correlated group), the random mapping condition (random group), or both (mixed group).In summary, the three groups corresponded to predictable predictability (consistent group), predictable unpredictability (random group) and unpredictable predictability (mixed group). We were particularly interested in learning in the mixed group relative to the other two (predictable) groups.For the consistent mapping condition, spatial configurations of targets and distractors were randomly generated for each participant (8 layouts for the mixed group and 16 layouts for the correlated group). In the random mapping condition targets and distractors appeared in random locations throughout the task. The order of layouts was randomized every 16 trials . The identity of the target (left or right rotation) was chosen randomly on each trial and did not correlate with the spatial regularity. Participants completed 512 trials in the experiment. Only correct trials were included in the analysis.Participants were not informed of the regularity in the visual search task. Upon completing the task, participants in the mixed and correlated groups (when the task contained regularity) completed an explicit memory test, designed to reveal explicit knowledge of the regularity: participants saw the layouts that were presented to them during the search task mixed with new, randomly generated layouts. For each layout participants had to indicate whether they have seen the layout during the visual search task or not. We then computed an Explicit Memory Test (ET) score  that is considered to reflect each participant’s explicit knowledge of the regularity, so that higher scores correspond to better explicit knowledge .Electroencephalographic signals were recorded inside a shielded Faraday cage, with a Biosemi Active Two system , from 32 scalp electrodes at a subset of locations from the extended 10–20 system. The single-ended voltage was recorded between each electrode site and a common mode sense electrode (CMS/DRL). Data was digitized at 256 Hz  with 50% overlap of the sliding window. Here we describe the process by which these hub channels were selected. Based on previous work with the same approach we bandpass filtered the data at 13–100 Hz using IIR filter at 20th order . Two samThe analysis that is at the heart of the current work relies on the micro movements (MMS) spikes. This type of data and analytical platform, developed in the Torres lab , and patEquation 1 scales out allometric effects owing to anatomical differences . Each loThe result is then plotted, reflecting the unitless standardized MMS , which dThe Earth Mover’s Distance, EMD  was usedUpon deriving the MMS, we proceed to sweep through them using 5-s-long windows of MMS activity, with 50% overlap. This gives us a local estimation (at each window) of the stochastic process. Using MLE, we empirically estimate the shape and scale of the best PDF in an MLE sense. Examples of frequency histograms are shown in The Gamma was the best continuous family fitting the MMS in a MLE sense. The Gamma (a) shape and (b) scale parameters were then plotted on the Gamma parameter plane . The GamThe continuous Gamma family spans distributions of different shapes and different scales. Prior research has empirically characterized maturation of human neuromotor development, showing over the human lifespan a tightly linear relationship between the log shape and log scale of this family . As humaWe will use in our descriptions 1/NSR = SNR and will refer to it as the signal . here and now. At this level of randomness, prior research has shown corresponding highest levels of NSR . Such distributions are typical to the motor code at the start of neurodevelopment  these global signatures remain in the exponential range, randomly relying on the here and now and manifesting very low SNR. In this case, the system does not progress into acquiring a predictive code .For each Gamma PDF derived from the MMS in each window, the shape and scale parameters are plotted with 95% confidence intervals as points along a stochastic trajectory, on the Gamma parameter plane.  shift quadrants from moment to moment, they describe probability-positions over time (the dynamics of the stochastic process) on the Gamma parameter plane. Differentiation of this probabilistic positional trajectory yields an abstract velocity field whereby each velocity vector tangent to the trajectory, expresses the direction and the magnitude of the stochastic shift. To track the direction, we use the location of the landing point on the quadrants (the LUQ or the RLQ). The shift may leave the process in the same quadrant, or it may shift it away to the other quadrant. As shown in y-axis represents the difference between the histogram at time t and the histogram at time t + 1, taken at consecutive windows of activity. Notice that this is not physical distance. It is abstract distance in probability space. Likewise, this is not physical time, but time that depends on the length of the window and the overlapping % of the sliding window process.As the stochastic signatures a posteriori, that we can track the spikes of the EMD at a global time scale, i.e., across the entire session. This is shown in As we accumulate the above discussed stochastic trajectories, we are locally tracking the shapes of the PDFs over the empirically estimated Gamma parameters. We use EMD to trace the moment-by-moment evolution of the stochastic Gamma process, as it unfolds across all trials and blocks. But initially we are naïve to the fluctuations in this process. It is then as we contemplate the full stochastic profile, a is the shape parameter and b is the scale parameter.The general formula for the PDF of the Gamma distribution is shown below (equation 3), where a shape and b scale parameter), we can also obtain its statistical moments. We will use this alternative description of the distributions later to help visualize the results. The moments  are Although the continuous Gamma family of PDFs can be parameterized with two parameters , and incorrectly classified new layouts as familiar on 50% of the trials . In the correlated group, participants correctly classified previously seen layouts as familiar on 55% of the trials (hit rate), and incorrectly classified new layouts as familiar on 48% of the trials . For both the correlated and the mixed groups the differences between hit rate and false alarm were not significant, M = 1.37, SD = 0.9) and the mixed group  were not significantly different, F < 1.To assign a memory score (ET) we calculated the ratio between hit rate and false alarm rate for each participant. Higher scores correspond to better explicit memory of the visual layouts presented during the search task. The Overall memory scores of the correlated group (V) and inter-peak-interval timing (ms) to unitless, standardized MMS trains that were then analyzed using a sliding window of 5 s with 50% overlap (see section Methods). The window-by-window analyses for each participant revealed two subgroups in the mixed group. On the Gamma moments parameter space, along the Gamma variance dimension, one subgroup of learners (subgroup A of broad-variance learners) expressed higher variance of the fluctuations in the MMS amplitudes at the start of the experiment. This departure from the other subgroup (B of narrow-variance learners) can be appreciated individually for each participant over the entire experiment in For all three groups  we isolated the MMS from the continuous EEG data. We converted the fluctuations in the EEG amplitude .To show the overall differences in stochastic signatures of each case, we pooled the Gamma variance data from all blocks and for each mixed, correlated, and random group respectively . The mixp < 0.01 . The PDFM = 0.94, SD = 0.4 vs. 13 subjects in the narrow variance group subgroup B, M = 1.52, SD = 0.75, p < 0.01 non-parametric Wilcoxon ranksum test . Here the mode refers to learning mode or phase and in the next results, we provide a stochastic characterization of these two fundamentally different modes of learning which, nevertheless, converged in block 8 to a similar variance range.The two subgroups broad-variance A and narrow-variance B of the mixed group did not differ in reaction times or accuracy, suggesting that all participants were able to reach the same level of online performance. Instead, they were differentiated by their explicit knowledge of the regularity imbedded in the task, as reflected by their memory scores in the explicit memory test: 10 subjects in the broader variance subgroup A, sum test . The subsum test . We coinM = 1.37, SD = 0.9) were like the scores observed in subgroup B of the mixed group. This result is consistent with the similar stochastic learning signatures of the correlated group and this high memory subgroup , we were naïve to the global dynamic nature of the stochastic Gamma process, as we were locally estimating the Gamma parameters  and the Gamma moments. Upon estimation of the full stochastic trajectory across the entire session, trial by trial and block by block, we are no longer naïve to the process. As such, we can make a global statement at the time scale of the entire session.Among the moments of the distributions of the shape parameter, the variance of the evolving Gamma PDF shape parameter revealed the separation of the mixed group from the random and from the correlated groups . FurtherWe show the stochastic shifts of each of the error correction (lower Gamma shape variance and higher explicit memory test score) and exploratory (higher Gamma shape variance and lower explicit memory test score), as they unfold across the blocks.The empirically estimated Gamma family shape parameters of the subgroup with high explicit memory scores (subgroup B) starts in the symmetric Gaussian range but trends down and converges towards the skewed, heavy tailed distributions, shown in y-axis, is larger as learning occurs, broadening the bandwidth of distribution shapes as learning takes place. The switch from exponential to heavy tailed to Gaussian distributions reflects the more systematic confirmation of a regularity in the stimuli. Initially, all future stimuli are equally probable , but in time, correct prediction of futures events increases, consistent with the transition from a detected regularity to a systematic goal. Once a goal is in place, error correction is the learning regime reflecting Gaussian predictive process embedded in this overall Gamma process. Here is where we see a tendency to symmetric shapes approached by both modes along the horizontal axis of the Gamma parameter plane. One mode (the exploratory) approaching it from the left, away from the memoryless exponential. The other approaching it from the right.Notice here that we are capturing the distribution of the fluctuations in the estimated Gamma shape parameter with a Gamma process as well. We are referring to the Gamma shape and Gamma scale parameters of the distributions derived  from the fluctuations in Gamma shape of the MMS derived from the EEG hub channels. On this Gamma parameter plane, the dispersion  along the The stochastic transition depicted in In a posteriori, across the entire experimental session) we assessed the change in stochastic variations of the signals over time. To do so, we examined the evolution of the fluctuations in the change of Gamma distributions’ shapes using the Earth Movers Distance (EMD) metric  the self-discovery of a goal through self-referencing and (ii) transitioning to the error-correction mode. Eventually the latter can lead to fast and accurate performance. To aid interpreting these results, we leverage prior research on the broad characterization of human biorhythmic activity  provide the most opportunity for learning, as characterized by higher SNR on both the global and local levels of analyses. Next, we show that on an individual basis, this unstable environment may give rise to different learning profiles: within this mixed group, two subgroups of participants self-emerged from the analyses. For one subgroup-B, coined error correction mode, the learning profile shows narrow variance in the MMS from start to finish and higher explicit memory test scores, reflecting better recall of the regularity. However, for the second subgroup-A, coined exploratory mode, the learning profile reflected an early stage of broad variance and memoryless learning which later converged into the signatures of the error correction mode. Crucially, this subgroup showed lower scores in the explicit memory test, as they did not recall the regularity with the degree of accuracy of the other subgroup. In their initial learning performance, all future events were equally probable, without a bias towards a particular regularity being reliably noted or recalled. We now turn to discussing each of these results in detail, while considering their implications on our understanding of SL in general.When comparing the stochastic signature of learning within an unstable environment mixing the stimulus regularity between random and correlated trails (mixed group) with stable conditions providing reliable regularity (correlated and random groups), the process proved to be less stationary, more predictable in nature, and was characterized by higher SNR. These characteristics suggest that complex environments provide higher opportunity to learn than reliable environments. Moreover, within our theoretical framework, higher SNR corresponds to more efficient learning. These results are consistent with neuroimaging studies that have identified brain systems that track uncertainty in a curvilinear U-shaped function . Within Given that the real world is indeed complex, with our cognitive system continuously bombarded with variable regularities, it seems natural that we should be more attuned to learning under relatively unreliable (yet richer in information) conditions. However, suggesting that learning under such conditions is more efficient may seem to contradict the behavioral pattern previously observed in these data: participants in the mixed group reached slower RTs than both in the random and correlated groups  distribution shape unfolded through the Gamma process itself.The present work goes beyond assumptions of normality, linearity and stationarity in the data and exploits the moment-by-moment fluctuations that prior work discards as gross data. Embedded in that gross data we uncovered the phase transitions in probability space that distinguished two fundamentally different modes of learning and revealed one  that converges to the other (predictive Gaussian). Both modes are well characterized by the continuous Gamma family of PDF s at the local level, when we are naïve to the upcoming moment-by-moment distribution, and at the global level, when here and now.” Presumably, this exploratory stage was elicited by the high levels of surprise in an environment that contained rules that were not followed consistently over time. Crucially, this subgroup A also exhibited low scores in the explicit memory test. We posit that for participants in the narrow-variance subgroup B showing higher level of explicit knowledge, the stochastic signature reflected an error correction mode of learning throughout, from the beginning to the end of the task.For a cohort of participants, the unstable environment (mixed group) triggered an initial stage of memoryless exploratory learning. During this stage, the stochastic signature of the process reflected a type of learning whereby initially all future events were equally probable. The stochastic signature unveiled in this initial period of learning for the broad-variance subgroup A of this cohort, suggests that the system was not relying on prior knowledge but was instead gathering as much information as possible from the “The behavioral differentiation between subgroups A and B, suggests that the transition from exploratory behavior into error correction depends on some minimal level of explicit knowledge that needs to be obtained. This conclusion contradicts the current assumption that both explicit and implicit SL always reflects error correction . For insvs. spontaneous movement classes  the minimization of predictive error or the consequences of predictive error. Crucially, the fact that the transition between the memoryless exploration mode and the error correction mode could be predicted from an independent assessment of behavioral data  lends a predictive validity to our analysis of the neuronal correlates of a new aspect of learning. Only after a goal self-emerges it can be incorporated into an objective function or model, transitioning from trial-and-error model-free, to error-correction model-based learning, as an objective function gets defined. At that stage, minimizing expected surprise, as in active inference, fits well with the error-correction phase that all participants eventually converged to. However, active inference, as other learning frameworks, will need to be modeled differently from its current conceptualizations of optimal expectation-driven exploration to include the newly discovered spontaneous and memoryless stage of learning.In active inference, behaviors are chosen to maximize both expected value and expected information gain  . Statistspontaneous autonomy  exploratory mode described here, scaffolds the emergence of what we have coined autonomy , differeautonomy . We posiWe have in summary shown that using new analytical techniques, we can get a precise characterization of the dynamic nature of SL, the rich stochastic signal embedded in fluctuations that are traditionally treated as gross data and the differential nature of contrasting learning modes. Investigation is warranted on whether these results generalize to other SL paradigms, and to the acquisition of predictive information in learning in general. Of particular interest, are questions of individual differences, and the degree to which the exploratory and error correction learning modes may be differently recruited on demand by the same learner under different contexts. We here offer methods that allow to investigate these and many new questions in future SL research from the perspective of the nascent, developing motor systems and their richly layered dynamic and stochastic motor percepts.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.The studies involving human participants were reviewed and approved by Institutional Review Board of Tel Aviv University. The patients/participants provided their written informed consent to participate in this study.AV ran the experiment, recorded the original EEG data, and analyzed the behavioral results. ET developed the novel analytical tools, analyzed the EEG data, and provided the statistical inference/interpretation from prior empirical and computational work. Both authors contributed to the conceptualization of this work, interpretation of the results, and the preparation of the manuscript. This work merges the motor control dynamic and stochastic perspective, brought in by ET, with the study of statistical learning, brought by AV.


However, the clinical auscultation of the masticatory system still lacks computer-aided support, which would decrease the time needed for each diagnosis. This can be achieved with digital signal processing and classification algorithms. The segmentation of acoustic signals is usually the first step in many sound processing methodologies. We postulate that it is possible to implement the automatic segmentation of the acoustic signals of the temporomandibular joint (TMJ), which can contribute to the development of advanced TMD classification algorithms. (2) Methods: In this paper, we compare two different methods for the segmentation of TMJ sounds which are used in diagnosis of the masticatory system. The first method is based solely on digital signal processing (DSP) and includes filtering and envelope calculation. The second method takes advantage of a deep learning approach established on a U-Net neural network, combined with long short-term memory (LSTM) architecture. (3) Results: Both developed methods were validated against our own TMJ sound database created from the signals recorded with an electronic stethoscope during a clinical diagnostic trail of TMJ. The Dice score of the DSP method was 0.86 and the sensitivity was 0.91; for the deep learning approach, Dice score was 0.85 and there was a sensitivity of 0.98. (4) Conclusions: The presented results indicate that with the use of signal processing and deep learning, it is possible to automatically segment the TMJ sounds into sections of diagnostic value. Such methods can provide representative data for the development of TMD classification algorithms. Acoustic signals collected on the surface of the body carry diagnostic information and can be successfully used in the initial evaluation of various body parts, such as joints. In particular, auscultation is used in the diagnosis of the masticatory system. Conducted studies have shown that healthy temporomandibular joints (TMJ) work either noiselessly or generate only the expected physiological sounds. Thus, any unexpected noise in the TMJ acoustic signal can be used as an indicator of temporomandibular disorders (TMD) . It is bAccording to studies, masticatory dysfunctions statistically affect up to 20% of the population ,4. TMJ dOne of the crucial steps of every biomedical signal-processing method is segmentation, which can be defined as the process of finding sections of diagnostic value in the repetitive signal. In the case of masticatory system auscultation, the patient is asked to periodically open and close the jaw to generate diagnostic segments representing the mandible movements. A phase between each representative movement is not relevant to the diagnosis and makes further signal processing less effective and prone to noise artifacts. Automated segmentation aims to extract only the abovementioned representative sections.Based on the literature review, we found a lack of references concerning the segmentation of the TMJ auscultation signal. Phonocardiography, on the other hand, constitutes a field which has been studied in depth, and is well recognised in literature in the context of heart-sound segmentation. Therefore, we believe that it can become a reference for the processing and segmentation of sounds generated by other physiological systems, such as the masticatory system.Sun et al.  obtainedAmong different designs of CNNs, U-Net is one of the top fully convolutional network architectures for medical image segmentation tasks ,16,17. IIn contrast to 2D data segmentation, one of the most popular neural network architectures in the field of acoustic modelling and speech recognition are recurrent neural networks, mainly long short-term memory (LSTM) models. Zhang and Lu  proved tBased on the conducted review of available literature, we found no references which mentioned the existing TMJ sound database. This is why we assume there is a lack of documented open-access repositories of masticatory system acoustic signals. Therefore, in this study, we used our own database created from the sounds recorded with an electronic stethoscope  during a clinical diagnostic trial of TMJ performed in the Dentistry Clinic of Jagiellonian University Medical College in Krakow, Poland (consent of Bioethics Committee number 1072.6120.71.2019). All patients were first diagnosed with standardised official Diagnostic Criteria for Temporomandibular Disorders (DC/TMD) ,22,23 anThe first step of the developed segmentation algorithm is the determination of the signal envelope. In principle, there is no single formal definition of a signal envelope, even though the exact mathematical equivalent of this term is an envelope understood as the limitation of a specific family of curves . In pracIn order to emphasise the local maxima, the signal subjected to the moving average was then squared, amplified twenty times and filtered again with a low-pass filter of a moving average character in order to eliminate local fluctuations. To localise diagnostic segments, it is necessary to determine the local maxima of the signal obtained in the previous step. This procedure was based on the assumption that the minimum distance between successive peaks is half of the time window, calculated on the basis of the main frequency (the frequency with the highest energy) of the original TMJ acoustic signal. The final step of the algorithm is proper segmentation, which consists of determining the boundaries of the episodes of the masticatory system activity based on the calculated local maxima, assuming that the opening and closing of the mandible is included in 80% of the amplitude of each peak of the envelope. A diagram of the proposed method is presented in The algorithm was tested on patients who suffer from TMD and also on healthy subjects. In both cases, as presented in One can find a variety of research in which the authors take advantage of neural networks, especially convolutional and recurrent networks, in the analysis of different kinds of acoustic biomedical signals ,29,30. TConvolutional neural networks consist of convolutional layers which can be considered as a set of filters with parameters (weights) adjusted during training by back-propagation algorithm . A serieNeural network architectures for specific problems are usually designed on the basis of the current state-of-the-art, as there are no formulas for designing architectures. Since 2012, when the AlexNet model was presented , many exOne can distinguish two parts of the U-Net’s architecture. The first part consists of convolutions and pooling layers extracting the signal’s local features. Because each convolutional layer is composed of multiple filters, it outputs a set of signal representations. Moreover, after each convolutional layer, there is a pooling layer which discards some of the processed samples, depending on the specific strategy. Setting up a pooling layer after a convolutional layer is a common practice in designing a neural-network architecture ,35,36,38In order to restore the original signal length, the second part of our model extends signal representation to match the original length by placing zeros evenly between existing values and convolving them in order to fill the missing gaps. This operation is possible because of filters for which the parameters are determined during the backpropagation optimisation process. Finally, the output has the same length as the input, but instead of signal amplitude, each value represents belongingness to a class (segment or non-segment).A series of experiments were conducted as far as the network’s architecture is concerned. In the beginning, we started with an original U-Net architecture, which provided no promising results during training because the loss function value was not decreasing. An analysis of feature maps provided us with the conclusion that some important signal features were lost, probably due to max pooling operations after each set of convolutions. For that reason, we designed a min-absolute-pooling layer  which enx is a signal, and M is max-pooling.The min-absolute-pooling operation can be represented as (1):Because acoustic signals are time series in their nature, another modification in U-Net’s architecture was introduced. Instead of simply up-sampling the middle layer, we decided to utilise recurrent layers such as LSTM. Having data in three-dimensional tensor  we passed it to an LSTM layer where the time series was the second dimension  and the features were feature maps. This modification increased the average performance of the resulting model by 5% in sensitivity and 12% in specificity. As a result of numerous experiments, we proposed the final architecture of the model. The model consists of two convolutional blocks and three LSTM layers in the middle of the network. Each convolutional block consists of two convolutional layers, max pooling and min-absolute-pooling, which are set up in parallel. The output of each layer is concatenated and then passed to another convolutional block. The second part of the network—the up-sampling block—was left as it is in the original U-Net architecture. In order to justify a design of our model, we decided to visualise outputs from intermediate steps: convolutions and pooling (pre-processing), recurrent analysis with LSTMs, processed signal up-sampling, and the classification of the single input sample.The original dataset was split into a training set, a validation set and a test set at the very beginning of the experiment. Splitting was done patient-wise, not signal-wise, in order to avoid a situation when signals from the same patient are eventually placed in more than one of the aforementioned sets.ty is a vector of example signal samples’ reference classes, py is a vector of the example signal samples’ predictions, * and + operators applied on vectors are element-wise multiplication and addition.To evaluate the whole test set in the described way, we used the following metrics: sensitivity (2), positive predictivity (3), and Dice coefficient (4). The calculation was performed based on confusion matrix classes such as true positive (TP), true negative (TN), false positive (FP), and false negative (FN).Moreover, specificity (5) and negative predictivity (6) were calculated in order to provide a better evaluation of the results, especially for the negative class (outside of the segment):−4 at the beginning, which was then decreased by a tenth every 400 epochs. The input of the network was defined as 16,000 samples (eight seconds) of the signal because this appeared enough for the model to extract all necessary information . As the majority of signals were much longer than eight seconds, we introduced a regularisation technique in order to avoid overfitting by randomly choosing an eight-second-long section of each training example for every epoch. As a result of this approach, the network was not exposed to the same example more than once throughout the whole training process. Such an approach enabled the network to learn features which best described segmented episodes instead of just memorising training examples by heart. The deep neural network model was trained for 1200 epochs with a learning rate of 10The drop of validation error in In order to evaluate the model, we needed to supply the eight-second network input with signals of various lengths. To achieve this, we simply split each signal into eight-second intervals and fed them one after another to the network’s input. We then merged each result into the vector of the length of the original signal.To finish, we calculated the metrics defined in As presented in Nonetheless, the achieved evaluation scores indicate that both of the proposed solutions are potential options in the field of the segmentation of TMJ acoustic signals. The expected expansion of the dataset should improve the results even more.Despite having been in clinical use since the nineteenth century, the stethoscope still constitutes an indispensable diagnostic tool for most physicians . The simCardiac auscultation and related heart-sound segmentation methods are well established clinical and scientific subjects. Thus, in this paper, we utilised phonocardiography as a reference when developing the presented segmentation methods of TMJ acoustic signals. As it turned out, DSP techniques and LSTM-based models can provide promising results in both fields.The most significant difference between the proposed deep learning approach and the DSP algorithm is that the first method attempts to mimic the dentist’s perception of TMJ sound diagnostic segments, whilst the second method is theoretical and thus more objective. There is still room for the improvement of both methods. The deep learning architecture may be refined, and the DSP algorithm might be supported by time-frequency signal features. The advantage of the latter is, however, the ease of interpretation, as deep learning models are obscure and nondeterministic in their nature, require an empirical approach, and usually entail a long training process.Disorders of the masticatory system are common throughout the population; however, the related diagnosis based on auscultation lacks comprehensive computer-aided support. In this paper, we attempt to fill this niche by presenting two different approaches for the automated segmentation of TMJ acoustic signals. The obtained results suggest that both methods display potential in medical applications, even though the research utilised only a limited dataset. This conclusion confirms the hypothesis that it is possible to develop automated segmentation methods of TMJ acoustic signals.More acoustic signals collected during clinical TMJ trials would enable the refinement of the proposed algorithms and make them applicable in the development of dedicated TMD classifiers, which in turn may support dentists in the time-consuming TMJ diagnostic process. However, the database created in our research may constitute a reference for further studies and the design of new TMJ sound-processing methodologies.We hope that our study will contribute to the development of new TMJ sound analysis methods and raise awareness that acoustic diagnosis of the masticatory system is important and should be supported by novel computational methods, as it is done in the case of cardiovascular auscultation.


In recent years, robotic minimally invasive surgery has transformed many types of surgical procedures and improved their outcomes. Implementing effective haptic feedback into a teleoperated robotic surgical system presents a significant challenge due to the trade-off between transparency and stability caused by system communication time delays. In this paper, these time delays are mitigated by implementing an environment estimation and force prediction methodology into an experimental robotic minimally invasive surgical system. At the slave, an exponentially weighted recursive least squares (EWRLS) algorithm estimates the respective parameters of the Kelvin–Voigt (KV) and Hunt–Crossley (HC) force models. The master then provides force feedback by interacting with a virtual environment via the estimated parameters. Palpation experiments were conducted with the slave in contact with polyurethane foam during human-in-the-loop teleoperation. The experimental results indicated that the prediction RMSE of error between predicted master force feedback and measured slave force was reduced to 0.076 N for the Hunt–Crossley virtual environment, compared to 0.356 N for the Kelvin–Voigt virtual environment and 0.560 N for the direct force feedback methodology. The results also demonstrated that the HC force model is well suited to provide accurate haptic feedback, particularly when there is a delay between the master and slave kinematics. Furthermore, a haptic feedback approach that incorporates environment estimation and force prediction improve transparency during teleoperation. In conclusion, the proposed bilateral master–slave robotic system has the potential to provide transparent and stable haptic feedback to the surgeon in surgical robotics procedures. Master–slave teleoperated robotic systems are being applied to a wide range of fields, from the deep sea to space exploration and from mining to medical industries. Robotic minimally invasive surgery (RMIS) is one such field benefiting from the improved levels of control offered by teleoperated systems. Teleoperated robotics are capable of delivering improvements in motion scaling, hand tremor filtering, and hand-eye co-ordination, as well as enabling the automation of specific quantifiable actions, such as suturing ,2,3,4.Bilateral teleoperators are teleoperated systems capable of providing force feedback or haptic feedback to the operator ,6. PreseThe difficulty in providing haptic feedback for surgical applications stems from the fact that traditional bilateral controllers are unable to provide high levels of transparency and stability simultaneously . Ideal oThe literature contains a wide range of algorithms and approaches designed to address the issue of communication time delay in bilateral teleoperations. The majority of these approaches are based on the passivity control theory ,21. UsinThese restrictions have prompted the use of predictive control in place of the approaches mentioned above and direct force feedback methodologies ,27,28,29A similar predictive approach involves modelling the environment on-line, using estimation methods such as the Recursive Least Squares (RLS) or Exponentially Weighted Recursive Least Squares (EWRLS) ,38,39. BThe Kelvin–Voigt (KV) force model, consisting of the parallel connection of a linear spring and damper, has been widely used as an underlying environment force model ,41. HoweThe Hunt–Crossley (HC) force model demonstrates more consistency with the dynamic behaviour of soft bodies . PalpatiIn cases where remote environments included materials with different rigidity, the researchers also investigated a hybrid approach using a combination of KV and HC models called the threshold contact switching model (TSCM) . For adaHuman-in-the-loop teleoperation presents additional challenges, particularly when an estimation of the environment is desired. When the slave mechanism’s motion is driven by a human-controlled device , the question of persistency of excitation becomes important. Does the environment estimator receive enough varying information  to accurately estimate the contact environment’s parameters? Additionally, the feedforward/feedback nature of the bilateral controller means that teleoperator performance needs to be evaluated holistically. The goal of this work is to experimentally verify the performance of an on-line environment estimation–force prediction methodology; to evaluate teleoperator performance as a whole. It also aims to demonstrate that transparent teleoperation can be achieved with an estimated virtual environment, which is interacted with by the haptic master device.The main contributions of this research are as follows: first, development, characterisation, and experimental validation of a high-transparency predictive force-feedback methodology. In this study, an environment estimation–force prediction control architecture is developed and implemented into a three-channel bilateral teleoperated system. The use of a force predictor means that neither kinematic nor force correspondence is required to be strictly maintained. Secondly, investigation of the human-in-the-loop teleoperation transparency performance of the bilateral robotic system by studying various force modelling approaches in isolation, including a spring-based adaptor, KV, and HC force models. Lastly, a metric for transparency is developed that accounts for the positional lag between slave and master systems. By using the current slave position as a reference, and comparing forces at a prior comparable master position, an accurate measure of error can be determined on the fly.The remainder of this paper is organised as follows: An overview of the proposed control structure for the estimation–prediction-based force feedback methodology is presented in The Kelvin–Voigt (KV) linear contact model  is a comThe KV model has several physical inconsistencies that make it non-ideal for modelling soft-contact environments. A non-zero velocity at the boundary ln[Fes) indicates the use of the slave-specific quantities. The error term, The derivation of  can be fThe linearised force models are parametrised in the form of , where pk. The linearised vectors to  algorithm is particularly useful for environments with varying parameters, as the relative weight that past estimates have on the current estimate can be specified and adjusted as necessary. The EWRLS update algorithm is defined as:P is the covariant matrix, and Here, Using the predictive force approach, the EWRLS algorithm uses the measured slave dynamics , as they do not act in such a way as to produce or inject additional energy into the system; thus, nd Fe*=0 .The linear time-invariant (LTI) dynamics of the above system are:The impedance experienced by the operator is defined as:Lawrence  defines Equation  translatimpedance matching and is a consequence of the sought-after kinematic correspondence (force reflection (14b) between the master and the slave:This approach is known as pondence  and forchybrid matrix can be developed, as shown below:With these definitions, a ideal transparency is defined as :Using the kinematic and force correspondence conditions in , and the hybrid matrix of , the ideThe proposed bilateral controller has two definitions of error that can be used to evaluate teleoperator transparency performance: estimation error and prediction error, each with a definition describing the accuracy of different processes. The estimation error is localised to the slave–environment interaction and indicates the accuracy of the estimated parameters in reference to the measured forces at the slave. The prediction error relates the interaction between the master and virtual environment to the interaction of the slave with the measured environment. The prediction error encompasses the master’s dynamics, which typically extends beyond the range of the slave–environment dynamics. As such, the prediction error provides an indication of system transparency, whereas the estimation error indicates estimator transparency.Prediction error is the error between the predicted master force and the measured slave forces at a given position inside the real and virtual environments. It is used as a metric for transparency, given that it evaluates how well the predictive force feedback mechanism is operating in terms of the teleoperation system as a whole.A method for calculating the prediction error involves comparing the measured current force with the predicted master force at an unknown time in the past. This is because, at each time-step, the slave and the master are typically at different positions within the real or virtual environment and, thus, are expected to experience different forces and cannot be compared directly. Instead, the forces are compared when the slave and the master kinematics are as similar as possible.The prediction error, Considering the slave position  was gradually pressed perpendicularly into a soft polyurethane foam until an environment force of approximately 3 N was observed. From there, the slave oscillated up and down while in contact with the foam for several tens of seconds until fully retracted from the material. In the current stage of the experiments, force prediction and feedback have been limited to one DOF. Therefore, only the Omni’s Initial values for the EWRLS estimator were kept constant for each experiment and are listed in Direct force feedback from the slave to the master;A spring-based adaptor acting between the slave and master;Virtual environments from the EWRLS estimation–prediction methodology, with both the Kelvin–Voigt and Hunt–Crossley force models.Experiments were conducted using:initial insertion, palpation, and retraction phases. This figure demonstrates that prediction error increases significantly once palpation begins, ranging between directly from the measured slave force, is intrinsically tied to the slave position. When there is a substantial offset between the slave and master positions, poor transparency follows.The position trajectories of the master and slave are presented in The root-mean-squared error (RMSE) and mean prediction error during palpation for the three trials are given in The variation in RMSE seen across each trial is due to different teleoperation speeds used while controlling the system. For example, Trial 1 had comparatively faster motions of greater displacement when compared to Trials 2 and 3, which resulted in greater feedback error. adaptor  when theRMSE and mean prediction error for each trial of this experiment are given below in The prediction error seen in The damping coefficient, The RMSE and mean prediction error for each of the three trials is presented in Similar to the direct-feedback methodology, the variation seen in the RMSE values above is due to the different speed and range of motions used during each trial, a consequence of human-controlled teleoperation. Trial 2 had the highest average speed of motion (3.9 mm sTrial 1, which has the lowest RMSE of the three trials, had a more restricted motion range compared with the subsequent trials. The maximum environment force recorded for this trial was 2.51 N. Trials 2 and 3 had maximum forces of 3.31 N and 3.40 N, respectively. The Kelvin–Voigt estimator is less effective over larger ranges of motion as the polyurethane foam’s non-linearity becomes more pronounced. As such, the Kelvin–Voigt model is more accurate during Trial 1, as evident by the considerably lower RMSE.Overall, the virtual Kelvin–Voigt model demonstrates worse transparency for the polyurethane foam than the spring-based adaptor. However, when the motion range is more constricted and slower, as seen in Trial 1, transparency is slightly improved.n is able to capture the behaviour of the polyurethane foam, and the master force profile is well contained within the slave profile. Additionally, the position-dependent damping term removes the non-zero force at The RMSE and mean prediction error for the three trials are given below in Teleoperator performance was evaluated by analysing the prediction error, the results of which are presented in In this paper, an environmental estimation and force-predictive bilateral controller were presented using an experimental investigation with a human-in-the-loop palpation task. The controller uses an EWRLS estimator at the slave to generate an estimation of the environment parameters. Force feedback at the master is then predicted using these estimated parameters. By replacing the force-feedback channel with the estimated parameters, issues related to communication time delay between the two subsystems can be alleviated. The experimental results demonstrated significant improvements in teleoperator transparency when the environment is modelled on the non-linear Hunt–Crossley force model.There are a number of extensions that can be made to this study. Future work will focus on improving the speed of convergence and parameter stability within the environment estimator. The robotic platform is designed to perform more complex four DOF motions with the potential of articulating the end-effector for additional DOFs, but experimental verification was limited to a single DOF at this stage of the study. Future research aims to expand the system to its full four-DOF capabilities. There is also a possibility that the results may be affected by the extent to which the subject is familiar with the teleoperation system. In this study, experiments were conducted by a subject familiar with the robotic system. This was a validation and verification of the teleoperation architecture aimed at confirming whether the estimation–prediction models could be successful in achieving transparency. In a future study, the learning curve of the teleoperation task and the transparency results of users with different levels of familiarity with the system will be evaluated. Furthermore, the experimental verification conducted in this paper was aimed at determining the performance of a single force-feedback model within a single homogeneous soft environment. Studies will be conducted in the future that examine uneven environments with different rigidities at different sections. The study of predictive force feedback with flexible end-effectors would also be a potential future direction for this research. Finally, using vibrotactile feedback  can impr


Aiming to eliminate the hump phenomenon in a low specific speed centrifugal pump, its structural parameters were optimized using the computational fluid dynamics method. Based on the  Using a low specific speed centrifugal pump, it is easy to produce an unstable hump phenomenon under the low flow rate condition. This, in turn, increases vibration and noise, thus shortening the pump service life and reducing its working reliability. Currently, the hump phenomenon mechanism of low specific speed centrifugal pumps is not clear, and the hump phenomenon of the head curve cannot be eliminated from the design. Therefore, besides studying the centrifugal pump working mechanism, it is also necessary to optimize the critical pump structural parameters to improve its working performance.A low specific speed centrifugal pump is a type of centrifugal pump with a specific speed ranging between 20 and 80. It is characterized by low flow, high head, and low volume, and is widely used in production and life2 established ten design schemes and performed numerical simulation and performance prediction on low specific speed centrifugal pumps. The authors obtained distributions of static pressure, streamlines, velocity, and turbulent kinetic energy in the pump, and improved the internal flow characteristics. Zhang et al.3 used the SAS turbulence model to carry out the full-channel numerical simulation of the pump-turbine, determining the effect of the pump flow structure mechanism on the hump characteristics. Zhang Peifang et al.4 analyzed the hydraulic performance of low specific speed centrifugal pumps, observing the cause of the head curve hump phenomenon aiming to propose a solution. Li et al.5 used the three-dimensional constant value simulation equation to design the pump blades. Further, they analyzed the impeller flow characteristics and found the pump energy discharge characteristics in the hump zone mode. Chen et al.6 experimented by adding two partitions in the pump suction section. Experimental results have shown that the proposed method could effectively improve the IS centrifugal pump performance curve and eliminate the head curve hump.For a long time, reducing and eliminating the hump phenomenon of centrifugal pump head curve with low specific speed becomes an important segment of centrifugal pump research. Zhang Desheng et al.14. In this paper, a subtype of the centrifugal pump was used as the research object, and the CFD method was used to calculate the internal flow field of the centrifugal pump. Accounting for the influence of various structural parameters on the pump performance, structural parameters were optimized via orthogonal tests. The centrifugal pump head, shaft power, and efficiency were calculated using range analysis. The optimal parameter combination was obtained through weight matrix analysis. A centrifugal pump flow control test bench was built to verify the design method. Finally, the experimental results have confirmed that the centrifugal pump performance indicators improved after carrying out the surface optimization; the head curve humps were eliminated, and the desired effect was achieved.With the continuous development of computational fluid dynamics (CFD), numerical simulations have become one of the most important methods to study the internal flow field of centrifugal pumpsThe structural parameters of a centrifugal pump are as follows: flow The centrifugal pump calculation domain is shown in Fig. During the operation, the centrifugal pump is affected by the impact loss. There is a difference between the theoretical Further, in Eq. , \documeBy sorting equations given above, theoretical centrifugal pump head 15. When aiming to find the curve without a hump phenomenon, its slope should be increased. Using Eq.  is as follows:The second objective function weight matrix (shaft power) is as follows:The third objective function weight matrix (efficiency) is:Finally, the total weight matrix of the orthogonal test objective function is the average of the weight matrices of three objective functions:According to the weight matrix calculation results, factor influences on the orthogonal test objective function are ordered as To verify the optimization model feasibility, the full-flow CFD numerical simulation of the optimization model was carried out. The outputs were compared to results for the prototype pump.Counter-clockwise numbering was used near the tongue flow passage (No. 1 flow passage). The internal pressure surface speed of each flow passage was greater than the suction surface speed. Additionally, the outlet speed reached the maximum value in the last flow passage. Due to the hydraulic impact between the flow passage and the tongue, the liquid flow direction changed. The relative velocity distribution at the intermediate centrifugal pump interface is shown in Fig. When the small flow rate condition is active, the low-speed zone is mainly found on the blade pressure surface. Simultaneously, the liquid flow direction is disorderly between the blade inlet and outlet. Once the flow rate drops under For the optimization pump, the degree of liquid flow direction disorder can be optimized above the A flow control test bench was built based on the NGL002 centrifugal pump test device to verify the accuracy of optimization results, as shown in Fig. The prototype and the optimization pump impellers are shown in Fig. The experimental results containing the prototype and the optimization pump external characteristics are shown in Figs. 19.As can be seen from the test results, the variation law of the external characteristic curve practically remains the same despite varying working conditions. The numerical simulation results are in good agreement with the experimental values, reflecting the changing trend of all indexes under different flow rates. The head and efficiency obtained via numerical simulation are higher than the experimental values, mostly because the energy loss of each part and the impeller manufacturing errors were not considered in the numerical simulation. Using the verification results obtained for external characteristics, it can be seen that the optimization pump performance index is higher than that of the prototype. The The external characteristic test results were extracted for various indexes, as shown in Table Based on Table Hydraulic design of low specific speed centrifugal pump was carried out. Three-dimensional impeller and volute models were created.Nine groups of test schemes were designed using the orthogonal test, and the influence order of each factor on each of the indexes was obtained through range analysis. Further, by using the weight matrix analysis and the weight relationship between the factor levels, a set of optimization models were obtained: The centrifugal pump flow control test bench was built and the simulation and test values of each prototype and optimization pump index were obtained under different working conditions. Test results have shown that the performance indexes were higher in the optimization pump, eliminating the hump phenomenon and improving its hydraulic performance. Accuracies of the design process and optimization method were further verified.In this paper, a type of centrifugal pump was taken as the research object and hydraulic design was carried out based on the CFD. The main structural parameters were optimized via an orthogonal test. A test platform was built to enable carrying out verification tests, which verified the orthogonal test scheme accuracy and improving the working performance of the centrifugal pump. Compared to the prototype pump, the outlet impeller diameter of the optimized pump was increased, while the impeller outlet width, the number of blades, and the blade outlet angle were smaller. With the increase in the impeller outlet diameter and the decrease of its width, the flow passage area increased, reducing the hydraulic losses. In that case, the overload would be avoided, and the "jet-wake" phenomenon and hump would be eliminated. At the same time, with the decrease in the number of blades and the blade outlet angle, the area of each flow channel increased. Thus, the static pressure at the blade outlet decreased, mostly to avoid local backflow and flow separation. It can be seen that, by optimizing the structural centrifugal pump parameters, the pump head was reduced, and the internal flow stability and working efficiency were improved. Thus, a series of negative problems such as vibration, noise, and high energy consumption of pump and pipeline systems caused by the hump phenomenon could be mitigated effectively. In the next step, multi-objective optimization design will be carried out around the external characteristics of centrifugal pump head, shaft power and efficiency, focusing on in-depth research on centrifugal pump flow-induced vibration noise, in order to comprehensively improve the working performance of centrifugal pump.


Commercially available dry powder inhalers (DPIs) are usually devices in a fixed combination with the intended formulation, and a change in medication by the physician often forces the patient to use a different device, requiring the patient to relearn how to use it, resulting in lower adherence and inadequate therapy. To investigate whether DPIs can achieve successful outcomes regardless of the formulation and flow rate used, a novel DPI and two commercially available devices were compared in vitro for their deagglomeration behavior for different binary blends and a spray-dried particle formulation. The results demonstrate that the novel device achieved the highest fine particle fraction (FPF) regardless of the formulations tested. In the binary mixtures tested, the highest emitted fraction was obtained by shaking out the powder due to the oscillating motion of the capsule in the novel device during actuation. For DPIs with high intrinsic resistance to airflow, similar FPFs were obtained with the respective DPI and formulation, regardless of the applied flow rate. Additionally, the development and use of binary blends of spray-dried APIs and carrier particles may result in high FPF and overcome disadvantages of spray-dried particles, such as high powder retention in the capsule. Dry powder inhalers (DPIs) are widely used for the therapy of lung diseases such as asthma, chronic obstructive pulmonary disease (COPD), or bacterial infections ,2. To acSince the success of inhalation depends on many factors, as described, marketed delivery systems are designed as combination products consisting of the formulation and the DPI to minimize the number of potential sources of error. The formulation and the device are designed to work together to achieve a satisfactory therapeutic effect ,20. ThisTD/EF = fraction of particles with an aerodynamic diameter < 5 µm of the total dose/emitted fraction) regardless of the formulation tested and actuation conditions used. To determine the influence of capsule movement on powder ejection and the number of particles that can potentially reach the lungs, devices with axial capsule vibration ), capsule rotation ) = RS01 equivalent device), and oscillating capsule movement (Presspart prototype DPI = PP-DPI) were compared  is an interactive blend of micronized albuterol sulfate and alpha lactose monohydrate, a spray-dried rifampicin formulation was also tested ..0.5 L/miSEM pictures in Testing the albuterol sulfate formulation with the different devices at 50 L/min resulted in a high powder deposition in the preseparator in every case a. While For the binary amoxicillin blend, the lowest powder retention in the capsule combined with the lowest powder deposition in the IP was observed after actuation with the Presspart prototype DPI, regardless of the flow rate applied . CompariEF/TD .0.5 L/min and a flow rate of 45 L/min at a pressure drop of 4 kPa, the new capsule-based DPI can be classified as a device with medium–high intrinsic resistance to airflow. Based on the studies described in the introduction and the published results for this class of DPIs, the novel device should deliver the loaded powder uniformly to the patient regardless of the inhalation conditions, which could lead to higher treatment adherence in vivo [Classifying the novel device as a DPI with low or high intrinsic resistance to airflow and determining the pressure drop as a function of flow rate is an important parameter for verifying test conditions for conducting future studies. Due to the designed geometry resulting in intrinsic resistance of 0.044 kPa in vivo . With re in vivo ,22. HoweFor both binary mixtures, oscillatory capsule motion was shown to result in a higher emitted fraction than axial capsule vibration  or capsule rotation , regardless of the flow rate applied. It can be concluded that for well-flowing binary blend formulations, the powder can be easily shaken out ,6. In thDespite the higher powder retention of the rifampicin formulation in the capsule, better deagglomeration of the emitted powder was observed with the Presspart prototype DPI than with the other two DPIs. The capsule movement described above, in combination with the device design, geometry, and airflow within the DPI, promotes powder deagglomeration so that the weak powder output from the capsule can be compensated.TD, the Presspart prototype DPI achieved the highest FPF regardless of the formulation tested, and the flow rate applied, highlighting the functionality of the deagglomeration behavior for binary blends and spray-dried particle formulations. Since there were differences in powder deposition in the capsule–preseparator compartments, this resulted in differences in FPFEF when comparing the two flow rates for a device and the respective formulation. While whether the powder remains in the capsule or DPI after inhalation is not important for therapeutic success, FPFTD appears to be better suited to assess the independence of powder deagglomeration from inhalation conditions, as it describes the amount of powder of the loaded dose that can be adequately deagglomerated by the device and delivered to the lung in vitro, regardless of the inhalation conditions applied. Comparing the FPFTD obtained for a formulation with the DPIs, it is noticeable that the two DPIs with high intrinsic resistance to the airflow achieved identical values at both flow rates, indicating that powder deagglomeration was not affected by the airflow.Comparing the FPFFrom the device side, it can be summarized that the deagglomeration mechanism of the new DPI appears to offer an advantage over the known devices tested here in terms of powder deagglomeration, regardless of whether the formulation is a binary mixture or a particle engineered formulation. Although a high FPF was always achieved under the tested conditions, further development of the oscillating capsule movement mechanism could lead to increased powder ejection, especially for spray-dried particles.From this study, firstly, it can be concluded that progress is being made in the development of DPIs with respect to the aerosolization behavior of various formulations not designed for the specific device and that, for this reason, the DPI could be marketed in the future regardless of the formulation. The development of DPIs with device geometry that provides high intrinsic resistance to airflow could lead to powder aerosolization that is much more independent of inhalation conditions than is currently the case. Second, the development of binary mixtures consisting of spray-dried active ingredients and carrier particles could be an interesting approach for future formulations, as the advantages of both formulation techniques can be combined to increase the number of particles that have suitable aerodynamic particle properties to reach the lungs.


Natural language processing has been established as an important tool when using unstructured text data; however, most studies in the medical field have been limited to a retrospective analysis of text entered manually by humans. Little research has focused on applying natural language processing to the conversion of raw voice data generated in the clinical field into text using speech-to-text algorithms.In this study, we investigated the promptness and reliability of a real-time medical record input assistance system with voice artificial intelligence (RMIS-AI) and compared it to the manual method for triage tasks in the emergency department.From June 4, 2021, to September 12, 2021, RMIS-AI, using a machine learning engine trained with 1717 triage cases over 6 months, was prospectively applied in clinical practice in a triage unit. We analyzed a total of 1063 triage tasks performed by 19 triage nurses who agreed to participate. The primary outcome was the time for participants to perform the triage task.P<.001). Most variables required for entry in the triage note showed a higher record completion rate by the manual method, but in the recording of additional chief concerns and past medical history, RMIS-AI showed a higher record completion rate than the manual method. Categorical variables entered by RMIS-AI showed less accuracy compared with continuous variables, such as vital signs.The median time for participants to perform the triage task was 204  seconds by RMIS-AI and 231  seconds using manual method; this difference was statistically significant (RMIS-AI improves the promptness in performing triage tasks as compared to using the manual input method. However, to make it a reliable alternative to the conventional method, technical supplementation and additional research should be pursued. An essential role of a hospital emergency department (ED) is to prioritize treatment for patients according to urgency and symptom severity . This roBecause the results derived through the triage system must be immediately recorded and shared with the medical staff in charge of the next process, a prompt triage system is crucial for an efficient ED. In addition, the results from the triage system are reported to have a significant influence on clinical outcomes -10. TherRecent advances in machine learning and natural language processing (NLP) are a prominent development in health informatics and are relevant in emergency medicine . AlthougThis study was conducted in accordance with the revised Declaration of Helsinki and was reviewed and approved by the Institutional Review Board of Severance Hospital, South Korea .We performed a prospective interventional study. This study was conducted at a Level 1 ED at a tertiary hospital located in northwestern Seoul , where 90,000 patients visit annually. The hospital’s ED is responsible for receiving patients who cannot be stabilized in this catchment area. Participants were recruited through an official announcement period from November 1, 2020, to the end of January 2021. Among the nurses performing triage work in the hospital’s ED, 19 nurses who listened to the contents and process of the study voluntarily agreed to participate in the study. They had more than three years of ED work experience. Exclusion criteria included candidates who (1) withdrew their intention to participate, or (2) had physical symptoms that made it difficult for them to wear a voice recognition microphone. Informed consent was obtained from all participants before enrollment.Because conversations in the triage unit contained a large amount of information and noise, a device that can select and record these conversations was needed. A machine learning framework created by Selvas AI Inc  was used in this study. The voice recognition solution provided by Selvas analyzes sound information and converts it into text, commands, and various forms of information. The application of continuous word recognition engines, which recognize unstructured speech, has expanded to different fields; for example, a speech recognition engine in this study has been exclusively developed for the medical field. In our ED, the triage nurses are supposed to record the results of performing a task in a triage note. This triage note consists of the following items: chief concern, past medical history, the presence of allergic diseases, vital signs such as systolic and diastolic blood pressure, heart rate, respiratory rate, body temperature, and oxygen saturation. To train the engine, triage nurses who agreed to participate in this study performed the clinical practice wearing Bluetooth microphones . Voice recording files that passed through the engine were immediately converted into textual data, without prior editing, and stored as log records. Subsequently, the engine repeatedly trained the NLP to fill the items constituting the triage note using the transcribed textual data. The Bluetooth microphone was selected as a component of a noise-resistant system in accordance with the ED environment where various noises exist, and a mobile recording system was built to ensure its mobility. The Bluetooth microphones, voice recognition software, and systems using computers connected to them were installed in the triage unit, and voice data were recorded during the data collection period. For 6 months, 1717 triage cases were collected, and the machine learning engine was trained to recognize the sound using these voice data, convert it into textual data, and perform the subsequent NLP. Consistent with the current triage note format, the system was trained to classify the chief concern of each patient into 1 of 52 categories, and the past medical history was processed into 13 categories through NLP. In the triage note used in this ED, up to 3 chief concerns and the medical history can be entered. The presence of allergic diseases was configured to be treated as a binary input, and variables representing vital signs were treated as continuous variables.For accurate voice interval detection in a noisy environment, the end-point detection module was optimized in the machine learning engine. By distinguishing various nonstationary noises through continuous adaptive learning for noise coming through the Bluetooth microphones, a deep neural network end-point detection module was developed with high accuracy in detecting energy-based voice sections of the existing method. The voice interval detection module optimized for the voice environment input to the Bluetooth microphone was advanced, and sound using the collected and processed purified voice database and converted textural data was applied for language model learning.From June 4, 2021, to September 12, 2021, a real-time medical record input assistance system with voice artificial intelligence (RMIS-AI) built using a trained engine was prospectively applied to the clinical practice in the triage unit where the patients meet the medical staff for the first time. RMIS-AI is a tool that assists in recording triage notes through voices. In other words, it secures the mobility of a triage nurse by replacing the record input means with voice instead of the desktop computer keyboard. RMIS-AI was implemented on a cloud-based network separate from the hospital electronic medical record (EMR) system. During the study period, participants wearing Bluetooth microphones recorded triage data in the EMR by asking detailed questions to each patient and checked vital signs. Simultaneously, they also recorded the data through RMIS-AI in the same format using their voice. Because the participants used a closed-loop communication method that reconfirmed the meaning of the patient’s words and uttered them, the information obtained from the patients could be delivered by the participant’s voice rather than the patient’s voice. The input process of charting through RMIS-AI was blind to the nurses, and they monitored the EMR input process as usual when performing the triage task. The contents and time of the triage log finally created in both ways were stored in the hospital EMR log and cloud storage, respectively . The datThe primary outcome was the time for participants to perform the triage task. It was defined as the time from the patient’s arrival at the triage unit to the completion of the triage note. We measured these times using data stored in the hospital EMR for manual input and cloud storage for RMIS-AI. The secondary outcome metrics were the record completion rate and the accuracy of RMIS-AI compared to manual input by EMR.P<.05, statistical power=0.95). Therefore, the required sample size was calculated to be 952 cases by G-power 3.1.9.7, requiring a total of 1057 triage cases considering a 10% dropout rate. In this paper, categorical variables are presented as counts and percentage. Continuous data are presented as mean or median and SD or interquartile range. The Mann-Whitney U test was used to identify the differences of primary outcome between the 2 groups. Differences in record completion rates between the 2 methods were compared using the McNemar test. The result was considered statistically significant at P<.05. The intraclass correlation coefficient (ICC) using the 2-way mixed effects model, absolute measurement, and single measurement were used to evaluate the interrater reliability of continuous data between the 2 groups [The sample size was calculated from the mean time taken by performing the triage task in a conventional method for 100 cases before the intervention was started. We considered that the RMIS-AI producing a mean difference of 20 seconds with standard deviation difference of 2 seconds would be considered clinically significant  triage tasks were performed by the participants. After 146 cases were excluded by the criteria shown in P<.001), as shown in the box plot in The median time for participants to perform the triage task was 204  seconds with RMIS-AI and 231  seconds using manual input by EMR. The difference between the 2 methods was statistically significant (The record completion rates of both methods for all triage cases are shown in The accuracy of reproducing records by RMIS-AI for all variables is summarized in Previous study results have proven that prolonged waiting times and crowding are factors that reduce patient satisfaction and impair safety in the ED -25. LongThe record completion rates of RMIS-AI were inferior to the manual input by EMR in our study, especially in the input of allergy history or past medical history. In the case of categorical variables, such as allergy history or past medical history, NLP is more difficult than in the case of continuous variables, such as systolic blood pressure and pulse rate, because it is expressed in a wide variety of phrases rather than simple utterances. Korean is an agglutinative language and one of the morphologically rich and typologically diverse languages. Auxiliary, adverbial case markers, word spacing inconsistency, and the variety of expressions of predicates with the same meaning make NLP using Korean difficult. . The engIn our study, it is assumed that the difference between the variables with and without relatively favorable accuracy is due to the complexity of NLP. NLP is still being developed as an artificial intelligence field, and because there is no standardized format, its performance is different depending on the type and amount of training data as well as the deep learning method applied ,32. For The reliability of pulse rate was lower than that of other vital sign values because there was a time difference between the input through RMIS-AI and the manual input because triage nurses record the pulse rate by watching the monitoring being measured as a continuous waveform. This result can be explained by the Bland-Altman plot, where the error range in the input value is narrow. In addition, the low ICC value of the respiratory rate was due to the less amount of data and low variability.NLP is a tool that can structure unstructured textual data and enable the use of unstructured voice data that historically have not been used in the medical field. Previous studies have reported that the predictive performance of clinical outcomes is improved when unstructured textual data are used for machine learning in the medical field ,15,16,33. Third, the reduction in time taken to perform the triage task does not guarantee improvements in patient outcomes. Therefore, the relationship between the use of RMIS-AI and improvement in clinical outcomes on patients in the ED should be investigated. Finally, the study was performed at an ED in a single tertiary hospital; thus, there is a limit to generalizing the research results.This study has several limitations. Although our study was conducted in a prospective design, a study using a randomized controlled design is needed to obtain definitive evidence that the RMIS-AI can replace the conventional method. Second, the completeness and accuracy of the triage note by the current RMIS-AI are insufficient to safely replace the conventional manual input method. If NLP for the recording of triage note recording is learned using additional training material, it can be improvedIn this study, we confirmed that the promptness in performing triage tasks improved using RMIS-AI developed with STT and NLP technology compared with the manual input method, but technical supplementation was required to deal with the current level of inferiority in sensitivity and accuracy. If similar studies are conducted to confirm the potential of such technologies in clinical practice, artificial intelligence could evolve as a supportive tool to improve patient experience.