However, clinically processed 2D ovarian-tumor ultrasound images contain many artificially recognized symbols, such as fingers, crosses, dashed lines, and letters which assist artificial intelligence (AI) in image recognition. These symbols are widely distributed within the lesion’s boundary, which can also affect the useful feature-extraction-utilizing networks and thus decrease the accuracy of lesion classification and segmentation. Image inpainting techniques are used for noise and object elimination from images. To solve this problem, we observed the MMOTU dataset and built a 2D ovarian-tumor ultrasound image inpainting dataset by finely annotating the various symbols in the images. A novel framework called mask-guided generative adversarial network (MGGAN) is presented in this paper for 2D ovarian-tumor ultrasound images to remove various symbols from the images. The MGGAN performs to a high standard in corrupted regions by using an attention mechanism in the generator to pay more attention to valid information and ignore symbol information, making lesion boundaries more realistic. Moreover, fast Fourier convolutions (FFCs) and residual networks are used to increase the global field of perception; thus, our model can be applied to high-resolution ultrasound images. The greatest benefit of this algorithm is that it achieves pixel-level inpainting of distorted regions without clean images. Compared with other models, our model achieveed better results with only one stage in terms of objective and subjective evaluations. Our model obtained the best results for 256 × 256 and 512 × 512 resolutions. At a resolution of 256 × 256, our model achieved  Medical ultrasonography has turned out to be the preferred imaging technique for many illnesses due to the fact of its simplicity, speed, and safety ,2,3,4,5.Nevertheless, hidden but crucial trouble has been recognized in practice: most 2D ovarian-tumor ultrasound images incorporate extra symbols. Actually, in clinical operations where ovarian ultrasound images are acquired, the physician will mark the location, size, and border of the tumor in the ovarian ultrasound image, and observe where the lesion is positioned (left or right ovary). Due to equipment factors and the clinical practice environments, the artificially marked component of these aids to image recognition  cannot be separated from the original image. This phenomenon is also widespread in different medical fields ,16,17,18We observe that these symbols are centered in ovarian tumor lesions, which negatively affects the training of the model to a positive extent, causing the network to focus more on the symbols in the lesions, which in turn reduces the recognition accuracy of ovarian tumors in the clean images and the segmentation accuracy of the lesions. The different types of images in this paper are shown in Currently, image inpainting in medical images is in the process of booming and has a lot of potential for development. Existing methods are primarily divided into traditional methods and deep learning-based methods. Traditional methods make use of patch-based or diffusion-based methods, the core of which is to use the redundancy of the image itself to fill in the missing areas with low-level texture features of the image. The following four methods are historically used for inpainting: interpolation , non-locDeep-learning-based methods use convolutional neural networks to extract and learn high-level semantic features in the image to guide the model to fill the missing parts. Inspired by EdgeConnect , Wang etIn addition, various attention mechanisms have been proposed and are broadly used in image processing. These attention mechanisms have been steadily utilized in the image inpainting task. Zeng et al.  expandedDue to the above issues, in this paper, a one-stage generation model based on GANs is proposed, which swaps the regular convolution with fast Fourier convolutions to enhance the image-wide acceptance field of the model and includes a channel attention mechanism to minimize the model’s focus on symbols to fill the holes using effective features. To the best of our knowledge, we are the first to accomplish image inpainting on 2D ovarian-tumor ultrasound images with large and irregular masks, and our approach achieves more convincing results than others.We refined 1469 2D ovarian-tumor ultrasound images for irregular symbols and obtained binary masks to establish a 2D ovarian-tumor ultrasound image inpainting dataset.We introduced fast Fourier convolution to enhance the model’s global perceptual field and a channel attention mechanism to enhance the model’s attention to significant features, and the model uses global features and significant channel features to fill the holes.Our model achieved better results both subjectively and objectively compared to existing models while for the first time performing image inpainting without clean images.We use the restoration images for segmentation training, which significantly enhances the accuracy of the classification and segmentation of clean images.Our contributions are as follows:The rest of the paper is organized as follows: In recent years, research about ovarian tumors has increased, and researchers have combined ovarian tumor sonograms with deep learning for ovarian tumor classification and lesion segmentation ,31,32,33With annotation, a corresponding mask for each image is generated, which masks the various symbols in the image. In this study, we used a complete, 2D ovarian-tumor ultrasound dataset with 1469 images that we produced, of which 1200 images were used for training and 269 images were used for testing. Arbitrarily shaped masks were used during training and testing. To make certain the equity of the experiments, we generated unique irregular masks for the images used for testing. The inputs in our experiments had two specifications: one specification was We propose an image inpainting model based on fast Fourier convolutions (FFCs) with a channel attention mechanism. Transforming the input tensor to the frequency domain using the real fast Fourier transform: Concatenating the real and imaginary parts in the frequency domain: Obtaining convolution results in the frequency domain through the ReLU layer, BatchNorm layer, and 1 × 1 convolution layer: Separating the result of frequency domain convolution into real and imaginary parts: Recovering its spatial structure using Fourier inverse transform: Regular convolution is mostly used in deep learning models; however, it cannot capture the global features. Fast Fourier convolutions  can be aAs shown in The approach of mask generation during training has been extensively mentioned in previous research, and it is crucial for the inpainting effect of the model. In early studies, the generated masks are rectangular in shape , centerem is the final mask.There are many symbols in the image that obscure the clean image. If these areas are repaired, the results cannot be evaluated realistically due to the fact there is no clean image. We need to guide the network to learn to use features of the non-symbolic regions to fill holes. In our task, we propose a new mask generation strategy by generating random irregular masks at random locations outside the symbolic regions in the image. The generation formula for the masks is as follows: The loss function in the generation task is essential for the training of the model, and it calculates the distinction between the ground truth and the inpainting image as the loss value. The loss values are back-propagated, and the model is penalized to update the parameters of each layer. In the end, the loss value is reduced, and the result is closer to the ground truth.m denotes the corresponding mask, for which one denotes the missing pixels and zero denotes the existing pixels. The symbol ⊙ denotes the multiplication of the matrix. G denotes the generator, Several extraordinary loss functions were used in our task. In our model, the input uses the corrupted image Equation .(2)IinpThe perceptual loss  is derivual loss  using a D is the adversarial discriminator.To make the generated inline images more realistic and natural in detail, we additionally use adversarial loss. The adversarial loss function p represents the pixel at the same location in both images.In our total loss, we also use the rk LDisc . The forEquation . The L1 Our total losses are calculated as follows:ollowing ,39,42, wWe used the evaluation metrics of y (SSIM) , Frechetre (FID) , and lea (LPIPS)  to measuA, B, The SSIM is calculated between two windows of size H × W. The value of SSIM is between The Frechet inception distance score (FID) is a metric to calculate the distance between the real image and the generated image feature vectors. It uses the 2048-dimensional vector of Inception Net-V3 before full concatenation as the feature of the image to evaluate the similarity of the two sets of images. The value of FID is greater than or equal to zero. A lower score means that the two sets of images are more similar, and the FID score in the best case is 0.0, which means that the two sets of images are identical. The FID calculation formula is described as follows:l is the current computed layer; L layers and unit-normalized in the channel dimension. The vector LPIPS is used to measure the difference between two images in terms of deep-level features, and LPIPS is more consistent with human perception than traditional methods such as k is the number of categories, MIoU is a widely used standard metric in semantic segmentation, which calculates the mean of the ratio of intersection and merges sets of all categories. The value is between zero and one. Closer to one means better the segmentation, and closer to zero is the opposite. Its calculation formula is defined as follows:FFCsFast Fourier convolutions have a larger and more effective field of repetition, which can effectively enhance the field of repetition of our model and improve its capability. We performed quantitative experiments on fast Fourier convolution, dilated convolution, and regular convolution. The convolution kernel size was set to 3 × 3, and the expansion rate of the dilated convolution was set to 3. Mask generationThe types, sizes, and positions of the mask during training impact the generative and generalization capabilities of the model. In our task, we focused on exploring the effect of mask generation location on the model. Regular, irregularly shaped masks will overlap with a variety of symbols in the image, and this part of the region was devoid of realistic background for a realistic inpainting quality assessment. Additionally, we avoided network learning to use the features of these symbols. We compare our mask generation approach with the conventional method, and Attention mechanismFor the network to attenuate the focus on symbolic features in the image and enhance the focus on other features in the real background, we introduced the SE layer. By introducing the channel attention mechanism, our model pays more attention to the features of non-symbolic regions rather than the features of symbolic regions and chooses useful features. By this method, the restored image is more similar to the original image in terms of content and no yellow pixels show up in the restoration region. To verify that our approaches do reduce the capabilities of the model, we designed ablation experiments for the baseline model. The dataset used for the experiments was our inpainting dataset. We used solely FFCs as the baseline in this experiment.As we noted in the introduction, our aim of inpainting of 2D ovarian-tumor ultrasound images is to enhance the accuracy of currently popular segmentation models such as Unet and PSPnet for the segmentation of ovarian lesions.the Unet  model ane PSPnet  model inIn this paper, we proposed a 2D ovarian-tumor ultrasound image inpainting dataset to investigate the effect of prevalent symbols in images on ovarian-lesion segmentation. Based on this image inpainting dataset, we proposed a 2D ovarian-tumor ultrasound image inpainting model based on fast Fourier convolution and a channel attention mechanism. Labeled images are used as a priori information to guide the model to focus on features in the non-symbolic regions of the images, and fast Fourier convolution is used to extend the receptive field of the model to make the texture and structure of the inpainting images more realistic and the boundaries smoother. Our model outperformed existing methods in both qualitative and quantitative comparisons. It received the highest scores in all three metrics, LPIPS, FID, and SSIM, which proves the effectiveness of our model. We used the inpainting images for training and validation with Unet and PSPnet models, which appreciably enhanced the accuracy of lesion segmentation in clean images. This additionally demonstrates the great significance of our study for computer-aided diagnosis of ovarian tumors.Our study in this paper did not currently use ground truth of lesion segmentation in the dataset, which may further improve the similarity of lesion boundaries in inpainted images. In future work, we will do further exploration on how to apply the edge information of the lesion to the model to make the boundaries more similar to those in the original image and extend our model to other types of medical images—CT, MRI, etc.


Auditory feedback perturbation studies have indicated a link between feedback and feedforward mechanisms in speech production when participants compensate for applied shifts. In spectral perturbation studies, speakers with a higher perceptual auditory acuity typically compensate more than individuals with lower acuity. However, the reaction to feedback perturbation is unlikely to be merely a matter of perceptual acuity but also affected by the prediction and production of precise motor action. This interplay between prediction, perception, and motor execution seems to be crucial for the timing of speech and non-speech motor actions. In this study, to examine the relationship between the responses to temporally perturbed auditory feedback and rhythmic abilities, we tested 45 adult speakers on the one hand with a temporal auditory feedback perturbation paradigm, and on the other hand with rhythm perception and production tasks. The perturbation tasks temporally stretched and compressed segments (onset + vowel or vowel + coda) in fluent speech in real-time. This technique sheds light on the temporal representation and the production flexibility of timing mechanisms in fluent speech with respect to the structure of the syllable. The perception tasks contained staircase paradigms capturing duration discrimination abilities and beat-alignment judgments. The rhythm production tasks consisted of finger tapping tasks taken from the BAASTA tapping battery and additional speech tapping tasks. We found that both auditory acuity and motor stability in finger tapping affected responses to temporal auditory feedback perturbation. In general, speakers with higher auditory acuity and higher motor variability compensated more. However, we observed a different weighting of auditory acuity and motor stability dependent on the prosodic structure of the perturbed sequence and the nature of the response as purely online or adaptive. These findings shed light on the interplay of phonological structure with feedback and feedforward integration for timing mechanisms in speech. Compensation was classified as purely an online response when adjustments occurred ∼120–200 ms after perturbation onset in the ongoing production process  have been extensively studied, a few recent studies have started to investigate the role of auditory feedback for speech timing  or can be rather unspecific responses to a disturbance in the feedback that demands attention and time to process. Reactive feedback responses seem similar to responses elicited by generally delayed auditory feedback, where speakers were found to slow down their speech rate or prolong speech elements in response  while the sensory experience of the produced outcome, such as auditory or somatosensory feedback, serves to monitor and evaluate the process constantly (feedback system). The interaction of feedback and feedforward systems in speech production has been of significant interest in speech research and has mainly been probed with real-time auditory feedback perturbations. In auditory feedback perturbation paradigms, speakers hear their voice over headphones while one or more parameters in the acoustic signal are altered in  real-time. In response, speakers were found to counteract the applied feedback shift  in feedforward control. These speech targets are established via auditory and somatosensory feedback in speech acquisition . TherebyWhile individual abilities in feedback control have been considered to be crucial influencing factors in building and controlling speech targets, much less attention has been given to the thought that also feedforward mechanisms, more precisely motor execution abilities, are governed by limits of individual abilities. The study by A link between non-verbal sensorimotor timing abilities and speech production was found when testing finger tapping performance in non-impaired speakers and speakers with speech timing disorders. For example, Altogether, these studies point toward a strong link between motor behavior in speech and non-speech actions. This link is noteworthy in particular when investigating the role of feedforward stability for timing mechanisms in fluent speech. Indeed, it can be hypothesized that temporal stability in non-speech motor behavior is connected to temporal stability in speech motor control. Thereby, it has to be taken into consideration that, domain-independently, different motor timing tasks might require different underlying neural mechanisms. Neuroscientific research outlined different such mechanisms dependent on the demand of the timing task. The previous section outlined how perceptual abilities and general motor behavior connect to speech production. Further, the introduced timing mechanisms contribute to the complexity that is assumed to underlie the planning and control of speech timing.The main goal of the following study is to shed light on the contribution of auditory feedback and motor timing abilities to speech production. Therefore, we examine the contribution of general internal timing stability as predictor for temporal speech feedforward stability, and the importance of feedback and feedforward mechanisms for executing and planning the temporal structure of fluent speech. To follow this aim, the present study assessed individual capacities in paced and unpaced finger tapping tasks and beat-based and duration-based perception tasks, and set them in relation to behavior during temporal auditory feedback perturbation from the data collected in our previous study . In doinstructural motor stability on timing behavior to extend to individual abilities in motor stability, which may also shape timing mechanisms in speech.First, concerning the contribution of perception and motor execution, we expect speakers with better perceptual abilities (auditory acuity) to compensate more for temporal auditory feedback perturbations as found analogously for spectral properties of speech. This hypothesis is based on the idea that the better an auditory mismatch is perceived, the more (precisely) speakers can counteract it. Moreover, we expect speakers with a worse performance in motor execution in finger tapping tasks (speakers with a higher motor variability) to compensate more. This hypothesis ties up with the findings of Second, regarding the nature of responses to the auditory feedback perturbation as an online response or adaptive, we expect to find perceptual acuity equally relevant for both online reactions and adaption, since both types of reaction require the ability to perceive the auditory mismatch and identify the direction for a compensatory response in the first place. General motor stability, on the other hand, should be a greater predictor for adaptation, since a less stable feedforward system should provide a greater tolerance toward updating the less stable representations. This hypothesis is tied to expectations about the relevance of auditory feedback and motor stability for different parts of the syllable, since the coda showed adaptation while the onset did not .Forty-five native speakers of German performed three testing blocks (described further below) in one testing session of approximately 2.5 h. Participants were between 19 and 30 years of age  and received financial compensation for their participation. Musicality was assessed with a questionnaire. Thirty-two participants stated they have received musical education on various instruments. Five of them reached a semi-professional level, indicating that they could earn money with music. Musicality was not a main focus of interest in the current study. However, additional analyses about effects of musicality on the response data can be found in the The following sections outline the three testing blocks Perturbation, Tapping, and Perception.The perturbation response data was taken from the perturbation experiment reported in Onset condition) or vowel + coda (Coda condition) of the first syllable in a three-syllabic word was temporally altered . Thereby, the first segment per condition was stretched in real-time  and the following segment was compressed  leading to an on-time signal after completion of both shifting directions. While this alteration results in perturbation being completely contained within a single syllable, it should be noted that the second segment of the perturbed part starts delayed by the amount of stretching of the first segment (plus the systems delay of approximately 25 ms that is needed for online manipulation).The temporal auditory feedback experiment tested the sensitivity to temporal perturbations with a special interest in position within the syllable. In two experiment conditions, the temporal structure of either onset + vowel (better) allowed for an online status tracking of the signal by the Audapter software to trigger the intended part within the target word  was determined manually by the experimenter , and the classification of the response as either online control (when productions revert to the Baseline immediately in the After-effect phase) or adaptive . For the purposes of the current study, the mean productions in the Hold phase as compared to the Baseline per participant are examined as an indicator for the strength of reaction during maximum perturbation. This measure will then be set in relation to measures from the tapping and the perception blocks. The analysis of the After-effect phase to classify responses as an online reaction or adaptive response was performed previously in For analyses, durations of the segments of interests were segmented manually in Praat. Production differences in word-normalized durations in the Hold phase (with maximum perturbation) relative to the Baseline for each segment of interest (CC /pf/ and V /a/) in each perturbation condition (Onset and Coda condition) were examined. Accordingly, four compensation measures are considered in the following calculations: Compensation to the onset segment in the Onset condition (Onset CC), compensation to the vowel in the Onset condition (Onset V), compensation to the vowel in the Coda condition (Coda V), and compensation to the coda segment in the Coda condition (Coda CC). These measures are the same as reported in The perturbation data per participant and trial were scanned for correct triggering of the perturbation section at the intended part within the utterance. Since the online status tracking is based on previously determined intensity and duration thresholds, in some cases due to inter- and intra-speaker variability in speaking rate and style, the perturbation did not cover the intended speech sequence. Single trials were removed from calculations when perturbation did not cover the onset + vowel or vowel + coda section as intended. Participants who had less than 16 trials in the hold phase with accurate location of the perturbation were removed from further calculations. Linear regressions confirmed no effect of number of trials on compensation magnitude after removing the failure trials. Based on this exclusion criterion, 34 participants remained in the Onset condition and 33 participants in the Coda condition , revealing that speakers who compensated more for the vowel perturbation in the Coda condition (by shortening it in production) also compensated somewhat more for the Coda CC segment (by lengthening it in production). Regarding our hypothesis, we assume that for these segments, which both showed adaptive behavior, a certain level of motor malleability is given in speakers that compensate and adapt more. However, since both segments appeared within the same word/trial, the magnitude of perturbation might have contributed to equally strong within-participant responses. A relationship was also found between Onset V and Coda CC responses, which both were second segments in the perturbation section, hence compressed in the auditory feedback but lengthened in production . Further, both segments were displaced in time, due to the stretch of the first segment and assumable subject to online control effects triggered by the stretch of the previous segment. The relationship supports the hypothesis that there might be an individual auditory sensitivity in speakers to react to effects of delayed/shifted auditory feedback in the online control. Both relations reinforce the aim of the current study to find individual motor or auditory abilities that enhance or decrease articulatory timing malleability in the face of an auditory perturbation, and more generally in the speech production process.Before turning to the motor and perceptual tests that will be related to the perturbation response patterns, we introduce here some brief analysis not included in the original Battery for the Assessment of Auditory Sensorimotor and Timing Abilities  to a computer controlled by MAX-MSP software (version 6.0). Loudspeakers delivered sound stimuli in free field at a fixed volume which was kept constant over participants. The experimenter instructed the participant to tap with their writing hand’s index finger on the tapping pad. Practice trials preceded each of the following tasks, which could optionally be skipped when the following task was very similar to the preceding one. Tasks 1, 2, and 3 are adopted from the  BAASTA, . Tasks 4motor variability.All tapping data were pre-processed following the procedures as reported in The third block tested for individual perceptual abilities. Five adaptive staircase tasks assessed individual auditory acuity performances for temporal properties of various stimulus types. The listener was seated in front of a computer and provided with headphones. Volume was set to a comfortable level as tested and determined by the experimenter and was not changed between listeners unless requested. After the experimenter started the procedure in MATLAB, listeners performed the tasks by entering their responses directly into the testing computer. The first three staircase tasks captured duration discrimination abilities (hence duration-based timing mechanisms) in a 2-interval 2-alternative forced-choice paradigm. These tasks required judgments about the two perceived stimuli as identical or different. Task 1 required judgments about pure tone durations. Tasks two and three comprised monosyllabic words with temporal manipulations analogous to the auditory feedback perturbation paradigm described in section “Temporal auditory feedback perturbation.” In the onset perception task, the onset of a word was stretched and the vowel compressed. In the coda perception task, the vowel was stretched and the coda compressed (see psola (see task-specific description below). The presented stimuli were randomized, whereby the original stimulus was either in the first or in the second position. In the two beat-alignment tasks (4 and 5), one endpoint of the continuum was a stimulus with perfect beat-alignment, and the other endpoint a stimulus with the maximally shifted beat. In these tasks, always one stimulus from the continuum was presented while the degree of shift in alignment varied along the continuum. The difference between the two presented stimuli in Tasks 1, 2, and 3, or the degree of metronome shift in Tasks 4 and 5 is referred to as delta (in ms). In each task, the delta could be varied in increments of 1 ms. The maximum/start delta defines the largest difference between the stimuli or between the metronome and the stimulus. Estimations of the lowest correctly identified delta  will serve to measure each participant’s individual auditory acuity.For all five tasks, continua of stimuli between two endpoints were generated, whereby one endpoint consisted of the original stimulus and the second endpoint was a manipulated version. The manipulations target durations exclusively. For the three duration discrimination tasks (1–3), stimuli were presented in pairs  in which one stimulus was always the original stimulus and the other stimulus varied in degree of manipulation between the two endpoints. Manipulations of segment durations were performed in Praat using catch-trials). Listeners who did not identify more than 50% of the presented catch-trials correctly or did not reach a score below 70% of the start delta of a test were suspected of answering by chance or classified as incapable of performing the task. However, none of the participants had to be excluded from further calculations based on these criteria. To measure individual performance, for every participant and task, an individual auditory acuity score  was assessed by calculating a mean over the delta of the most stable consecutive six reversal points in each task , visualized in All five staircase tasks had fixed but adaptive step intervals. In other terms, the next presented stimulus pair was triggered by the listener’s response. The tasks always required two correct difference detections in a row to the same stimuli pair to mark a successful identification, but only one incorrect response to mark a false identification (2 down/1 up protocol). The two stimuli in both presented trials appeared in random order. Following a successful mismatch identification, the current delta was multiplied by 0.5; with every not detected mismatch, the delta was multiplied by 1.5 see . WheneveThe data provided by the Tapping and Perception blocks capture many motor and perceptual abilities for rhythmic speech and non-speech tasks. The following sections aim at summarizing the data, since the single tasks per block are expected to be highly collinear. In perception, the expected underlying mechanisms of the tasks as event-based or duration-based timing provide motivation to group tasks together based on timing mechanisms. However, it is also possible that dividing the tasks into speech and non-speech tasks reduces collinearity to a greater extent and explains the largest variance in the dataset. Further, in tapping, unpaced tapping can be accomplished successfully by engaging either beat-based mechanisms  or duration-based mechanisms  and further expected the PCA to give more insight into the nuances of the individual tasks.The perception and tapping data were submitted to principal component analysis (PCA) using R’s  package . The PCAAs PCAs cannot deal with incomplete data, participants with no data in more than one out of the nine tapping tasks (including unpaced tapping) were not submitted to the PCA. Based on this criterion, five of 45 participants were removed before submitting the data to the PCA. For those participants who had missing data in one task, the missing value was filled with the k-nearest-neighbor imputation method .PC1 Auditory Acuity. The music BAT perception task correlates negatively with PC2, as does the speech BAT task, although less intensely. The coda discrimination task correlates positively with PC2, and so do the onset and duration discrimination tasks, but less intensely. This clustering indicates that PC2 distinguishes beat-alignment judgments (event-based timing mechanisms) in speech and music from duration discrimination (duration-based timing mechanisms) and will further be named PC2 BAT Perception. Hereby, higher (positive) PC-scores are associated with better beat-alignment perception  but worse duration discrimination . In view of this interpretation of PC2 BAT Perception, we expect this measure to be more closely connected to compensation in onsets, since we previously assumed onsets to rely more on event-based timing mechanisms. Duration-based perception tasks might be more closely coupled with compensation in the coda, since coda timing requires more likely absolute estimations of the time-lag from the onsets. Concerning the vowels, we have no precise hypothesis, since in the transition from onset to the vowel the p-center might play a role. The end of the vowel, on the other hand, might be more likely estimated with an absolute (duration-based) timing mechanism. The perturbation data and predictors provided by the Tapping and Perception blocks are summarized in For the Perception PCs, all of the tasks correlate negatively with PC1, with a higher PC score indicating a better perception (meaning a lower auditory acuity threshold). Therefore, PC1 reflects general auditory acuity and will further be referred to as From the complete set of 45 participants in the beginning, the predominant basis for participant exclusion came from the perturbation data. Thirty-four participants remained in the Onset condition and 33 in the Coda condition after scanning the data for correct triggering of the perturbation . The full set of participants was submitted to the PCAs for Tapping and Perception to get more reliable scores based on a larger dataset. Excluded from the full set were participants who had missing values in more than one of the tasks , while data was imputed in the Tapping block for missing data in maximally one task per participant (applying to two participants). Data was also imputed for the Unpaced Tapping task (for two participants).After calculating the PCAs, outliers of the generated principal components and the unpaced tapping task  were removed to reduce noise in the data. The same outlier treatment was applied to the perturbation response data.The data were scanned for missing values based on outlier exclusion in the four Tapping/Perception measures . No participant had more than one missing value in the data. The single missing values were replaced with knn-imputation as performed on the raw data prior to the PCA . Since the perturbation response data serve as the dependent variable in this study, none of the missing values  were imputed for the perturbation measures. The data was then divided into four datasets, each comprising one perturbation measure as the dependent variable  and the four Tapping/Perception measures as predictors . After interpreting the PCs in section “Interpreting PC scores,” we further assumed the PC2 BAT Perception to be more related to Onset CC compensation, since syllable onset timing in speech production has been suggested to rely on event-based timing mechanisms. However, since we did not find a significant effect of compensation for the Onset CC in the first place, this effect might not show in the analyses. We do not have a precise hypothesis regarding timing mechanisms of the tapping tasks, since the distinction is less pronounced than for the perception block.To examine the most relevant predictors for responses per perturbed segment (CC or V) and perturbation condition (Onset vs. Coda condition), we make use of a machine learning technique by fitting regression trees to the data. Regression trees should provide insight into the most relevant predictors for splitting the data. Subsequently, linear models are fitted to the data with the predictors provided by the regression trees including their interactions to examine how well these predictors describe the variance in the data. In doing so, the regression tree analysis can be seen as an exploratory approach used for describing the most prominent qualitative features in the data. The linear models are then used as a confirmatory analysis to assess the robustness of the subdivisions into groups and provide the explained variance and statistical significance.n ≈ 29) to extract the most salient splitting criteria (predictors) of our data and to further model linear relationships between the predictors and the response data as suggested by the tree. Regression trees were fitted with the rpart function using the rpart package by rpart.plot function/package  and therefore reduces the risk of overfitting the data. For each node, the variable and a threshold along this variable are chosen to reduce the variance in the child nodes. At each splitting point, the variables are scanned for the error between the predicted and the measured values, and squared to get the sum of squared errors (SSE). The lowest SSE defines the splitting variable and the splitting point within the variable. This approach has some similarities with previous studies in which participants were split into groups of better performers vs. worse performers for a given variable. For example, Classification and Regression trees CART,  are form/package . With thmethod parameter to anova. The minimal number of participants for each split was set to four (minsplit), including the final splits (minbucket). Further, a cost complexity parameter (cp) has to be set to define the complexity of the tree. The cp decisively shapes the complexity of the tree and is a tuning parameter that should provide the best tree for predicting future data . A cp of 0 fits the most complex tree by predicting each observation (overfitting); a large cp might reduce the cross-validation error but increases the relative error and might underfit the data. To avoid underfitting of the small dataset, the cp per model was chosen based on the cross-validation error, but with a slightly greater tolerance toward a greater cross-validation error than in machine learning approaches. This approach was further motivated by the fact that in our case finding relationships in the data is of higher priority than actually predicting future data. Overfitting, in turn, was avoided by suppressing recursive splits of the same predictors that lead to a non-linear relation between the predictor and the outcome variable .1Four regression trees were fitted (one to each dataset) with the perturbation response data (compensation) as the dependent variable and PC1 Auditory Acuity, PC2 BAT Perception, PC1 Paced Motor Variability, and Unpaced Motor Variability as predictors, setting rpart’s The four tree models are presented in The quality of the predictions will be examined in turn for each of the four perturbed segments by looking at the tree models individually, supported by a more detailed visualization of each tree in The tree for Onset CC compensation shows the average amount of compensation (0.9%) for 28 participants in the blue box on the top . The firFor vowel compensation in the Onset condition, only one split was achieved, namely with PC1 Auditory Acuity . ParticiFor vowel compensation in the Coda condition , only UnFinally, the tree for CC compensation in the Coda Condition  was firsIn summary, the most relevant predictors for compensation in the Onset condition were the perception PCs, while for the Coda condition Unpaced Motor Variability was most relevant. Vowel prediction was generally less pronounced than CC prediction, as indicated by the overlap of the groups in the “Compensation” strips in lm function of R’s included stats package (v4.1.2). These aimed at providing an indication of the accuracy of the modeled trees without a test and a training set, but by including the predictors provided by the respective regression tree and their interaction into a linear regression model. The prediction of each linear model, i.e., predicted compensation values versus measured compensation values, is visualized in To further quantify the quality of the subdivisions achieved by the trees, linear models were fitted with the predictors derived from the regression tree analysis. Analyses were performed using the F = 3.28, p = 0.038] and explained 20.2% of the variance in the data (adjusted R-squared). The model revealed a significant contribution of PC2 BAT Perception to modeling the data , but no significant contribution of PC1 Auditory Acuity  nor the interaction between both Perception PCs .The linear model for CC compensation in the Onset condition was therefore modeled with compensation as the dependent variable, and both perception PCs and their interaction as predictors. The result indicated that the model was significant [F = 4.74, p = 0.038. The model revealed a significant contribution of PC1 Auditory Acuity to modeling the data .The linear model for vowel compensation in the Onset condition was computed with compensation as the dependent variable and PC1 Auditory Acuity as predictor. The model explained 11.8% of the variance in the data and was significant, F = 3.44, p = 0.075.Vowel compensation in the Coda condition was modeled with Unpaced Motor Variability as a predictor. Overall model fit was quite weak (adjusted R-squared = 0.083), and non-significant, F = 8.617, p < 0.001, and accounted for 47.8% of the variance. Unpaced Motor Variability contributed significantly to the model , and so did PC2 BAT Perception . The interaction between both predictors did not contribute significantly .CC compensation in the Coda condition was modeled with Unpaced Motor Variability and PC2 BAT Perception as predictors as well as their interaction. The model was significant, While the previous section indicated that non-verbal motor abilities relate to responses to temporal auditory feedback perturbation, one could ask if similar effects can be seen for speech motor variability and perturbation. The following section briefly examines temporal speech variability in the baseline of the perturbation experiment and its relation to compensatory behavior. For the assessment of speech motor variability, data from the perturbation experiment was examined. The coefficient of variation (standard deviation divided by the mean) of the word-normalized segment durations (V and CC) produced in the baseline phase per experiment condition (Onset/Coda) per participant was calculated. The first nine trials of the baseline were not included into calculations, as they were excluded in analyses in The present study investigated the connection of individual perceptual and general motor abilities to responses to temporal auditory feedback perturbation. Our results support the idea that individual perceptual abilities and individual capacities in precise motor execution both shape the speech production process. The extracted qualities were summarized by their underlying dimensions obtained from a principal component analysis and served as predictors in statistical modeling. The analyses followed an exploratory-confirmatory approach: Regression trees selected the most relevant predictors, which subsequently were included in linear modeling. In model prediction, beat-alignment perception, general auditory acuity, and motor variability of an unpaced tapping task explained variance in the perturbation response data. In general, the perceptual dimension generated the most prominent predictors for describing variance in response to temporal onset perturbation of syllables (applying to both complex onsets and the following vowel). In contrast, motor variability of unpaced tapping was most relevant to predict responses to temporally perturbed auditory feedback of syllable codas . This relationship supports our second hypothesis, suggesting that greater motor variability allows for more adaptation (whereby adaptation was only found in the Coda condition). Auditory acuity, on the other hand, was suggested to be relevant for responses in the online control as well as adaptation. Indeed, auditory acuity explained variance in compensation to the segments in the Onset condition, and the Coda CC in the Coda condition. Thereby, the vowel in the Onset condition and CC in the Coda condition were the second perturbed segment and therefore exposed to online effects induced by the stretching of the preceding segment. This result is in line with the findings by structural effects: Syllable onsets are articulatorily more stable than syllable codas and therefore less malleable in the face of an auditory feedback perturbation. The data of the current study suggest that structurally given malleability can be further modulated by individual motor stability in temporal auditory feedback perturbation. Especially, perturbation to coda segments (the prediction accuracy of R-squared 0.47% was by far the highest of our four conditions) showed that low motor variability  was linked to less compensatory responses, more precisely to less adaptation. This assumption is further supported by the correlation of the compensation measures in Coda condition with each other. Both segments seem to share a certain malleability, although this has to be interpreted carefully because they were also manipulated within one perturbation frame. For speakers with higher motor variability, better perceptual performance increased compensatory responses, and weaker perceptual performance weakened compensatory responses. This interplay indicates a tradeoff between perceptual and motor abilities, and supports the finding that better perceptual abilities do indeed lead to more compensation/adaptation, but that adaptation only occurs if a certain system malleability is given. For future paradigms that aim at capturing strong adaptive responses, it might be particularly revealing to focus on speakers with this specific combination of high auditory acuity and high motor variability, and provoke adaptive shortening responses rather than lengthening, since the latter might always contain not just an adaptive component but also an online response.Concerning the motor variability component, there is, to our knowledge, until now no study that explicitly investigated the connection between individual motor timing capacities and responses to perturbed feedback. However, the current study’s results align with our main conclusion drawn from Regarding the perceptual components, these findings are in line with previous studies that examined the link between auditory acuity and responses to spectral feedback alterations. For example, in speech motor variability. Certainly, these results need to be interpreted cautiously. While the unpaced tapping task tests pure, task-unspecific internal timing stability with a low-complexity motor task, speech production requires a complex coproduction of muscles, each of them allowing for variability/degrees of freedom in the execution. Further, speech variability measures were assessed from only 11 trials in the baseline phase, which might not give a solid mean for such analyses. Regarding the motor variability measure from finger tapping, we nevertheless admit that in using this measure as an indication for internal variability/malleability, we cannot precisely disentangle imprecision in motor execution from imprecision in the internal motor plan. However, PC1 Paced Motor Variability did not seem relevant in explaining the data of this study and was dropped as a predictor for compensation in the regression tree analyses. One might conclude here that the difference between paced and unpaced timing tasks is not the ability to precisely execute motor commands according to an internal plan, but rather the internal rhythmic representation that is needed in unpaced tapping but externally provided in paced tapping. By closely examining previous studies, the results turn out to be partially in line with investigations on spectral speech variability and compensation to spectral perturbation. In previous studies, compensation to spectral perturbations correlated with the variability of contrast of different speech sounds in production  oduction . Howeveroduction , 2011. Moduction . In thisoduction , 2011, PThe findings of the current study gave insights into how feedback and feedforward mechanisms in speech and non-speech are connected and how their interaction shapes timing in speech production. We also believe that the study provides a valuable foundation for guiding future studies in the selection of more targeted perception and tapping tests for similar research approaches. In particular, motor variability in tapping tasks seems worth further exploration; especially unpaced tapping as a measure of general internal motor stability should be considered. Regarding the significance of the current and similar studies, it should be noted that to date not much is known about the reproducibility of reactions to  auditory feedback perturbation. Saying this, there is certainly a need for establishing a firm understanding of how compensatory responses to the same perturbations vary within participant across multiple sessions. Further, in future investigations, it might be worth looking into groups of participants with different levels of auditory acuity and motor stability, such as musicians and non-musicians. In follow-up investigations, the perception staircase paradigm could be improved by using a 4-interval 2-alternative AABA design, which would probably provide more reliable threshold estimations than the 2-interval paradigm with catch-trials or an ABX paradigm . FinallyThe measurement data supporting the conclusions of this article will be made available by the authors, without undue reservation.The studies involving human participants were reviewed and approved by the Ethics Committee of the Medical Faculty of the Ludwig Maximilian University. The participants provided their written informed consent to participate in this study.MO, PH, SF, and SDB: study design, data processing, and writing. MO: data acquisition and analyses. All authors contributed to the article and approved the submitted version.


The internet of things, a collection of diversified distributed nodes, implies a varying choice of activities ranging from sleep monitoring and tracking of activities, to more complex activities such as data analytics and management. With an increase in scale comes even greater complexities, leading to significant challenges such as excess energy dissipation, which can lead to a decrease in IoT devices’ lifespan. Internet of things’ (IoT) multiple variable activities and ample data management greatly influence devices’ lifespan, making resource optimisation a necessity. Existing methods with respect to aspects of resource management and optimisation are limited in their concern of devices energy dissipation. This paper therefore proposes a decentralised approach, which contains an amalgamation of efficient clustering techniques, edge computing paradigms, and a hybrid algorithm, targeted at curbing resource optimisation problems and life span issues associated with IoT devices. The decentralised topology aimed at the resource optimisation of IoT places equal importance on resource allocation and resource scheduling, as opposed to existing methods, by incorporating aspects of the static (round robin), dynamic (resource-based), and clustering (particle swarm optimisation) algorithms, to provide a solid foundation for an optimised and secure IoT. The simulation constructs five test-case scenarios and uses performance indicators to evaluate the effects the proposed model has on resource optimisation in IoT. The simulation results indicate the superiority of the PSOR2B to the ant colony, the current centralised optimisation approach, LEACH, and C-LBCA. The internet of things (IoT), the umbrella word for extending the internet beyond smartphones and computers to a whole range of things such as appliances, smart sensors, cars, and traffic lights , requireReal-world scenarios can all be described by the resource allocation problem (RAP); several formulations for the RAP have been proposed in accordance with different problem scenarios ,8,9. RAPEdge nodes entailing varying devices also comprise sensor nodes; these sensor nodes are more prone to sensing coverage, that is, the sensors capacity of supervising a specified area of interest; connectivity, specifically sensors’ communication capacity; and energy-consumption problems owing to its rechargeable battery constraints . WirelesIn a bid to avoid the problem associated with potential node failure consequent of the centralised topology, the authors of  incorporIt is to this effect that this paper aims for further better resource optimisation for the heterogeneous IoT by integrating concepts of EC and clustering as well as a hybrid algorithm consisting of the static round robin (RR) and the dynamic resource-based (RB) algorithms. Amalgamating these concepts will aid in re-configuring IoT’s central topology into a decentralised topology, thereby mitigating the problems associated with energy consumption and quick death of nodes. This approach, taking into consideration the varying energy storage and computational capacity, will also lessen the workload of all the nodes within a cluster, thus ensuring energy dissipation is managed. The CH will be saddled with the responsibilities of providing the shortest path for resource allocation as well as offloading excesses to the cloud, thus guaranteeing the CH’s energy consumption is considerably lowered, consequently evading overall system performance degradation. The major contributions of this article areAn improved there-tiered framework for efficient resource optimisation in IoT. The framework, as opposed to existing frameworks, performs resource optimisation by placing equal importance on resource scheduling and resource allocation.A mathematical model incorporating the proposed framework. This model following the proposed framework takes into consideration the heterogeneous nature of IoT devices and harnesses their distributed nature to reduce nodal energy dissipation, subsequently prolonging the system’s lifespan by mitigating the quick death of nodes.An analysis of the simulation, depicting the effectiveness of the proposed model. The model’s ability to perform adequate load balancing generates results that illustrate its efficacy. Comparisons made with the current state of the art illustrates the superiority of the proposed model.The rest of the paper is organised as follows: This section further intricates related works and considers the lapses that need addressing. The section also contains the resource optimisation problem and motivation for the proposed model.IoT’s increasing popularity equates to the incumbent surplus of data generation that requires competent management schemes. Equally, with an increasing number of IoT devices connecting to the cloud, users’ expectations and demands for high-quality services have heightened. To efficiently meet these expectations that hinge on solutions of the prevailing RAPs, recent work has shifted focus to issues of the EC paradigm that may affect the effectiveness of service provision . IoT’s hClustering, an efficient topology control method, which balances the traffic load of the nodes consequently, improving the scalability and lifetime of system, has a lingering problem that may lead to quick death of the CHs, thereby degrading the overall performance of the system . When onControlling the problem associated with the quick death of nodes, especially CHs, will make room for the advantages associated with cluster-based systems to further impact the optimisation capacity of any given system. Cluster-based systems have multiple advantages, which make the resource optimisation of IoT systems necessary; such advantages include the discarding of redundant and uncorrelated data, thereby reducing energy consumption by nodes within the cluster; the easy management of routing, resulting in improved scalability; and the conservation of communication bandwidth. Clustering algorithms are also beneficial for the improvement of energy efficacy and a reduction in transmission delay . The cluParticle swarm optimisation (PSO), a stochastic optimisation technique based on the movement and intelligence of swarms, is an increasingly popular algorithm for solving resource-scheduling problems owing to its simplicity in operation and speed in convergence . PSO, alAn effective way of mitigating and or completely avoiding data collision is to introduce some communication protocols to schedule the data transmission . The rouResource-based (RB) algorithms, on the other hand, use a heuristic methodology by means of a greedy approach, prioritizing the allocation of resources from the largest to the lowest . That isThe dynamic RB algorithm’s ability to adapt gives the RR-RB hybrid algorithm the opening of becoming a controlled adaptive algorithm for load balancing and consequently the resource optimisation of IoT systems. The deficiency of falling into the local optimum experienced by the PSO algorithm is tackled in the proposed model by the incorporation of the RR-RB hybrid algorithm. This meld ensures the best path is always presented as the best of both worlds is deployed, a cluster that is both efficient and void of quick CH death due to excess energy dissipation, thus allocating resources, balancing load efficiently, and prolonging network lifespan.This subsection delves into the problem formulation; it additionally expounds the motivation for developing the simulation.IoT’s amalgamation of diversified devices implies a varying assortment of activities ranging from sleep monitoring to the tracking of activities. Devices people wear and use are becoming increasingly sophisticated, being able to connect to media accounts and track data, thereby helping enrich lives; however, with such activities comes ample data transfer and storage problems. With a vision of an all-communicating world, the number of connected devices and the user data-rate is to be increased by ten to a hundred times; the extended battery life by up to ten times for massive machine communication devices; and end-to-end latency by five times . The IoTThe value-added services and real time applications of IoT come with ample data generation and consumption; therefore, the major problems that require great attention are multidimensional optimisation problems, where concerns involve determining in real-time how to select the optimal service configuration and provide efficient edge service scheduling scheme . AdditioThe use of the EC paradigm increases security and quality assurance as it encourages the processing of data closer to source of generation, thus reducing the number of data transferred between a device and centralised node. The work of  shows thThis section presents the proposed model’s description in addition to the model’s algorithm. The focus is on the algorithms functionality in achieving resource optimisation for IoT.The architecture of the proposed model in a bottom up fashion, as portrayed in The proposed model intends to optimize resource allocation by incorporating aspects of the RR-RB hybrid algorithm, EC paradigm, and clustering (particle swarm optimisation) technique. The trio tackle the aforementioned complexities that come with IoT by scheduling and allocating resources to provide a solid foundation for an optimised and secure IoT. The trio algorithm accomplishes this by working together to efficiently bundle the edge nodes into clusters, designating a CH, and taking into consideration the resource/server rates and task/request size, thus sustaining set parameters. The scheduling and allocation processes require taking decisions, and the decision makers are comfortable in using Pareto-optimal solutions because if the final solution is not Pareto-optimal, it can be improved in at least one objective without deteriorating the solution quality in other objectives ,12,13.ith particle’s position in the tth iteration; ith particle’s velocity in the tth iteration; ith particle’s history best position, which has visited until the tth iteration; and g(t) is the particle swarm’s global best position of the tth iteration. Additionally, the key control performance parameters of the algorithm are w—inertia weight constant, used to control the impact of g(t). The RR-RB, on the other hand, further buttresses the allocation by routing requests/tasks to available resources/servers, making certain that the tendency of falling into local optimum is avoided.The allocation of resources is governed by the PSO as well as aspects of the RR-RB algorithms. The PSO determines the motion direction of each node by a kinetic equation that includes two fundamental properties: velocity l vector . The fitQ signify the quantum time, that is, the time the scheduler allows for a task to run; TTS the total task size coined by the edge nodes for execution; and TR the transfer rate. If the quantum time of the ith task in the jth resource is Q, the task size is TS, and the transfer rate is TR, then the turnaround time (TAT) can be expressed as:Q is defined as:TTS is the total task size, that is, the summation of all tasks’ (T) awaiting resources.The scheduling of resources governed by aspects of both the RR and RB algorithm ensures that all requests obtain an economical allocation of resources, that is, ensuring that requests/tasks are allocated a time slice/quantum time according to their size without prioritising requests, by this means providing the best performance in terms of average time. The hybrid avoids starvation or convoy effect, that is, slowing down because of several time-consuming processes. Finally, the hybrid allows for the allocation of requests to resources based on request size and resource capacity. This allocation criterion further improves resource optimisation for IoT. The quantum time is taken into consideration when scheduling resources. Let: Typical PSO clustering algorithms form clusters by allowing the nodes to join CHs based on locality, assuming that nodes are equally distributed. If the formation of clusters allows nodes to join the nearest CH, then the CHs of densely deployed areas will be overloaded with higher number of member nodes . The proA plethora of recent works on the PSO algorithm focus on using PSO as a means of finding the appropriate number of clusters according to data characteristics; the average trajectory and velocity of nodes; the time cost of nodes; the energy consumption of CHs; the lifetime of nodes; and the total residual energy of nodes ,13,17,18Unlike flat routing algorithms, the proposed model as depicted in Algorithm 1 PSO Algorithm for proposed model. 1: Initialize position  2: Check particle’s position and velocity 3: Evaluate current fitness 4: Mapping of the updated position to the corresponding particles adjusting minimum and maximum position if necessary 5: Obtain the fitness value of each particle 6: if  7: Update the velocity  8: Update the position  9: Update the fitness  10: Repeat Step 5 to Step 9 until it reaches maximum number of iterations is reached. 8: Particle with position closest to resource is designated cluster head 9: Cluster head broadcasts its position to adjacent nodes and completes cluster creationAlgorithm 1 is an algorithm showing how PSO works in the proposed model, and the flow chart represented in Assume the set of edge nodes are denoted by N = This section delivers the simulation setup and analysis of results acquired. The simulation setup describes the approach taken to evaluate the efficiency of the proposed model, and the analysis of results acquired from the simulation setup will give insight as to the efficacy of proposed PSOR2B model.The parameters and dataset remain the same in all test scenarios. The simulations were on one to one hundred (1–100) nodes and five (5) servers, thus reflecting corporeal situations where the edge nodes are multiple in comparison to the limited number of resources and buttressing the requirement for the management of limited resources. The nature of the model, which simulates a real-life situation, implies that all nodes have varying battery energies and are randomly deployed; the message and packet sizes vary with each request; the mode of communication is bidirectional—wireless, or not, M2M, and M2P; and the mode of communication to cloud and edge servers is wireless—M2M. In the simulation run, the simulation parameters as shown in TAT). The proposed model leveraging on the sever rates aims to balance resource allocation, consequently improving the server rate by allocating appropriate resources/servers to request/edge nodes. The TAT performance indicator, which is the amount of time it takes to complete a given process, is a worthy measure of the effectiveness of the proposed model as in effect it is the measure of the server/resource rate. As previously mentioned, the turnaround time (TAT) is expressed as Equation (5).Five (5) test case scenarios of the proposed model were simulated using the python programming language and the anaconda framework. These test case scenarios were initiated to evaluate the performance of the proposed mode are grouped into preliminary and secondary test scenarios. The preliminary scenario, which equates the first three test case scenarios, is concerned with establishing the relevance of the PSO algorithm. The secondary test scenario, which includes the last two test scenarios, is concerned with portraying PSOR2B’s efficacy over existing approaches. The scenarios enabled one to measure the effectiveness of the model as a means for the further optimisation of resources in IoT systems. The performance of the proposed model is assessed based on the returned turnaround time  was applied, and evident improvement over the random allocation can be seen. This result not only illustrates the improvements clustering has on resource optimisation in IoT, it also reaffirms the decision of incorporating PSO in the proposed model as it justifies the necessity for the use of clustering as an additional means of resource optimisation in IoT technologies.After the establishment of PSO’s alignment with the proposed model’s objective, and its superiority over the ant colony clustering algorithm, the second preliminary test conducted set out to measure the efficacy of the PSO clustering algorithm as an algorithm for resource optimisation in IoT. The first two preliminary test cases establish the dominance of PSO over Ant Colony and PSO’s effectiveness over random resource allocation; the aforementioned therefore aided the proposed model’s design. Assumptions made in the model’s design regarding the clustering of nodes and consequently the creation of CHs is that the server’s performance is a function of proximity to request on a Cartesian plane; thus, the CH and subsequent cluster formation was dependent on their proximity to servers, with the CH being the closest with favourable storage and processing capacity.TAT is over three times more efficient than that of its counterpart the PSO. This third instance proves the advantage of using the hybrid in the proposed model, and although PSO has multiple advantages and has better efficiency than the ant colony and random allocation algorithms, its efficiency is limited and can be further heightened.The third preliminary test scenario looks at the effects of the previous proposed hybrid algorithm contrasted against the PSO has on resource optimisation. Having established PSO’s ability to optimize resource allocation effectively as depicted in The results presented in TAT. An improved TAT suggests improved power rates/battery lifespan and performance. TAT; however, when PSO is incorporated, the TAT is seen to further reduce with the TAT closer to zero (0). This again shows the importance of the use of appropriate clustering techniques as an add-on and not stand-alone. It suffices to say therefore that the generated results depicted in The fourth (4th) test case therefore compares the trio combination  against the PSO, as well as the trio combination against the hybrid algorithm. In both cases, the aim is to draw attention to the effectiveness of the proposed PSOR2B model, emphasising how it produces more optimal results in terms of the The PSOR2B, a merger of the EC paradigm, clustering techniques and the RR-RB hybrid algorithm work together for efficient, flexible, and scalable resource optimisation in IoT. PSOR2B’s employment of the incorporated concepts ensures its adaptability and consequent efficiency in varying network conditions. The PSOR2B’s capacity as illustrated in the generated results, taking into consideration the devices’ varying storage, processing, and computation capacity, explicitly portrays its efficiency as a solution for the MORAP IoT problem. Despite these positives, security provided by the incorporated EC paradigm can be further improved. Future work can also explore the implementation of the PSOR2B model in real-life scenario. Although the results generated suggest PSOR2B’s scalability, especially in the number of requests it can handle, a further implementation with a focus on PSOR2B’s performance in large-scale IoT can be further explored.In this paper, an efficient resource optimisation algorithm for IoT labelled PSOR2B was developed. The algorithm utilizes aspects of the EC paradigm, the PSO clustering technique, round robin, and resource-based algorithms. Incorporating the advantages of the trio, the PSOR2B solves the resource optimisation problem in IoT, ensuring that equal importance is placed on resource allocation and resource scheduling. Given that IoT tends to grow in complexity with an increase in size, the designed and developed PSOR2B model abates the complexity problem by tackling the allocation and scheduling problems, consequently providing solutions for scalability and latency issues that directly influence QoS and users’ QoE.The experimental results show the proposed PSOR2B’s efficient solutions for the resource optimisation problem in comparison to the existing ant colony and the LEACH and C-LBCA algorithms. Once the right task is assigned to the right resource, an optimised turnaround time (TAT) is inevitable. An efficient execution time implies an excellent TAT, and an improved TAT indicates less energy dissipation at nodal levels. Less energy dissipation promotes a longer lifetime for all edge nodes and subsequently the entire system. The results indicating the efficiency of the PSOR2B imply that QoS and consequently users’ QoE is assured. The promising results inevitably show the effectiveness of using clustering algorithms on existing algorithms to further resource optimisation in IoT. It suffices to say that although certain parameters are controlled, the random nature of data transfer and communication between nodes are not controlled, therefore reflecting real-life situations. The proposed model can therefore be replicated easily into real life situations with little or no adjustments. The results generated have been able to shift the focus to factors worth more consideration, such as taking into account and placing equal importance on resource scheduling and resource allocation; the request size and server/resource size; how efficiently these resources are used; and how situations such as bottleneck can be avoided.Future work can further improve this model by making it secure, thus tackling the problems associated with IoT’s privacy and security, consequential of decentralising the IoT’s topology. The PSOR2B model can further be deployed into real life scenarios.


Tricuspid valve (TV) interventions face the challenge of imaging the anatomy and tools because of the ‘TEE-unfriendly’ nature of the TV. In edge-to-edge TV repair, a core step is to position the clip perpendicular to the coaptation gap. In this study, we provide a semi-automated method to localize the VC from Doppler intracardiac echo (ICE) imaging in a tracked 3D space, thus providing a pre-mapped location of the coaptation gap to assist device positioning.A magnetically tracked ICE probe with Doppler imaging capabilities is employed in this study for imaging three patient-specific TVs placed in a pulsatile heart phantom. For each of the valves, the ICE probe is positioned to image the maximum regurgitant flow for five cardiac cycles. An algorithm then extracts the regurgitation imaging and computes the exact location of the vena contracta on the image.Across the three pathological, patient-specific valves, the average distance error between the detected VC and the ground truth model is This study presented a method for ultrasound-based localization of vena contracta in 3D space. Mapping such anatomical landmarks has the potential to assist with device positioning and to simplify tricuspid valve interventions by providing more contextual information to the interventionalists, thus enhancing their spatial awareness. Additionally, ICE can be used to provide live US and Doppler imaging of the complex TV anatomy throughout the procedure. Previously labeled as the forgotten valve, the tricuspid valve (TV) and its repair surgeries have gained prominence recently –3. For tCurrently, there are several devices and procedures approved for tricuspid valve repair. These repair techniques can be classified into two major categories—annuloplasty and coaptation devices. The edge-to-edge repair techniques are particularly successful in treating severe TR with the benefits of reducing the need for hospitalization as a result of heart failure .Earlier success in TV repair includes the use of the MitraClip on the TV to reduce the regurgitation by at least one grade , 10. SinThe leaflet repair via the TriClip is performed percutaneously via transfemoral access, and while a transjugular approach has also been developed, the transfemoral approach has shown superior performance . The cliThe tricuspid valve and its interventions have been declared “TEE-unfriendly”. The TV is located anterior to the mitral valve, rendering it challenging to image using a TEE probe . The larICE ultrasound provides high-resolution imaging of cardiac structures. Unlike TEE, the insertion of an ICE probe can be performed under local anesthesia only and without the need for specialized operator. ICE is also well tolerated by the patients. The major drawback of this technology is the high cost of each single-use probe, but it has the potential to offer a better cost/benefit ratio, by reducing the procedure times and length of post-op hospitalization in patients. ICE has made its mark in interventional cardiology for structural heart diseases and electrophysiology –19. ICE Image-guided systems (IGS) have helped simplify many interventions, as well as having made them safer and more reproducible . In a meTo facilitate the positioning of a TriClip device, the identification and localization of coaptation gap is a crucial step that can potentially be simplified by pre-mapping the 3D location of the coaptation gap prior to the device being positioned. This mapped location serves as an important landmark during the TriClip positioning stage. In ultrasound imaging, the neck of the regurgitant jet, as seen in the color Doppler, is called the vena contracta (VC) and it corresponds to the location of the coaptation gap. In this study we aim to map the coaptation gap by localizing the vena contracta in Doppler ICE imaging.While there is currently no commercially available automatic VC and annulus detection system, several automatic VC quantification techniques have been published in the past for the assessment of mitral regurgitation using TEE  and TTE https://github.com/hareem-nisar/VC-localization.In order to assist the positioning of coaptation device at the site of regurgitation, we propose to use a tracked ICE probe with Doppler imaging to identify vena contracta from ultrasound images and representing its location in 3D space. Tracked devices can then navigate to reach the targeted vena contracta. To the best of our knowledge, this paper presents the first image-guidance system for tricuspid valve interventions. A proof of concept study, performed on a simple silicone wall phantom, has been conducted by our lab and accepted for publication at the SPIE 2022 conference. In this paper, we present a guidance system which uses ICE imaging and EM tracking technology to identify the site of regurgitation from a patient-specific tricuspid valve in a beating heart phantom. This system is developed on 3D Slicer and implemented as an open-source, one-click, 3D Slicer module. The module, as well as some test data along with a video demonstration, can be found at In this study, we used a 10-French, forward-looking and radial bilities , thus enTo achieve magnetic tracking (MT) of the ultrasound, we utilized the Aurora Tabletop Field Generator  and a 6-DoF sensor to track the ICE probe in 3D space during data collection.LV Plus Simulator  was used as a pulsatile heart phantom to simulate a ventricle and an atrial chamber. The phantom can be equipped with patient-specific valves, which includes the valve leaflets embedded in a silicone flange for support . The detThe negative mold of a silicone flange  with patient modeled tricuspid valves  was creaAn MT sensor was attached externally to a Foresight™ ICE probe using an adhesive. Prior to imaging, the ICE probe (in Doppler mode) was spatially calibrated using a point-to-line registration method , 36.The pulsatile heart phantom was placed over the table-top MT field generator Fig. . The phaImages were acquired from the Hummingbird console display using a frame-grabber  at a rate of 15 frames/second. The data were recorded using the Plus Server to communicate ultrasound and tracking information to 3D Slicer. For each of the three valves, five datasets were acquired, with each containing at least 5 seconds of imaging and tracking information. These data were processed to localize the vena contracta in 3D from the tracked, Doppler imaging of pathological tricuspid valves in real-time.The first step, to isolate the images with maximum regurgitation Fig. a, was peThis sequence of peak-regurgitant Doppler ultrasound images was then processed to remove all the grayscale, B-mode information from all the images. Since the objective is to isolate the vena contracta, the non-regurgitant blood flow (depicted in cool colors) was also removed from the images by suppressing the pixels with blue channel information. It must be noted that the quality of the regurgitant jet from individual cardiac cycles can sometimes be suboptimal. Therefore, to acquire an adequate jet image, all the images in the sequence were compounded together into one resultant image with the regurgitant flow Fig. c. This sThe resultant combined Doppler image contained the complete regurgitant jet, depicting blood flowing backward from the ventricle to the atrial chamber Fig. c, and waThe next step was to identify the axis of the regurgitant jet. The segmented region was subjected to principal component analysis (PCA) to identify the major and minor axis of the jet, as well the principal moments. This information was used to transform the segmented region to lie along the major axis Fig. b. The noFrom the transformed regurgitant jet, our proposed algorithm then identified the location of the vena contracta. At each point along the major axis, the height of the segmentation was measured, and the point with the minimum height was recorded. This minimum height was estimated as the vena contracta width (VCW), while the midpoint along the VCW was noted as the transformed vena contracta location. The inverse transformation from the PCA was applied to retrieve the original coordinates for the location of vena contracta in the US image. Figure The Foresight™ ICE images are conical in nature and lie in 3D space. The ICE image displayed on the console is a projection of the conical surface image along the height-axis. As such, the location of vena contracta on 2D images is not accurate and lacks the third dimension. Using the imaging angle information provided on the console screen, the location of the VC with respect to the true 3D image is calculated. The details of this conversion can be found in Nisar et al. . FinallyPrior to data collection for each valve, the ground truth VC and annulus were identified for validation. A pre-tracked and pre-calibrated needle was used to identify the VC in 3D tracking space, where the tip of the needle and the orientation of the needle shaft are tracked. The position of the ground truth of the VC was obtained by visually identifying and manually tracing the periphery of the regurgitant orifice using the tracked needle tip. The points were used to construct a 3D model of the ground truth vena contracta. Similarly, the outline of the annulus points was marked, and the model was constructed. The VC point locations detected by the algorithm were compared to the manually isolated ground truth VC model by estimating the closest distance between them.For each of the valves, the distance between the ground truth VC model and the ICE-derived vena contracta locations were computed. The distance error for all the datasets can be seen in Fig. Qualitatively, the position of the ground truth vena contracta, corresponding to the coaptation gap, can be seen as an irregular shaped body in yellow in Fig. During interventions, clinicians rely on anatomical landmarks to guide and align the tools properly. Traditionally, the identification of landmarks and the positioning of tools takes place simultaneously, thus making the procedure intricate and demanding. Pre-mapping these landmarks can simplify these procedures by providing more information to the clinician while they position the devices. In this study, we present a method to semi-automatically extract the location of vena contracta, a clinically relevant landmark, from ultrasound images and represent it in 3D space. ICE imaging is used to generate 3D models of important anatomical features to potentially enhance the spatial awareness of the interventionalist as well as give information about the relative positioning of the procedural tools with the cardiac anatomy. Moreover, the presence of ICE allows the clinicians to acquire up-close live ultrasound imaging as the procedure is being performed. Since ICE probes can be manipulated to acquire non-traditional anatomical views, the technique presented in this paper can be particularly useful during complicated TV edge-to-edge repairs where the clip has to be positioned at more challenging positions such as at the posteroseptal and anterioposterior commissures .The results from this study indicate that the designed 3D Slicer module can reliably localize the VC in most cases. Literature suggests that for cardiac interventions an error margin of up to 5 mm is acceptable . In compA major limitation of this study is the presence of the outliers when the algorithm is unable to identify the VC accurately and instead the VC is localized near the annulus. In a use case, an outlier can be easily identified when the detected VC was positioned too close to the TV annulus. Outliers indicate that the valve should be reimaged and processed by the algorithm again. We suspect these outliers to be a result of the lower frame rate used in the study which meant that the regurgitation was not captured in the imaging data. During the experiments, it was observed that the recorded data in Slicer lacked some of the imaging frames showing high regurgitation on the Hummingbird console screen. The frame grabber was operating at a rate of 15 frames per second and in some cases missed capturing the image frame with the maximum regurgitation. To record the complete regurgitant Doppler imaging, we recommend using a frame grabber with a higher frame rate. Ideally, the imaging data should be transmitted directly from the ultrasound machine but this infrastructure is not yet available in most of the clinical console, including the Hummingbird console, used in this study.A consideration while imaging the valve would be to use a narrower field of view for Doppler imaging to optimize and focus in the direction of the regurgitant jet. This simple factor can greatly enhance the overall efficiency of the designed algorithm.Besides the VC, the annular ring of the tricuspid valve is another important landmark during TV interventions. In this study we manually identified the annulus ring; however, the procedures can benefit from automated ultrasound-based techniques to identify the TV annulus in 3D space. Future work can involve implementation of the existing methods in the literature that can extract and model the annulus from ultrasound. Li et al.  present Since the valves used in this study are modeled after real patient-specific TV, there is room for collecting more and complex tricuspid regurgitation cases. The valve modeling technique and the beating heart phantom allow mimicking realistic conditions, reducing the need for in vivo testing at such an early stage of the study. With a variety of TV models, the algorithm can be made more robust by testing and modifying it to accommodate more versatile patient cases. Future work can involve making the Slicer module more robust and suitable for even more complex tricuspid valve pathologies. The ultrasound guidance approach can also be enhanced with the emerging 4D ICE technology, like VeriSight Pro (Philips) and NuVision (Biosense Webster), which provides improved imaging of the subvalvular apparatus during transcatheter TV repair.Tricuspid valve interventions and related technology are evolving as more cases are being performed with imaging being a major challenge in them. A suitable alternative to the existing TEE-based workflows is to employ ICE imaging to visualize the anatomy. In this paper, we presented a method to provide more contextual information to the interventionalists during the TV repair procedures to reduce the regurgitation. A tracked ICE probe can be used to localize and pre-map significant landmarks in order to assist the meticulous task of device positioning during TV repair. Image guidance systems with mapping technology have successfully simplified complex cardiac procedures like ablation therapy . This st


Bone scintigraphy with 99mTc labeled radiopharmaceuticals is a valuable method in nuclear medicine for assessing the bony structure. In clinical setting, bone scintigraphy is accomplished after the injection of 99mTc labeled diphosphonate complexes into a peripheral vein. Inadvertent intra-arterial injection on the antecubital region may cause a special form of artifacts leading to problems inaccurately interpreting these studies as functional images.We present a 44-year-old man with history of chest wall pain for bone scintigraphy as part of a work-up for determining the pain source. The patient received an injection of 740MBq 99mTc-methylene diphosphonate (MDP) into a blood vessel at the right forearm. Two hours later, an increased uptake of activity was observed on the right forearm and ulnar half of the wrist-hand in the whole body and spot images. The scan findings were consistent with the anatomical and physiological expectations of the ulnar arterial perfusion range. This case displays that an incidental injection with a 99mTc labeled diphosphonate into the ulnar artery results in a hot ulnar half of the palm and ulnar-sided three digits, because these areas are directly exposed to radiopharmaceutical, therefore more radiopharmaceutical is deposited.It is important for the physician, and especially for the nuclear medicine technician, to know the specific appearance created in the bone scan due to such an event so that they do not make a diagnostic mistake. Bone scintigraphy with 99mTc labeled radiopharmaceuticals is a valuable method in nuclear medicine for assessing the bony structure. In clinical setting, bone scintigraphy is accomplished following the injection of 99mTc labeled diphosphonate complexes into a peripheral vein. Today bone scintigraphy is commonly used to assess a wide range of osseous abnormality from benign to malignant diseases. Bone scintigraphy can be performed as a multiphase study to evaluate regional blood flow, soft tissue hyperemia and finally delayed images for osseous abnormalities , 2. The A 44-year-old man with history of chest wall pain was referred to our nuclear medicine center for bone scintigraphy as part of a work-up for determining the pain source. The patient's pain had started 2 months ago and there was no history of trauma or fracture recently. The patient received an injection of 740MBq 99mTc-methylene diphosphonate (MDP) into a blood vessel at the right forearm. Two hours later, an increased uptake of activity was observed on the right forearm and ulnar half of the wrist-hand in the whole body and spot images . Mild foIntra-arterial injection of a 99mTc labeled diphosphonate compounds such as Tc-99m MDP in the antecubital fossa into the brachial artery leads to in a glove-like pattern of dissemination of radiopharmaceuticals. After intra-arterial injection, radiopharmaceutical was circulated via the radial, ulnar, and interosseous arteries to the bones of the forearm, wrist, and hands and soft tissue distal to the antecubital fossa, creating the appearance of a glove . The effect of the first pass, when inadvertently injected into an artery, contributes to a different distribution pattern of the radiopharmaceutical. Radiopharmaceuticals are more likely to be uptake by the forearm or hand and cause the hot forearm or hot hand sign. The changes in image pattern are related to the involved artery and its perfusion . Because However, reflex sympathetic dystrophy, which mimics findings, should be excluded for differential diagnosis. Other differential diagnosis consists of frostbite damage and tourniquet effect , 4, 5. Ith metacarpal  and supplies blood to the third, fourth, and fifth fingers via the digital arteries.Thus, radiotracer from the right ulnar artery is deposited in the above-mentioned fingers as seen in our patient, explaining the hot right ulnar-sided three fingers. Besides the hot three digits, bone scan revealed increased uptake at the right forearm and hand which was significant on the ulnar half of the metacarpal region consistent with blood supplying from ulnar artery. No increased uptake was seen in the first and second metacarpals and fingers, because radial artery  is responsible for supplying blood to them. We report the interesting case of unintentional injection into the ulnar artery during bone scan and took a unique view of it. The scan findings were consistent with the anatomical and physiological expectations of the ulnar arterial perfusion range. This case shows that an injection with a bone-imaging radiopharmaceutical into the ulnar artery results in a hot ulnar half of the palm and ulnar-sided three digits, since these areas are directly exposed to the radiopharmaceutical, consequently, more radiopharmaceutical are deposited. Since, in our patient the specific view is seen as a hot half-hand, so this pattern is easily distinguished from the RSD, because the common pattern of the RSD is involvement of whole hand like a glove appearance. Physicians should be alert of intra-arterial injections before making a diagnostic assessment of asymmetric increased uptake, particularly if clinical information is insufficient. The injection site and vascular variations can even result in the different patterns of uptake . It is bIn conclusion a unique finding in this study is unintentional injection into the ulnar artery that has not been reported so far. It is important for the physician, and especially for the nuclear medicine technician to know the specific appearance created in the bone scan due to such an event so that they do not make a diagnostic mistake.