Leakage is undesirable in water distribution networks, as leaky pipes are financially costly both to water utilities and consumers. The ability to detect, locate, and quantify leaks can significantly improve the service delivered. Optical fibre sensors (OFS) have previously demonstrated their capabilities in performing real-time and continuous monitoring of pipe strength leak detection. However, the challenge remains due to the high labour cost and time-consuming process for the installation of optical fibre sensors to existing buried pipelines. The aim of this paper is to evaluate the feasibility of a submersible optical fibre-based pressure sensor that can be deployed without rigid bonding to the pipeline. This paper presents a set of experiments conducted using the proposed sensing strategy for leak detection. The calibrated optical fibre device was used to monitor the internal water pressure in a pipe with simultaneous verification from a pressure gauge. Two different pressure-based leak detection methods were explored. These leak detection methods were based on hydrostatic and pressure transient responses of the optical fibre pressure sensor. Experimental results aided in evaluating the functionality, reliability, and robustness of the submersible optical fibre pressure sensor. Optical fibre sensors (OFS), notably the fibre Bragg gratings (FBGs) and distributed optical fibre sensors (DOFS) have gained a reputation over the years and demonstrated their potential applications in pipeline structural health monitoring applications ,2,3,4. FBGs are well-known for point sensing. An interrogator is used to monitor the Bragg wavelength shift of intrinsic sensing elements, which have been inscribed into the core of an optical fibre. They are commonly used for monitoring localised physical quantities, such as strain, temperature, and pressure ,6. The dDOFS, namely distributed strain and temperature sensing (DSTS), distributed temperature sensing (DTS), and distributed acoustic sensing (DAS), offer distributed and continuous monitoring at every single point along a sensing cable. DOFS technologies have been developed based on measuring the intrinsic backscattering, including Rayleigh, Brillouin, and Raman scattering ,12, baseEisler et al.  proposedIt is noted that there are three most common fibre orientations used on pipelines, which are identified as (1) mounted or embedded in circumferential (hoop), (2) longitudinal , and (3) helical  directions along the pipeline. These three orientations are illustrated in Almost all of the optical fibre sensors’ deployment methods would require the optical fibre to be rigidly bonded to the pipe or embedded in the soil. These optical fibre deployment methods are feasible for new and smart pipelines (sensors-based self-monitoring pipeline). A challenge remains in the use of fibre sensors for the assessment of existing buried and old pipes. The present study presents a cost-effective sensor deployment strategy that can be utilised in buried and old pipes. The approach presented facilitates the application of optical fibre sensors when the defect location is not well defined along the pipes—a common practical problem. For this purpose, an “attachment-free” fibre deployment method is explored and examined. An optical fibre-based pressure sensor, based on the “attachment-free” concept, was designed and constructed. Investigation of the use of this sensor for the quantification of pipe leakage was performed. Based on observation and results, the feasibility of the design concept for indirectly quantifying pipe leak rates through the monitoring of the pressure transients are discussed.The submersible optical fibre-based pressure sensor developed in this work is a customised optical fibre-based prototype pressure transducer constructed at Monash University, Australia. The proposed pressure sensor can be deployed into the water pipes via a hydrant. The advantage of this deployment method is that the location of the sensors can be adjusted based on the needs, whereas the commercialised pressure transducer can only be installed on the location of the hydrant.A schematic of a single unit customised sensor is given in The optical fibre from the device in The schematic drawing of the experimental setup is presented in The optical fibre-based pressure sensor was first calibrated with the PVC pipe filled with water under a quasistatic condition. The ball valve was left open during the filling process to allow the air within the pipe to be exhausted. After the pipe was filled, the ball valve was closed. It is noted that there was a baseline offset due to the presence of water, as the sensor is sensitive to change in the surrounding temperature. As there was no thermal couple installed in our prototype sensor to compensate for the temperature change, it was proposed to reset the baseline before each experiment. Three pressurising and depressurising tests were conducted on the pipe by connecting and disconnecting the inlet of the pipe to the water supply with a known pressure (500 kPa). The optical fibre sensor was calibrated against the pressure transducer. The measurements were recorded at a sampling rate of 100 Hz.ODiSI was used to monitor the strain along the entire fibre sensors. The strain measurement along the fibre is plotted in the contours plot in As the fibre sensors are sensitive to both strain and temperature, the unbonded fibre was also monitored throughout the test so that the change in temperature can be compensated. By monitoring the unbonded region of the fibre, no significant change in temperature was measured throughout the test, as shown in Following calibration, the optical fibre device was verified under dynamic loading conditions. To achieve this test condition, a series of varying operating conditions (involving the connecting and disconnecting of the water supply) were used to generate the dynamic conditions. This allowed a validation of the dynamic response of the submersible optical fibre-based pressure sensor to be conducted. Results obtained from both pressure measuring systems are shown in (i)Determine the operating condition of the pipe (from both steady and unsteady pressure occurring within the pipe section);(ii)Perform leak detection when the pipe was subjected to hydrostatic pressure;(iii)Perform leak detection when the pipe was subjected to pressure transients;(iv)And detect the presence of pipe fracture (pipe burst).The submersible optical fibre-based pressure sensor demonstrated in Results for each of the experiments are summarised in the relevant sections below.A series of changes in operating conditions were carried out using the same experimental setup, shown in Next, the ball valve previously installed on the tee-junction see  was turnBefore the transient pressure dampened out due to the fast closing of the valve (Action 4), the ball valve was turned back on to approximately 45° and was held for 15 s (Action 5). In The two common pressure-based leak detection methods are the hydrostatic-based and transient-based leak detection methods. In the following section, the submersible optical fibre-based pressure sensor was used to demonstrate the capability of leak detection under both static and dynamic water pressure fluctuations by analysing the signature of the monitored pressure.Hydrostatic testing is a common leak detection method in the field. A pipe section will be subjected to a higher static pressure for a long period of time . The high pressure will accentuate all the features on the pipe and thus enhance the process of structural health assessment. Any abnormal pressure drop will indicate the presence of leakages . The preWhen the ball valve was opened partially, the internal water pressure dropped, as shown in m’ is the initial pressure maxima, ‘ξ’ is decay or damping coefficient, and ‘t’ is time. Wang et al. [In the last two decades, many fluid transient-based leak detection techniques have been developed ,35,36,37g et al.  have alsg et al.  have als2, 15.5 mm2, 40.5 mm2, and 71.4 mm2, respectively. Initially, the end cap without any damage (reference end cap) was installed on the tee-junction  to simulate a no leak situation. By turning on the valve that connects the pipe to the water supply with 500 kPa (Action 1), a transient event is generated and propagated into the pipe. The transient event was monitored using the optical fibre device at a data acquisition rate of 100 Hz. The experiment was then repeated with the four different (damaged) end caps  was performed on the results presented in Pipe burst is a very common failure mode for pipes in the field when defects in plastic pipes, or severely corroded pits and patches in cast iron pipes are subjected to excessive  loadings ,43. WithThe results thus far presented in this paper are only based on one (point) sensor with a pressure sensitive length of 50 mm and acquisition rate of 100 Hz. To the best of the authors’ knowledge, at this stage, there is no long-range distributed optical technique that offers both high spatial and temporal resolution simultaneously. A lower spatial resolution interrogator will measure a lower strain along the pressure sensitive length, which reduces the sensitivity of the developed device. However, discrete sensing, like FBG technology, may be adopted for practical field deployment. With FBG technology, the same sensor array can also easily be multiplexed to form an array of sensors that can be deployed as quasi-distributed sensing in the future. With an increment of sensing points, it can potentially improve the determination of the leaks and anomalies along the pipelines. Multiple  sensors with different monitoring purposes (DTS and DAS) can also be installed into the submersible pressure sensors so that a data fusion can be performed and hence reduce the chance of a false positive scenario ,45. It is always challenging to install optical fibre sensors to existing buried water pipelines. Almost all of the deployment methods of optical fibre sensors require the optical fibres to be mounted to the surface of the pipe to provide useful information for the condition assessment of the pipe and leak detection. Any damage to the optical fibre sensors would require digging up to access to the fibre sensors. An “attachment-free” deployment method was proposed and examined experimentally in this paper. A submersible optical fibre-based pressure sensor (prototype) was built and tested under laboratory conditions. The optical fibre device was first calibrated and successfully demonstrated the capability to monitor both the static and dynamic (transient pressure) water pressure. This paper also reported the application of the optical fibre device to perform a pressure-based leak detection method and detection of pipe burst. If the proposed sensors are damaged, a replacement can be done by extracting the whole sensors back from the insertion point. Given that ODiSI has sufficient spatial resolution to measure strain, as required for the working mechanism of the developed device, it was used to monitor the strain (or pressure) of the submersible optical fibre-based pressure sensor. However, the inner working mechanism of the sensor does not rely on this specific optical technique to measure the strain and temperature, as it might not be practical to be applied in the field due to the short monitoring range (up to 20 m for ODiSI). To the best of the authors’ knowledge, the technology able to offer both high spatial and temporal resolution over long distance is yet to be developed. The FBG technique will be more preferable for practical field application of the proposed sensing device at this stage. By adopting the FBG technology, this submersible sensor can easily be multiplexed so that the monitoring range could extend to over several kilometres. The DTS and DAS technologies can potentially be infused in the submersible sensor to offer multiparameter monitoring devices.In the presented work, PVC was chosen as the material for building our first prototype due to the ease of fabrication and cost. However, it is also well-known that PVC is nonlinear, can exhibit hysteresis and creep, and is a strongly temperature-dependent material. Further development and material selection of the sensor is ongoing to improve the accuracy and sensitivity of the sensor. This can be achieved by varying the geometry of the sensor and material of the host structure (where the fibre is attached). In future work, different types of casing material and short and long-term material effects will also be studied to extend the application for a commercial deployment. Since optical fibre sensors can be multiplexed, it is possible to duplicate the submersible optical fibre-based pressure sensor along the pipe. With more pressure-monitoring points, potential improvement of the location of leak/s and quantification of leak rates can be monitored and ultimately, better prediction of the remaining servicing life of the pipe can be achieved.


Despite of a large number of imaging techniques for the characterization of biological samples, no universal one has been reported yet. In this work, a data fusion approach was investigated for combining Raman spectroscopic data with matrix-assisted laser desorption/ionization (MALDI) mass spectrometric data. It betters the image analysis of biological samples because Raman and MALDI information can be complementary to each other. While MALDI spectrometry yields detailed information regarding the lipid content, Raman spectroscopy provides valuable information about the overall chemical composition of the sample. The combination of Raman spectroscopic and MALDI spectrometric imaging data helps distinguishing different regions within the sample with a higher precision than would be possible by using either technique. We demonstrate that a data weighting step within the data fusion is necessary to reveal additional spectral features. The selected weighting approach was evaluated by examining the proportions of variance within the data explained by the first principal components of a principal component analysis (PCA) and visualizing the PCA results for each data type and combined data. In summary, the presented data fusion approach provides a concrete guideline on how to combine Raman spectroscopic and MALDI spectrometric imaging data for biological analysis. Different analytical methods could be utilized for biomedical analysis  to highlight a certain aspect of the sample e.g., morphological microstructure, distribution of electronic chromophores, molecule classes, or special proteins. Among the label-free imaging approaches, matrix-assisted laser desorption/ionization (MALDI) spectrometry, and Raman microscopy are certainly among the most powerful imaging techniques for the investigation of biomedical samples. Raman spectroscopy is a non-destructive spectroscopic method, which provides complex molecular information about the general chemical composition of the sample with a rather high spatial resolution (Abbe limit) to highlight subcellular features  fusion of Raman and MALDI imaging data. Because the experimental implementation of correlated imaging is challenging in many aspects  of 10 μm cryosection. The sample was cut on a cryostat, and then dried on a precooled conductive ITO-coated glass slide. Subsequently, Raman spectra were obtained using a confocal Raman microscope CRM-alpha300R  and excited with a 633 nm HeNe laser (Melles Griot). The laser irradiation was adjusted in order to have about 10 mW power. The laser was coupled through an optical fiber into a Zeiss microscope. A spectral map was obtained by a raster scan with a 25 μm grid with a dwell time of 2 s and a pre-bleaching time of 1 s.We demonstrated the data fusion on an example dataset of MALDI spectrometric and Raman spectroscopic scans obtained from the same mouse brain sample  in 50% acetonitrile and 0.2% trifluoracetic acid. The ImagePrep station  was used to prepare and apply the matrix on the sample. The MALDI-time-of-flight (MALDI-TOF) spectrometric map was obtained on a Ultraflex III MALDI-TOF/TOF mass spectrometer . A “smartbeam” laser  was used. The spectrometer was calibrated with an external standard, a peptide calibration mixture . The measurements were performed in the positive reflectron mode with 500 shots per spectrum and spatial resolution of 75 μm.Further experimental details for both data types and an example of a hierarchical data fusion implementation can be found in the report by Bocklitz et al. . NeverthThe influence of corrupting effects  on Raman spectra cannot be avoided completely. Thus, the development of complex preprocessing routines  is an optional step. However, among smoothing methods, only the running median with a relatively large window is applicable for cosmic ray noise removal. Unfortunately, filtering with a large window may corrupt the Raman bands themselves. Alternatively, 2–3 spectra per point can be acquired to eliminate the spikes that are not present in each spectrum. Nevertheless, this approach increases the measurement time dramatically. Therefore, this approach is not suitable for Raman imaging when a large number of spectra are recorded. Thus, specialized spike correction approaches like wavelet transform  clipping algorithm  was used. The data processing was also performed in R  grid by utilizing a co-registration framework. Based on the false color images of Raman spectroscopic and MALDI spectrometric scans, 6 points clearly representing the same positions on every scan were manually selected. The coordinates of the Raman spectroscopic map were then transformed to the coordinate system of the MALDI mass spectrometric map. Subsequently, the Raman spectra were interpolated to the grid of the MALDI mass spectral map. To perform this interpolation, every point within the Raman grid was assigned to the nearest point within the MALDI grid. After that, the average of the Raman spectra, assigned to the same point within the MALDI grid, was calculated. Two spectral maps were thus obtained and aligned in a point-wise manner.After the alignment, the Raman spectroscopic and MALDI mass spectrometric imaging data were preprocessed. During the preprocessing, the wavenumber calibration of the Raman spectra and the phase correction of MALDI spectra were performed. The MALDI mass spectrometric imaging data were subsequently subjected to noise removal, background correction, and TIC normalization. The Raman spectra were corrected for fluorescence background and vector normalized. The SNIP algorithm was used for background estimation in both cases.l1-norms of the matrices, which are sums over the absolute values in the matrix. After the weighting, the data were combined in a single matrix and analyzed with a PCA. To illustrate the benefit of data fusion and weighting, we also analyzed the un-weighted data in a combined manner and each data type separately. We also investigated the case, where the same normalization approach was applied to both data types and no additional weighting is required. When the Raman spectra were normalized to the total spectral intensity, which is equivalent to TIC normalization of mass spectra, the data matrices had equal l1-norms.After the preprocessing, Raman and MALDI mass spectral data differed in their dimensionality and in dynamic range. Data with different dynamic ranges would contribute unequally in a further analysis and consequently the spectral matrices have to be additionally weighed before performing the PCA. The weighting coefficient was selected as a ratio between the Centralized architecture Figure . The preDecentralized architecture Figure . This scDistributed architecture Figure . Each daBoth Raman spectroscopic and MALDI mass spectrometric imaging data provide different insights into the chemical composition of the sample. Information on a broad range of molecules can be obtained from the Raman spectra. This information can be complemented by detailed information on lipid content, obtained from the MALDI data. To utilize both types of information together, a data fusion must be applied. This data fusion may be performed during different stages of the analysis workflow. Therefore, the architecture of the data processing workflow is dependent on the selected data fusion approach. These approaches can be divided into the following types , of which a large number of independent features have to be identified for appropriate description. By applying a PCA for dimension reduction, a large portion of the data variance is expected to be spread among multiple principal components (PCs) and the optimal approach should correspond to the slowest raise of the cumulative proportion of variance with a number of PCs.l1-norms. By definition, this norm is a sum of absolute values. It takes dimensionality and scaling of the data into account, so no additional weighting is required. TIC normalization performed on MALDI data is already equal to l1-normalization because there are no negative values present in the mass spectra. The right side of Figure The variances of the data explained by PCA are shown in the Figure l1-norms, are additionally highlighted in Figure To further investigate the influence of weighting on data fusion, the weighting coefficient was varied in a range from 1 to 20 and a PCA utilized for every case. The extracted curves of the cumulative proportion of the variance were organized as a surface plot Figure . To makeAlthough an optimal data fusion has been achieved as above-mentioned, a direct comparison of cumulative proportions of variance explained by the PCA for data with different dimensionalities may be misleading. Hence, the results obtained from the combined approach and separated data analysis Figure  were che−1), and CH deformation band . Although these two peaks may also be associated with Amide I and CH deformations of proteins, there is a decrease in the protein-associated range . Thus, the third PC of the combined data represents the actual diversity in the lipid composition of the sample. The relationship of the CH stretching region of the Raman spectra to the changes in the lipid content can also be observed by a high correlation of the Raman spectral region with MALDI mass spectra  protein backbone is located in the first two PCs at 816 cm−1 . Besides the establishment of a weighting approach, we discovered that a nearly optimal result compared to the weighting is achieved if the spectra of both types are normalized to the same norm. We could demonstrate this by normalizing both types of spectra of an example dataset to the same norm. This was the l1-norm in our example. However, it is important to keep in mind that this method of comparing the cumulative proportions of variance should be used only when a researcher is interested in maximizing the number of extracted independent features.In this paper, a data fusion scheme was investigated to analyze Raman spectroscopic and MALDI mass spectrometric imaging data together. We described the most significant corrupting effects influencing the analysis of Raman spectroscopic and MALDI mass spectrometric imaging data. The preprocessing workflows were shown for the suppression of these corrupting effects by means of calibration, noise reduction, background correction, and normalization for both data types. After the pretreatment steps, the importance of data weighting prior to data fusion is highlighted, especially when the data are obtained from different sources and have different scales and dimensionalities. As there is no universal way of balancing the influence of data types on the analysis, optimization, and validation of weighting approaches should be done according to the specific data. In order to allow a judgment of the quality of a weighting, we proposed an approach that allows estimating the goodness of data weighting. This approach is based on analyzing proportions of data variance explained by PCs and we applied this approach by examining the cumulative variance. It was shown that the weighting, based on the ratio of −1, respectively. Therefore, changes in the lipid to protein ratio and changes in lipid content itself can be observed simultaneously through the data fusion of Raman spectroscopic and MALDI mass spectrometric imaging data.The revealing of additional meaningful features by means of optimal data fusion was demonstrated for the combination of Raman spectroscopic and MALDI mass spectrometric imaging data. We showed this by comparing the third PC extracted from each type of data separately and from the combined data. The MALDI-related part of the third combined component showed a clearer interpretation in comparison to the third loading obtained from the MALDI data alone. Moreover, the Raman-related part of the combined component reflected variations in lipid to protein ratio. This PC depicts a decrease in a protein-associated range that occurs along with an increase of bands related to the CH deformation and C = C stretching in lipids, which can be found in the regions 1,128–1,284, 1,420–1,480, and 1,655–1,680 cmFinally, the advantage of the combined analysis was illustrated by a comparison of the PCA results visualized as false-color RGB images. These images were obtained separately for the preprocessed Raman and MALDI imaging data and for the combined data. Visual investigation of the images showed that the combined approach provides a sharper image with less noise contributions. This allows the conclusion that the data fusion increases reliability not only for the spectral but also for the spatial features present in the data.This research is based on already published data provided to the authors by Bocklitz et al. . For thiTB and JP initiated the study, supervised the study and discussed the results. OR performed the analysis including the development of the R scripts. TB performed the pre-study including the co-registration step. OR, JP, and TB wrote the manuscript.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Leishmania mexicana, a parasitic monoflagellate of the family Trypanosomatidae. ImageJ Macros for flagellar identification are provided, and high accuracy and remarkable throughput are achieved via this unsupervised method, obtaining results comparable in quality to previous studies of closely-related species but achieved without the need for precursory measurements or the development of a specialised heuristic, enabling in general the automated generation of digitised kinematic descriptions of flagellar beating from videomicroscopy.Ubiquitous in eukaryotic organisms, the flagellum is a well-studied organelle that is well-known to be responsible for motility in a variety of organisms. Commonly necessitated in their study is the capability to image and subsequently track the movement of one or more flagella using videomicroscopy, requiring digital isolation and location of the flagellum within a sequence of frames. Such a process in general currently requires some researcher input, providing some manual estimate or reliance on an experiment-specific heuristic to correctly identify and track the motion of a flagellum. Here we present a fully-automated method of flagellum identification from videomicroscopy based on the fact that the flagella are of approximately constant width when viewed by microscopy. We demonstrate the effectiveness of the algorithm by application to captured videomicroscopy of  Despite varying in function and also in lengthscale, conserved across eukaryotic flagella is the characteristic ‘9 + 2’ axoneme2. Formed of a central pair of singlet microtubules surrounded by nine microtubule doublets, this cytoskeletal structure is present along the length of the flagellum. This gives the organelle a well-defined diameter, although this may be obscured or complicated by the presence of accessory flagellar structures, such as the outer dense fibres of the mammalian spermatozoon3, or the paraflagellar rod present in the parasitic family Trypanosomatidae4. Despite structural additions to flagella, retained in typical optical videomicroscopy of flagellated organisms is an approximately constant flagellar width, as is exemplified in Fig. Well-studied and present across a wide range of organisms, the eukaryotic flagellum is typically a long slender organelle that is known to perform a variety of functional roles throughout nature, perhaps most notably in spermatozoa where the beating of one or more flagella can render a gamete motileCrithidia oncopelti5 and mammalian spermatozoon10, whether for use in examining motility or exploring the mechanics of a beating flagellum. In particular, obtaining a quantitative description of a flagellar beat constitutes a key part of, and a barrier to, further research, with significant human effort being dedicated to the problem. Researcher input ranges from early manual approaches akin to those of Ishijima et al.7 and Vernon and Woolley12, where flagellum tracing is done by hand using acetate overlays, to semi-automated or tailored methods16. The work of Wan et al.13 and Klindt et al.15 on the alga Chlamydomonas sees the flagellum extracted from experimentally-fixed cells using custom software, with Smith et al.6 utilising tuned image thresholding to ascertain the beating pattern of a spermatozoan flagellum, exploiting the high contrast between the spermatozoon body and flagellum specific to this microorganism and experimental setup. Present in each of these methods is a significant need for human input, be that in the calibration of a specialised algorithm or in the manual processing of each captured image. Thus there is scope for a fully-automated method for the identification and extraction of flagellar kinematics from videomicroscopy, and any such scheme would be of relevance to a wide scientific audience.Videomicroscopy of flagella at high resolution is pertinent to the study of many microswimmers, such as the classically-investigated 17, the semi-automatic ‘Bohboh’ software 18, the generalised linear models of Xiao et al.19, and the ‘active contour’ methods of Goldstein et al.20, Hongsheng Li et al.21, Xu et al.22. Whilst invaluable in the tracking of free filaments, such methods are less applicable to the study of a moving flagellated microswimmer, where current methods may be unable to distinguish automatically between the swimmer body and any attached flagella, with the exception of specialised methods such as the probabilistic fitting approach of Yang et al.23 in application to spermatozoa. Figure There are numerous semi and fully-automated methods for the digital tracking of filaments, capable of accurately describing the kinematics of even overlapping structures to sub-pixel precision, some examples being the commonly-used ‘FIESTA’ software25, and has been applied to object tracking problems26, whilst the related Radon transform is widely used in tomographic reconstruction28. Less well-known than the Hough or Radon transforms, the medial axis transform has seen use in studies of cellular morphology30, with the transform encoding the approximate size and shape of image features. Owing to the noted characteristic shape of flagella, there is significant potential utility in this shape-encoding image transform in the context of flagellum tracking and identification.Image transforms, such as the Hough and Radon transforms, are commonly used for feature extraction and image processing. A pertinent example, the Hough transform is used primarily as a method for detecting straight lines in binary imagesLeishmania mexicana promastigotes, flagellated human pathogens of the family Trypanosomatidae. Evaluating our scheme against existing semi-automatic software, we then demonstrate the applicability of the proposed method to a spermatozoon dataset, showing multi-organism efficacy, and suggest and comment on a number of possible refinements of the automated scheme, considering also the performance of the scheme on reduced-quality datasets.Hence, in this paper we present a general fully-automatic method for the identification of flagella in preprocessed videomicroscopy, focussing in particular on the segmentation of an axoneme from a binary frame that includes a free-swimming non-axonemal body. We exploit a feature of the axoneme biology that is conserved across species and identifiable using the medial axis transform, and apply the resulting automatic segmentation algorithm to a large captured dataset of free-swimming 2. The presence of this underlying structure along the entire length of the flagellum gives the filament a diameter that is approximately constant at typical optical microscopy resolutions, even when accessory structures are present. This is in stark contrast to what is in general the more-varied visible morphology of the cell body, as can be seen in Fig. L. mexicana promastigote. Thus the flagellar morphology may distinguish the slender organelle from the remainder of a cell, and hence we will aim to isolate the flagellum from videomicroscopy by identifying this feature, recognising it as a region of consistent visible width.As touched upon in the introduction, the 9 + 2 cytoskeletal structure present across eukaryotic flagella defines the slender cylindrical morphology of the organelleIn order to locate regions of constant width and subsequently identify them as axonemal filaments, we compute the medial axis transform of an image already processed into a binary representation. The medial axis transform may be thought of as the pixel-wise product of a Euclidean distance map and a skeletonisation, with the distance map encoding at each point the shortest distance to a point not included in the binary mask. As an example, consider the binary image shown in Fig. The results of the medial axis transform may be used to identify a region of constant width, noted previously  to be a conserved morphological feature of flagella. Figure However, in many cases derivative analysis and sophisticated region identification from the width-profile may be an unnecessary complication. As indicated by the histogram of Fig. (i)Preprocess into a binary mask of the entire cell;(ii)Perform and analyse a medial axis transform, isolating flagella by derivative or modal analysis;(iii)Trace the resulting isolated filaments.Hence we propose that the tracing of flagella from a single frame may be achieved by the following overall procedure:et al.17, Xiao et al.19 and Goldstein et al.20. The isolation of flagella via either the modal or derivative analysis suggested previously can also be perfomed without user input, with an example of a simplistic but effective automated modal analysis being provided in the Supplementary Information  and false detection rate (FDR) as used in the evaluation of the specialised sperm-tracing scheme of Yang 32. Motivated by our findings in the Appendices, we opt in this instance to utilise the described modal analysis, creating a fully automated scheme that is computationally-trivial but proves to be effective in practice. Sample macros may be found in the Supplementary Information, where image preprocessing and flagellum tracing have been implemented in a basic manner that is sufficient for exemplifying the flagellar identification method.The proposed computational procedure was implemented in the ImageJ macro languageet al.31 were processed without user input using Macro The 150,000 frames of the dataset of Walker L. mexicana flagellar kinematics.To demonstrate the efficacy of the algorithm a 100-frame sample of the results is presented in the Supplementary Information et al.23, approximately represents the inclusion of 1–3 additional pixels per frame that are not present in the output of Bohboh, and thus further demonstrates significant agreement between the output of our implementation and that of Bohboh. Hence the results of our proposed fully-automated methodology and implementation are in strong agreement with the output of an established scheme. Further, our proposed method is approximately 60 times faster than using the semi-automatic software, and notably does not require any per-frame user input. Thus the proposed scheme is seen to give results comparable to existing methods, but with greatly-reduced user input and accompanying overall processing time.Treating the reconstructed output of Bohboh as a baseline we compute the MDR and FDR as defined above, giving an MDR of 0.00% and an FDR of 1.92%. This minimal MDR corresponds to our implementation successfully identifying all of the regions found in the baseline, and hence correctly tracing the flagellum in each frame. The low FDR, comparable in magnitude to the 1.78% reported in the evaluation of the method of Yang et al.37. Example composite frames are shown in Fig. L. mexicana demonstrate remarkable accuracy, but we particularly note the presence of accessory structures to the flagellum. We see that the midpiece of the flagellum has consistently not been identified, owing to the greatly-increased width in this section, whilst the principal piece is accurately segmented. However, the midpiece is itself of approximately-constant visible width, thus we hypothesise that repeated modal analysis will be able to correctly identify and even distinguish between these different sections of the flagellum, and easily adapted from the current implementation.To highlight the applicability of the method and implementation to a variety of flagellated microorganisms we analyse a dataset of a swimming human spermatozoon, specifically Supplementary Movie et al.38, where image intensity is used to infer the 3-dimensional location of swimming spermatozoa. Our approach may then be readily extended to 3 spatial dimensions.Processing was performed automatically using Macro Leishmania sample introduced previously we perform a single image degradation, with examples shown in Fig. In order to investigate the applicability of our methodology to a range of imaging data we evaluate the effectiveness of our implementation on intentionally-degraded datasets, emulating typical features of reduced-quality imaging. To duplicates of the 100-frame The results of this testing are presented in Table Leishmania mexicana and the outer dense fibres of particular spermatozoa, and change in visible width at typical optical resolutions and magnifications is either not significant in general or may be further exploited to identify subsections of a flagellum. Hence we proposed a procedure for the automated identification of flagella-like structures from videomicroscopy that exploited the morphological feature of consistent observed width, utilising the medial axis transform for quantification.In this work we have presented and verified a method of flagellar identification from the videomicroscopy of free-swimming bodies. The formulation of a general method optimised for all flagellated organisms or imaging contexts is not possible due to the diversity of flagellated life, hence we have presented a high-quality and robust method for addressing the basic case - that of a body with a single flagellum - which can be readily adapted to a broad range of organisms and contexts. We note the presence of a structural backbone ubiquitous in eukaryotic flagella and cilia, giving these axonemal filaments a well-defined cross-section and therefore a width. Additionally, we acknowledge that the presence of accessory structures to the axoneme may significantly alter the ultrastructure of the organelle, and therefore the width observed in videomicroscopy. However, the effects of some such structures have been considered, as for the cases of the paraflagellar rod of To extract the location of a flagellum from the results of the medial axis transform we have proposed two simple schemes: derivative analysis of the width-profile, where consideration of local extrema of the first and second derivatives is expected to be able to identify the proximal end of a flagellum, and a less-complex modal analysis of the width distribution. Having implemented and verified the latter approach, remarking that it is the most suitable of the two, we note the scope for future work to refine the precise methodology used here by performing more-sophisticated statistical analysis, however we emphasise that even the basic scheme shown in this work was sufficient for adequate identification of flagella.Chlamydomonas. In the Appendices we examined the potential effectiveness of the proposed methodology on datasets in which flagella are observed to cross one another, along with suggesting a suitable methodological refinement and thus demonstrating potential applicability to dense flagellate populations. This difficulty could be overcome in practice by diluting suspensions where possible.Our approach is suitable for cases where an image can be binarised effectively, and we provide functional examples of preprocessing for typical samples of cells swimming in a background-free environment. However, any background subtraction or noise reduction approach could be used prior to using the algorithm described here. Successful identification of a flagellum here additionally depends upon the organelle being consistently contained within the focal plane, and may be less successful in processing datasets in which large out-of-plane deviations are consistently present - in the case of organism which uses an extensively 3-dimensional flagellum motion, a 2-dimensional video does not contain the necessary data for a complete analysis. Our approach is also likely unsuitable for use when processing numerous cilia/flagella that may not be easily distinguished, such as those of ciliated epithelia, but is in principle readily applicable to multi-flagellated microorganisms such as et al.17 and Xiao et al.19 respectively, in addition to using established thresholding methods for the creation of an initial binary mask. However, despite the simplistic implementations used here, the proposed procedure was shown to provide desirable accuracy at little computational or human cost, enabling rapid quantification of flagellar motion in a sizeable dataset. Indeed, whilst the presented approach identifies flagella in individual frames, the achieved throughput is sufficiently high for use in video analysis, and has been demonstrated to be robust to reductions in image quality.Preprocessing and flagellum tracing were implemented here in a basic manner, and we recognise extensive scope for the incorporation of more-refined methods of filament segmentation, such as the ‘FIESTA’ suite of tools and active contour methods of Ruhnow A potential refinement to the proposed scheme involves combining our morphological analysis with segmentation based on signal magnitudes, valuable in cases where thresholding-based segmentation may be informative but only partially applicable. We also suggest an application to the tracking of free-swimming microorganisms, where the analysis of the width-profile may yield the location of flagellar attachment and thus enable the tracking of swimmer motion.In summary, our proposed method for the identification of axonemal filaments has been demonstrated to be of high accuracy and able to be implemented as a fully-automated algorithm. We have verified our method on a large dataset captured of a free-swimming flagellate, and achieved a high throughput and accuracy without optimisation or refinement of a basic implementation. Notably reliant only on a conserved morphological feature of axonemal filaments, the considered procedure and implementations may potentially realise future study of the kinematics of a wide range of flagellated and ciliated microorganisms.Leishmania dataset provided in the Supplementary Information and comparing the results against a modal analysis of the same width-profile. We will evaluate performance on the simplest example of a flagellated swimmer, that with a single flagellum that does not appear to intersect itself, with each method’s performance on this test case expected to be indicative of its performance in more general settings. The modal analysis is implemented as in Macros We consider the viability of the derivative analysis suggested in the main text, evaluating a simple implementation on the sample Presented in Macro Leishmania dataset both methods of flagellum identification achieve comparable pixel-wise accuracy to by-eye segmentation, with the derivative-based method more-faithfully tracing the skeleton of the medial axis transform than the modal thresholding method but the resulting difference in output being less than 1% of pixels, with typically only a single pixel being affected. Though the derivative analysis gives a marginal increase in pixel-wise accuracy over our implemented modal analysis, on the sample dataset the derivative method correctly segments only 97% of the given frames, in comparison to the 100% success rate of the modal analysis. Further, the success of the derivative-based scheme is seen to be highly sensitive to the details of the derivative approximation and method of outlier detection, with the use of 2-step and 4-step foward difference operators correctly identifying the approximate point of flagellar attachment in only 71% and 95% of frames respectively. This sensitivity is expected to result in reduced general applicability of the derivative-based approach in comparison to the modal analysis, the latter having no comparable configurable parameters and noted to perform satisfactory automated segmentation.On the 100-frame sample Thus we conclude that a simple modal analysis is better suited for application to the test dataset than our derivative-based approach, with the robustness and frame coverage of the former outweighing slight increases in pixel-wise accuracy afforded by our derivative-based method. We therefore expect a modal approach to be of the most utility in analysing large and diverse datasets, where a robust and reliable method may be preferable to a highly-sensitive scheme with marginally-improved accuracy.Unseen in our analysed datasets but more prevalent in general, overlapping flagella present possible complications to the application of our proposed method of flagellum segmentation and tracing, requiring additional consideration or more-sophisticated tracing methods. Here we briefly discuss adaptations of the presented method to address two such complications, though we emphasise that full consideration of complex flagellar interactions requires significant future study.The apparent crossing of two flagella may intuitively be thought to increase the observable width of any binary representation at the crossing point, correspondingly increasing the value encoded in the medial axis transform around that point. From manual examination of a range of artificial flagellar intersections we remark that the crossings do not appear to significantly increase the value of the transform in a large number of cases.However, large variation may be observed when intersections are approximately tangential, potentially doubling the observable width encoded in the transform. Such cases may render the flagella indistinguishable from one another, but in more-favourable cases will simply cause a large local increase in the values of the transform. The modal implementation proposed in this study would then typically result in disconnected components of the transform, and thus would fail to identify the entire flagellar region.We propose a simple augmentation to the existing methodology: the linear stitching of any nearby isolated segments of the thresholded transform to restore connectivity. Limiting the radius of reconnection to the approximate flagellum width prevents the closure of any properly-segmented regions, whilst enabling the joining of regions of flagellum overlap. Initial evaluation of a simple implementation of this refinement step on artificial test cases gives significantly improved accuracy, enabling segmentation of a range of crossing scenarios. However, this scheme is unable to reliably segment approximately-tangential crossings, with future work needed to address this.Crossing and self-intersection of flagella results in an ill-defined transform arclength due to branching, potentially of issue to flagellum segmentation and tracing. However, the considered and preferred modal analysis is independent of transform topology and arclength, and thus only the post-processing step of flagellum tracing is affected. In Macro Supplementary InformationSupplementary Datasets and Macros


Fingerprinting localization approach is widely used in indoor positioning applications owing to its high reliability. However, the learning procedure of radio signals in fingerprinting is time-consuming and labor-intensive. In this paper, an affinity propagation clustering (APC)-based fingerprinting localization system with Gaussian process regression (GPR) is presented for a practical positioning system with the reduced offline workload and low online computation cost. The proposed system collects sparse received signal strength (RSS) data from the deployed Bluetooth low energy beacons and trains them with the Gaussian process model. As the signal estimation component, GPR predicts not only the mean RSS but also the variance, which indicates the uncertainty of the estimation. The predicted RSS and variance can be employed for probabilistic-based fingerprinting localization. As the clustering component, the APC minimizes the searching space of reference points on the testbed. Consequently, it also helps to reduce the localization estimation error and the computational cost of the positioning system. The proposed method is evaluated through real field deployments. Experimental results show that the proposed method can reduce the offline workload and increase localization accuracy with less computational cost. This method outperforms the existing methods owing to RSS prediction using GPR and RSS clustering using APC. Indoor location-based service (LBS) has been attracting significant interest in recent times owing to an increase in the number of smart devices and technologies being used. Pervasive applications such as behavior recognition, smart medication, and smart building require accurate position information of the users to yield accurate and timely services . AlthougTrilateration and fingerprinting localization are used in most IPS studies. Trilateration estimates the position of a tag device by observing its distance from multiple access points (APs). The distance is estimated by employing time-of-arrival between devices and APs or obtaining the RSS from the neighborhood devices. Here, a few nonlinear equations are solved to estimate the 2-D position of the tag device, where three non-collinear and non-collocated APs are located.Fingerprinting is based on scene analysis that consists of two phases: offline and online. A radio map consisting of both RSS information from the deployed APs and reference point (RP) locations is built in the offline phase and is preserved in a database. Subsequently, it is compared with online-observed RSS data in the online phase to deduce the tag location. Apart from RSS, channel state information  and geo-Many studies have endeavored to reduce the offline workload of fingerprinting localization. A self-guided robot armed with inertial measurement unit sensors is employed in reference  to exploIn this paper, we present a probabilistic framework for handling sparse training data through APC-based fingerprinting localization. Additionally, the offline workload is decreased by employing the Gaussian process regression (GPR) in the localization to predict the RSS at locations with no prior measurements. The radio map constructed by using limited training data is further divided into clusters or sub-areas by applying APC. The RSS clustering with APC helps to reduce the localization estimation error and computational complexity. BLE beacon, which consumes less energy, is used in this study as the indoor positioning technology.The remainder of this paper is organized as follows: An up-to-date radio map database is required for pragmatic fingerprinting localization applications. Conventionally, it is performed by manually acquiring RSS data at RPs across the testbed. Hence, the offline workload of traditional fingerprinting localization makes the system tedious, time-consuming, and labor-intensive. Whereas in the online phase, the probabilistic approach of fingerprinting positioning has good localization accuracy, but it also has high computational complexity.The use of regression to predict RSS data with training datasets helps reduce the offline workload of fingerprinting localization. Some representative approaches based on regression or interpolation include polynomial fitting, exponential fitting, and the log model . HoweverClustering helps to minimize the searching space of RPs on the online phase of fingerprinting localization. Some well-known clustering algorithms are K-means, fuzzy c-means, and APC ,24,25. AAlthough the GPR and APC are used in many kinds of literature, we find that there is no related literature that combines them to address the real issues of conventional fingerprinting localization like offline workload (labor intensive and time-consuming) and computational complexity. In our study, we use the APC algorithm along with a machine learning approach of fingerprinting localization. The mean RSS and variance are predicted using GPR while using little training data to construct a radio map. RSS clustering is performed on the radio map using APC, i.e., the testbed can be divided into clusters of RPs generated by APC. In the online phase, any exemplar RP is estimated as coarse localization. Subsequently, the RPs grouped under the exemplar RP are used for fine localization. Here, we evaluate two exemplar decision rules depending on the posterior probability and RSS distance as a metric to determine the exemplar RP.A Gaussian process (GP) is a stochastic process and is an extension of multivariate Gaussian to an infinite-sized collection of real-valued variables. The GP defines a distribution over functions from the view of function space ,31. Let The mean function, ptimized . Note thAs it is assumed in GP modeling that the data can be represented as a sample from a multivariate Gaussian distribution, it can be inferred thatHere, the posterior distribution, Before estimating the mean RSS and the variance, the GP model needs to be trained by optimizing the unknown hyperparameter vector mization ,34,35. Umization ,31:(9)loThe above function is intrinsic to GPR and balances between model complexity and data fit to avoid overly complex models. In this work, we use the limited memory BFGS-B algorithm to solve the optimization problem ,37.APC divides a set of elements into clusters and selects an exemplar to represent each cluster . In IPS,APC uses a pairwise similarity The self-similarity or preference , The proposed APC-based fingerprinting localization method adopting GPR has two phases of operations, namely offline and online. In the offline phase, we collect the training data from a few sparse RPs that are a subset of the total RPs on the testbed. The testbed is divided into The observed online RSS data at the unknown location inside the testbed can be represented as follows:The framework of the proposed localization system is illustrated in Each RP is assigned a probability computed as the probability of Here,In reference (18), given the online RSS, any RP in RPs with their respective probabilities.Second, the final localization estimation is obtained as a weighted sum of The experimental results are presented in this section. The offline measurements were obtained on the third floor of the IT building, Chosun University, South Korea. The testbed considered is an academic environment with frequent movement of people along the hallways. Six BLE beacons were deployed to cover the whole range of the testbed area. All the beacons were set to have an advertisement interval of 300 ms and beacon transmission power of +4 dBm. The testbed area with the deployed beacons is shown in lid data .After the GPR operation, the predicted mean RSS was fed to the APC for RSS clustering. While updating the responsibility and availability messages in APC, we evaluated the APC operation by varying the damping factor γ = 0.6 yields the APC result with the least number of iterations or the least computational cost. We used this value for the localization estimation in our work. The cumulative probability of the localization estimation error in the testbed area is shown in We evaluated the localization methods under the deployment of different numbers of beacons on the testbed, i.e., 4, 5, and 6 beacons. For the deployment scenarios of 4 and 5 beacons in the testbed, {vely see . As showFaulty exemplar estimation in the online phase: Owing to the fluctuations in the received RSS, the system may lead to the estimation of an RP as an exemplar that is physically far from the real position of the tag device. As the RPs grouped under the cluster head or exemplar are responsible for the final localization estimation, the wrong estimation of exemplar eventually adds some localization estimation error.Outliers in the offline phase: The APC algorithm might lead to an RP that belongs to a cluster, but is physically far from the cluster head . This prAs shown in The localization estimation error of the typical GPR method is larger than that of the Horus (with APC) method. However, recording the RSS distribution at RPs in the Horus method is time-consuming. The localization accuracy of the typical GPR method can be improved by increasing the number of training data (is work) . HoweverFinally, we discuss the complexity of our proposed method and the conventional probabilistic fingerprinting method. For the conventional probabilistic fingerprinting method, its complexity is Despite the reliability of fingerprinting localization in terms of localization accuracy, the main issue with this method is the offline workload that is both tedious and time-consuming. We have employed GPR to overcome the labor-intensive task of the traditional fingerprinting method. In addition, we have performed clustering on the predicted RSS data to divide the whole testbed into many subsections or clusters, which helps to minimize the computational cost of online positioning. The choice of GPR for regression is due to its property of yielding both the mean RSS and the variance, which can be used for probabilistic fingerprinting. APC is selected for clustering owing to its initialization-independent property and better selection of cluster heads compared with K-means clustering.We used BLE beacons to implement the proposed method considering that they are embedded in many current smartphones and consume less energy than other technologies. We evaluated two different cluster head estimation approaches in the online phase. The experimental result on our testbed shows that the RSS-distance-based exemplar decision method performs better for estimating the cluster head. The positioning result of the proposed method is better than that of the existing methods.


While developing semiconductors, post-silicon validation is an important step to identify the errors that are not detected during the pre-silicon verification and manufacturing testing phases. When the design complexity increases, the required debug time also increases because additional debug data are required to identify the errors. In this study, we present a debug scheme that improves the error identification capability. The proposed debug approach concurrently generates three types of signatures using hierarchical multiple-input signature registers (MISRs). The error-suspect debug cycles are determined by analyzing the debug cycles that are commonly contained in the erroneous signatures of the three MISRs. To reduce the amount of debug data, we compare the high-level MISR signatures in real time with the golden signatures; further, we handle the remaining two MISRs based on the tag bits that are obtained from the results of the high-level MISR. The experimental results prove that the proposed debug structure can significantly improve the error identification capability using less debug data than that used in previous debug structure. Recent advances in semiconductor manufacturing technologies have caused the development of improved designs that contain faster and more diverse functions. However, these developments have also enabled the creation of more complex designs that require additional logic. To produce these complex designs, an additional effort is required on the part of the designers . The incBased on the controllability of error occurrence, post-silicon debug is generally classified into two phases: nondeterministic and deterministic , 5, 6. IIn this study, we propose an improved debug scheme to reduce the amount of debug data. Our proposed debug scheme is based on hierarchical compaction using three MISRs. The usage of multiple hierarchical MISRs increases the observability and reduces the number of error-suspect debug cycles in case of error identification. Additionally, a reduction in the amount of debug data causes a reduction in the debug time.The debug scheme in  was propTo reduce the amount of debug data, the debug method proposed in  reduces The debug technique in  is propoWe propose a debug scheme that applies hierarchical compaction using multiple MISRs and that improves the error identification capability in comparison with that used in the previous techniques. In order to tackle the results of the previous works, the proposed debug scheme supports not only the deterministic debug phase but also the nondeterministic debug phase. Because the error occurrence can be varied during multiple debug runs in the nondeterministic environment, reducing the debug runs is efficient to adapt the nondeterministic environment. In the proposed debug scheme, the error-suspect debug cycles are identified after the first debug run. On the contrary, repeatable debug runs are required to observe the error-suspect interval for the identification of the error-suspect debug cycles in the debug techniques in , 11. In In the proposed debug scheme, the high-level MISR signatures are compared with the golden signatures in the trace buffer using an on-chip comparator to predecide the error-suspect debug intervals. The remaining two low-level MISRs are used to selectively compact the debug cycles based on the result of predecision in the high-level MISR. The commonly contained debug cycles among the low-level MISR signatures are determined as the error-suspect debug cycles. In this case, the number of commonly contained debug cycles can be reduced because the length of the error-suspect intervals to generate the low-level MISR signatures is reduced by the on-chip comparison using high-level MISR structure. Thus, we can effectively reduce the number of unnecessary error-suspect debug cycles that are required to identify the erroneous debug cycles. At the same time, the required number of debug sessions to observe the error-suspect intervals is reduced in the overall debug operation. For this reason, the proposed debug scheme can be adapted to the variation of the nondeterministic error occurrence during the debug operation. Additionally, we apply a tag map to reduce the number of tag bits in the proposed debug scheme. Therefore, the debug time also decreases because the amount of debug data required for error identification is less than that required in case of the previous debug techniques.Starting with the overall debug process, this section presents the proposed debug scheme in detail. The proposed debug scheme contains the debug operation and the debug structure.Ci). In this case, the number of samples per signature (SPS) is set to eight. The signatures generated by the high-level MISR (XSk) are transmitted to the on-chip comparator and are compared with the golden signatures (GSk) that are prepared in the trace buffer. In this case, k means the k-th generated signature. Simultaneously, the trace debug cycles are compacted by the low-level MISRs and are temporarily stored in the signature register. Further, the comparator generates a parent tag bit (PTk) as a result of the comparison. If PTk is 1, the low-level MISR signatures that are stored in the signature register are transferred to the trace buffer. However, if PTk is 0, the low-level MISR signatures are discarded, and the signature register is prepared to overwrite the stored low-level MISR signatures. The generated parent tag bit is also transferred to the external debugger. In this example, C3, C19, and C21 are assumed to be erroneous debug cycles. After completing the on-chip comparison of XSk and GSk, the signatures XS1 and XS3 are observed to be different from the golden signatures, as depicted in PT1 and PT3 are denoted by 1. In this case, the trace debug cycles, C1-C8, and C17-C24, compacted by low-level MISRs are stored in the trace buffer. Additionally, the area of GSk that is already compared with the high-level MISR signature in the trace buffer can be overwritten to ensure the storage of low-level MISR signatures.The proposed scheme comprises three MISRs as described in the previous section. The high-level MISR is used to directly compact the overall debug cycles. In this case, the consecutive debug cycles are compacted by the high-level MISR with the compaction ratio computed as the number of overall debug cycles divided by the trace buffer depth. In the proposed debug scheme, three types of signatures are generated by a hierarchical MISR. The concept of hierarchical MISR structure can be similar with the MISR composition in . In the YSj for L_MISR1 and ZSj for L_MISR2) and the analysis of the error-suspect debug cycles. The ZSj of L_MISR2 is the delayed signature that is compacted at regular intervals from YSj. The trace debug cycles are compacted, and the generated signatures are stored in the signature register, as depicted in YSj and ZSj are  is 219 cycles. Various trace buffer depths  are used to perform simulations with five error rates . The error rates are estimated using E/N, where E is the number of erroneous cycles [In this section, we evaluate the proposed debug scheme and compare the proposed scheme with a previous method that is described in  based onFirst, the error identification capability depends on the number of error-suspect debug cycles that is required to identify the exact erroneous debug cycles during the debug operation for the overall debug cycles. Nin × 1/fin + Nex × 1/fex, where fin denotes the internal operational frequency, Nin denotes the total number of trace debug cycles, and Nex denotes the number of debug data cycles that are transferred between the trace buffer and the external debugger during the overall debug process. The external frequency is given by fex if TAP is used to transfer the debug data between the trace buffer and the external debugger. In this case, the ratio between the internal and external operation frequencies is assumed to be 10:1. It is usually difficult to temporarily pause the data-tracing process during the hardware debug. Therefore, data tracing should be reset to the initial state after completing one debug process within the observation window. Therefore, we repeat the front observations of the debug order until the debug operations of all of the trace data are terminated.To evaluate the overall debug time, the total amount of debug data is categorized based on internal tracing and external transmission during the overall debug process. Further, the overall debug time is given by In this study, we propose a debug scheme that can reduce the amount of debug data that is required to identify the erroneous debug cycles. We can obtain a better error identification capability using hierarchical MISRs for decreasing the commonly contained error-suspect debug data from among the generated MISR signatures. Therefore, we can decrease the unnecessary debug data transfer from debug operations. All these factors reduces the debug time, and the proposed debug scheme accelerates the post-silicon validation.


This research topic intended to highlight different philosophies and techniques for the correction of proximal hypospadias, a topic that continues to generate great controversy among pediatric urologists. Unfortunately, there were only four papers accepted to be included in this collection.One paper dealt with the results of one-stage repair of mid- and proximal hypospadias, a second paper dealt with a technique to facilitate second-stage repairs when the urethral plate created in the first stage with a free graft was difficult to tabularize, a third paper reviewed the current status of tissue engineering for urethral reconstruction, and the fourth one dealt with the results of a modified Mathieu technique for repair of the megameatus/intact prepuce variant of hypospadias.González et al.). The technique consists of the preservation of the urethral plate whenever possible (48 of the 49 patients) and then creating an onlay urethroplasty using the outer preputial layer of a total preputial flap transposed ventrally through a buttonhole in its pedicle to avoid rotation. The penile curvature was corrected with chordectomy alone in 10 patients; 38 patients required a dorsal plication of the tunica albuginea and 1 patient required an additional ventral dermal graft. Success, defined as a straight penis, a glanular meatus, and the absence of voiding symptoms, was achieved in 77.5% of the patients with a single procedure. Complications included urethrocutaneous fistulas in 16% of the patients, glans dehiscence in 6% of the patients, and recurrent chordee in one patient. Of the 13 patients that had uroflow examinations, 11 had bell-shaped curves. This report would suggest that one-stage repairs can achieve good results with fewer operations than the more commonly used staged techniques. Shortcomings of this report include relative short follow-up and lack of objective cosmetic evaluation.Authors from Germany (Hannover and Berlin) reported the results (measured by reoperation rate) in 49 children operated with a technique first described by the senior author in 1996 and followed for 2 years after the operation . The original idea in this report, however, is based on the fact that in two-thirds of the patients, a midline incision of the graft was considered necessary to achieve tubularization during the second stage. With a mean follow-up of 18 months, the complication rate was 22%, similar to that reported in the previously commented paper for a single-stage repair. The complication rate in this series was greater with the use of buccal mucosa than with the use of a free preputial graft. Complication rate in cases in which the plate was not incised  was just 8.3%. Despite these results, the authors conclude that a two-stage repair is the procedure of choice in the correction of severe hypospadias. Other shortcomings of this report include short follow-up, lack of objective criteria to decide to incise the plate, and lack of objective evaluation of the results of the urethroplasty, such as uroflow studies or calibration. Therefore, this paper adds a trick to facilitate the second stage, but the authors fail to demonstrate the real value of the maneuver. Perhaps it would be more productive to emphasize the need to have a very generous graft in the first stage and to wait until the graft is soft and pliable to perform the second stage.A group from Vienna reported on 35 consecutive patients from a cohort of 250 hypospadias cases who completed both steps of a staged repair (Cendron). Twenty-five patients operated in a 10-year period were divided into two groups. The first 10 cases underwent simple tubularization of the plate. One fistula developed. The next 15 cases underwent repair using a metal-based flap. There were no fistulas, but one patient experienced dehiscence of the glans. The author favors the metal-based flap technique, because in this series, it eliminated injury of the urethral plate during the dissection. Of course, the numbers are too small to draw definitive conclusions, but given the rarity of this condition, the author's opinion is worth considering.A surgeon from Boston describes his experience with the use of a metal-based flap to repair the megameatus intact prepuce variant of hypospadias (MIP) . The clinical experience cited is limited, but tissue engineering will certainly become a reality in the future that will benefit patients with undesirable results from surgical misadventures where local tissues are not available for reconstruction. The authors conclude that despite many significant advances, the search for a suitable tissue engineering approach for use in routine clinical applications continues.A paper by authors from Qatar and Denmark analyzed the current knowledge of tissue-engineered material for urethral replacement (Frontiers provides a comprehensive review of this topic and is highly recommended for surgeons interested in this topic (A vital part of successful correction of severe hypospadias is the straightening of the penis. Although not included in this research topic, a recent article in is topic .All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


A human gesture prediction system can be used to estimate human gestures in advance of the actual action to reduce delays in interactive systems. Hand gestures are particularly necessary for human–computer interaction. Therefore, the gesture prediction system must be able to capture hand movements that are both complex and quick. We have already reported a method that allows strain sensors and wearable devices to be fabricated in a simple and easy manner using pyrolytic graphite sheets (PGSs). The wearable electronics could detect various types of human gestures with high sensitivity, high durability, and fast response. In this study, we demonstrated hand gesture prediction by artificial neural networks (ANNs) using gesture data obtained from data gloves based on PGSs. Our experiments entailed measuring the hand gestures of subjects for learning purposes and we used these data to create four-layered ANNs, which enabled the proposed system to successfully predict hand gestures in real time. A comparison of the proposed method with other algorithms using temporal data analysis suggested that the hand gesture prediction system using ANNs would be able to forecast various types of hand gestures using resistance data obtained from wearable devices based on PGSs. The recognition of human gestures with the aid of wearable devices has found widespread application in modern manufacturing systems ,2 and smWe previously reported a simple and facile method to fabricate strain sensors and wearable devices using pyrolytic graphite sheets (PGSs) . These PWe introduced various types of ANNs, such as TDNNs and RNNs, for gesture prediction. ANNs are parallel information processing methods for learning complex cause-and-effect relationships between input and output data. In particular, TDNNs and RNNs are well-defined prediction models that provide insight into dynamic functions. In TDNNs, the dynamics of the model are stored using a time-delayed tap, and the nodes are updated by using recursive feedback from the output to the input at the nodal levels for each iterative cycle. TDNNs can only model data within a fixed-size sliding temporal window and are incapable of handling longer-term dependencies . On the First, we fabricated a data glove comprising three independent PGS strain sensors assembled on a commercially available ultrathin glove. These PGS strain sensors were homemade. The strain sensors were positioned over the second joints of the index, middle, and little fingers. In this setting, only these three fingers were considered because the index and middle fingers contribute most to functional gestures such as a closed fist, flat hand, and scissor gesture . The litThe proposed system consists of the following three main components: The data glove, a data acquisition device, and a PC, as shown in The TDNN-based model was established for the prediction of each finger motion to ensure high-precision prediction. The temporal dynamics in the sensor signal are learned by training the TDNN model on resistance datasets consisting of data representing the flexion and extension of each finger motion. As shown in a, b, and c are coefficients of the first-order linear function.The architecture of the RNN model used in our experiments was configured with 60 input units, 50 units in the first hidden layer, 70 units in the second hidden layer, and 3 output units. In addition, this RNN model also contained a context layer, which is known as a memory layer, in which the output previously generated by the first hidden layer is stored. The output of each hidden unit is copied into a specific unit in the context layer. The output of the context neuron is used as an additional input signal for all the units in the first hidden layer one time step later. Training data are important cues for hand gesture prediction. The key requirement is that the gesture dataset should contain various gesture patterns. Eight healthy male participants took part in our data collection experiments. All our resistance datasets contained approximately 50,000 resistance data from each participant. To learn the temporal dynamics of a hand gesture, we collected five datasets, each with a different resistance, consisting of the flexion and extension of each finger motion. Each of these five training datasets, which are summarized in The predictive ability of the TDNN model was verified by demonstrating the hand gesture prediction capability of the model in real time. We evaluated the predicted signal accuracy by using the mean absolute percentage error (MAPE) . FurtherWe verified the adequacy of the prediction performance by comparing the prediction results of the TDNN model with those of the RNN and MLR models using k-fold cross-validation . In thisIn the study presented in this paper, we investigated a hand gesture prediction system using hand gesture data obtained in real time from data gloves equipped with PGSs. The low-cost data glove incorporating the PGSs was fabricated by using a simple and facile method, and could accurately detect quick hand gestures in combination with the TDNN model. To demonstrate the potential of the hand gesture prediction system, we computed the 10-step (100 ms) and 30-step (300 ms) ahead predicted resistance in real time. The predicted resistance can determine the expected trend of an actual hand gesture without any obvious lag. As a result, the prediction system is able to successfully predict the hand gesture. Moreover, we measured the unbiased prediction accuracy of the TDNN, RNN, and MLR models by evaluating the performance of these models using a seven-fold cross-validation procedure. A comparison of the results of the TDNN model indicated that the impact of using different datasets for training and testing purposes was negligibly small. This confirmed the superior performance of the TDNN model and its ability to predict various hand gestures successfully. In addition, the results showed that the TDNN and RNN models are more suitable than the MLR model, and that the RNN model delivers the best prediction performance overall. These results suggest that the proposed hand gesture prediction system based on an ANN with PGSs holds immense potential for further development of existing hand gesture recognition techniques, especially for Ambient-Assisted Living (AAL) applications. This work is a pilot and feasibility work with the limited size of the dataset. Therefore, future research will seek to further validate the proposed framework in a larger dataset. Moreover, we plan to extend the structure of ANN models to enable them to accept and process various types of input data, such as the resistance data in combination with the bending angle of each finger.


Learning from nature, three types of hierarchical structures  have been developed and investigated.Micro- and nano-hierarchical structures  on the toe pads of many animals play key roles for generating strong but reversible adhesion for locomotion. The hierarchical structure possesses significantly reduced, effective elastic modulus ( According to Equation (1), the effE of the array of the “composite pillar” is much smaller than the array of the pillar with the flat tip and a diameter of D. The larger diameter D of the overhang can, on the one hand, further reduce effE, and on the other hand can form a larger contact area with the counterpart surface. Moreover, the mushroom-shaped tip can insert certain vacuum pressure during the pull-off, enlarging the adhesion forces . . Eeff ofe softer c. Increaagnitude .effE and therefore the adhesion properties of the corresponding structure [effE on the surface. Interestingly, this kind of effE pattern could even be sensed by living cells [Ppo) divided by the preload (PΔpo=0)) was reduced following the increase of pressure in the microchannels (Δp)  e.effE of the material, contributing to compliance enhancement. The self-similar hierarchical pillar has the highest similarity to the gecko setae; however, it suffers from the difficulties of manufacturing and low structure stability. The LPH structure has the best stability, and its adhesion could be further enhanced by optimizing the structural parameters. The porous structure for bioinspired adhesion is still at its early stage, and needs further development; it has the advantage of being able to mimic the adhesive secretion of some animals and aims at wet adhesion mimicking. The combination of structural elements may pave the way to reach or even surpass the abilities of biological structured adhesives. Hierarchical structures have a much higher possibility of forming reliable contacts on natural rough surfaces with various wavelengths and amplitudes, aiming at final applications in the real world. Inspired by the structured adhesives in nature, various artificial adhesives have been developed to gain controllable adhesions. Based on structural features, we summarized the artificial adhesives as belonging to three groups: self-similar hierarchical pillars, the lamella–pillar hybrid structure, and the porous structure. Although these structures differ greatly from each other, they can effectively reduce the


In order to restore traffic videos with different degrees of haziness in a real-time and adaptive manner, this paper presents an efficient traffic video dehazing method using adaptive dark channel prior and spatial-temporal correlations. This method uses a haziness flag to measure the degree of haziness in images based on dark channel prior. Then, it gets the adaptive initial transmission value by establishing the relationship between the image contrast and haziness flag. In addition, this method takes advantage of the spatial and temporal correlations among traffic videos to speed up the dehazing process and optimize the block structure of restored videos. Extensive experimental results show that the proposed method has superior haze removing and color balancing capabilities for the images with different degrees of haze, and it can restore the degraded videos in real time. Our method can restore the video with a resolution of 720 × 592 at about 57 frames per second, nearly four times faster than dark-channel-prior-based method and one time faster than image-contrast-enhanced method. Today, traffic video analysis plays a very important role in intelligent transportation systems. It has become a common way to help people track a vehicle, as well as locate and judge an accident. Because the images captured by outdoor cameras are often affected by different weather conditions, they suffer from poor visibility and lack of contrast. In the literature, there are many enhancements and dehazing algorithms that improve different images, such as traffic videos, underwater images, and satellite imagery ,2,3. TheIn order to restore traffic videos with different degrees of haziness in a real-time and adaptive manner, this paper presents an efficient traffic video dehazing method using adaptive dark channel prior and spatial-temporal correlations. This method can avoid overstretched contrast after haze removal and obtain satisfactory restored results for dense hazy videos by using a novel approach involving adaptive transmission estimation. This method also takes full advantage of the temporal and spatial correlations in traffic videos to meet the requirements of real-time dehazing, such as using time continuity to set the time slice, refining transmission by characteristics of block structure, decreasing restored area according to the lane space, and simplifying the calculation of parameters by using multi-camera distribution.Essentially, videos are composed of frames, thus the haze removal method for images can be used for videos. The image dehazing method is the most common way to restore hazy images. This method considers the inverse process of image degradation and describes the image degradation process in detail through an established physical model. The most critical step of this method is to obtain the parameters of the degradation model. Oakley et al.  improvedTo obtain the parameters of the degradation model effectively, some dehazing methods based on prior knowledge or assumptions were proposed, and they do not need to get reference images in advance or use an additional hardware device. Therefore, these methods have better adaptability than previous methods. Based on the assumption that a haze-free image has a higher contrast than a hazy image, Tan  proposedUsually, traffic video dehazing algorithms are proposed based on single-image dehazing algorithms. However, the computational complexity makes it difficult to apply single-image dehazing algorithms directly to video dehazing. Most existing research on video dehazing is to speed up the process of dehazing. Sun et al.  proposedThe characteristics of videos can be applied in specific video dehazing algorithms. Tarel et al.  proposedThe most common dehazing model is based on atmospheric optics , which cThe process of haze removal for every frame of a traffic video can be divided into three steps: calculating atmospheric light, estimating the transmission map, and restoring the image. In this paper, we present a novel adaptive method for transmission map estimation, thus the dehazing algorithm can be applied to images with different degrees of haze. The framework of the single-image dehazing algorithm is shown in Step 1:Divide an input image into four rectangular regions.Step 2:Define the score of each region as the average pixel value subtracted from the standard deviation of the pixel values within the region.Step 3:Select the region with the highest score and divide it further into four smaller regions.Step 4:Repeat Steps 1 through Step 3 until the size of the selected region is smaller than a prespecified threshold. The prespecified threshold in this paper is 200, which is that the height * width of the selected region is smaller than 200.We use a hierarchical searching method based on quad-tree subdivision  to find At last, we choose the color vector, which minimizes the distance In general, a hazy block yields low contrast, and the contrast of a restored block increases as the value of the estimated transmission decreases. We adopt the image-contrast-enhanced method  to maximMean squared error contrast (CMSE)  can defiAccording to the assumption that the scene depths are locally similar ,12,16, tIf Equation (3) is put into Equation (2), t value to reduce the information loss due to the truncation. Thus, we should not only enhance the contrast but also reduce the information loss.However, when a block contains dense haze, it has a relatively narrow value range for input pixels. Thus, even though it is assigned a small Therefore, we need to set quantitative evaluations for contrast and information integrity. The contrast cost d by Kim  to evaluIf we want to get a better restored image, the image contrast should be smoother, and the color information should be maintained as much as possible. Thus, these two factors should be taken into consideration synthetically, and the overall cost function is described as Equation (7).oss cost . The minT of an image. The haziness flag T is concerned with the effects of the degree of haze in images.We present a haziness flag el prior  can estiThe dark channel prior is based on the observation that most local blocks in haze-free outdoor images contain some pixels that have very low intensities in at least one color channel. In other words, the dark channel value of a haze-free image is close to zero . For anym filter .Assuming that the atmospheric light 1) by Ac :(9)Ic(p)Since the transmission quations .(10)minUsing the concept of a dark channel , if Jc iPutting Equation (11) into Equation 9), we can eliminate the multiplicative term and estimate the transmission , we can According to our experimental results, in a hazy image, the range of T, that is, The structural similarity (SSIM) index is a method for predicting the perceived quality of digital television and cinematic pictures, as well as other kinds of digital images and videos. To guarantee that the restored images are closer to ground truths, we adopted the SSIM index  to measuHowever, this method needs a haze-free image to get the optimal correction value T represents the degree of haze that degrades the image contrast. Thus, the image contrast and haziness flag value should be considered simultaneously. We set If the relatively reasonable correction value of initial transmission is Compared with static traffic images, traffic videos have some unique characteristics. First, a traffic video is a collection of images with time continuity. Second, the cameras are fixed on the road and capture videos of the same scene over a long time, thus the videos are consistent in space. Therefore, we can use the correlations of spatial-temporal information to speed up traffic video dehazing.Because the cameras are fixed, the scenes in traffic videos barely change over a long period of time, and the influence of haze is stable. In our experiments, we use the traffic videos from ZhongHe elevated freeways in Hangzhou City, set a cycle of five minutes, and regard the frames in one cycle as a collection of images with the same characteristics. After setting the 5 min cycle, we can take the first frame of a video segment as a reference frame. We can determine the image haziness flag value We estimate the optimal transmission based on the assumption that all pixels in a block have the same transmission. However, scene depths may vary spatially within a block, and the block-based transmission map usually has a blocking-artifact problem. Therefore, an edge-preserving filter is adopted to refine the block-based transmission map.The single-image dehazing method using dark channel prior  employs If the transmission is too small, the noise will be enhanced in the restored image . Thus, tAfter analyzing the spatial characteristics of traffic video, we found that the traffic lane is an obvious structure. In a traffic video detection system, the detected objects are mostly concentrated in the driveway regions. The areas outside lanes are not the regions of interest in traffic video processing. Therefore, we can process haze removal only in the driveway region of traffic video to reduce computing time.However, the estimations of atmospheric light and transmission are based on the whole image. If these values are achieved only through the driveway regions, it may cause some deviations, especially when the sky occupies a large area of the image, such as the cases shown in Calculate the global atmospheric light Step 1: Obtain the edge information in the video through edge detection.Step 2: Remove obviously wrong-angle lines by Hough linear fitting, and obtain lane candidates, as shown in Step 3: Find the far left lane and the far right lane, and set them as the driveway boundaries, then find the intersection of these two lines, as shown in Step 4: Identify a rectangular area as the driveway region, which is composed of the boundary of the image and a horizontal line across the intersection, as shown in Get the driveway region, as shown in Use the original pixel values and the optimal transmission of driveway region in the dehazing model to restore the image in the driveway region.We adopt a straight-line extraction algorithm based on the Hough transform to detect the lanes and separate the driveway region from the global image. The process of haze removal combined with the driveway region separation is described as follows:In a traffic video detection system, each camera is located at a fixed position and captures the same traffic scenes for a long time. Based on the time continuity, the result of lane space separation for the initial frame of a traffic video can be used over a long time period. Lane space separation can decrease the area of haze removal and improve the efficiency of the dehazing algorithm. With an increasingly complex layout of transportation networks, the number of traffic monitoring cameras also increases gradually, and sometimes there are multiple cameras in the same section of road. These cameras located in close physical proximity usually have the same hardware indicators. In a traffic video detection system, multiple cameras are connected to one system. These cameras have similar characteristics according to their spatial distribution. The weather is also an index with spatial characteristics, that is, the degrees of haze are similar in nearby regions. Thus, we can use the spatial distribution information of cameras to speed up dehazing and optimize the performance of the traffic video detection system.T calculated from each video are shown in the first column of In traffic video dehazing, the cameras are divided into different regions according to their locations, and one camera in a region is set as the calibration camera. The images from the calibration camera are used to calculate the initial transmission value, which is also applied to other cameras in the same region. Therefore, we can avoid repeatedly calculating the values of In the efficient traffic video dehazing method using adaptive dark channel prior and spatial-temporal correlations, a video sequence is converted into Our adaptive method can determine the initial transmission according to the image characteristics, thus it can produce a more satisfactory dehazing result than the method with fixed initial transmission. g method ,21, the To get better restored images, we restore the whole image for the first frame of a time slice and use the area outside the lane space of the restored frame to replace those areas of the following frames. Moreover, we adopt the parallel programming tools SIMD  and OpenAs we can see from the experiment results, our method produces better haze removal results by determining parameters according to image characteristics. It is also applicable to dense fog or a variety of fog densities. Moreover, it makes the restored images more similar to the real scene and avoids the problem that the restored images exhibit overstretched contrast. Therefore, it can solve the general problems in the existing dehazing algorithms—contrast distortion after video dehazing and failure to remove dense haze.In addition, our method adopts the spatial correlation, time continuity, lane separation, and spatial distribution of cameras to improve computational efficiency. Besides the processing time, the performance parameters of frames per second (fps) and SSIM of different methods for the video dehazing in Traditional haze removal methods fail to restore the images with different degrees of haziness in a real-time and adaptive manner under most circumstances. To solve this problem, we propose an efficient traffic video dehazing method using adaptive dark channel prior and spatial-temporal correlations. The dark channel prior is based on the statistics of outdoor haze-free images, but it cannot adaptively estimate the initial transmission value based on the degree of haze and contrast of images. Therefore, we adopt the image-contrast-enhanced method to obtain the best estimated transmission value as the initial transmission value of dark channel prior. The image dehazing method using adaptive dark channel prior can overcome the shortcomings of existing dehazing algorithms that overstretch contrast after haze removal and deal with images with dense haze to a satisfactory level. Additionally, we introduce the temporal-spatial correlation of traffic videos to speed up the traffic video dehazing using the time continuity to set a time slice, the characteristics of block structure to refine transmission, lane space structure to decrease the restored area, and multi-camera distribution to simplify the calculation of parameters. The experiment results show that our method can restore satisfactory image appearance, which can remove dense haze effectively and does not produce results with overstretched contrast. The temporal and spatial characteristics can reduce the computation time, especially for dehazing multiple videos.However, the dark channel prior is a kind of statistic, and it may not work for some particular traffic videos. When there are rapidly changing hazes in the videos, the dark channel of the scene radiance has a great difference at different times. In addition, if the scene objects are inherently similar to the atmospheric light and no shadow is cast on them, the adaptive dark channel prior is invalid. The dark channel of the scene radiance has bright values near such objects. As a result, our method may underestimate the transmission of these objects and overestimate the haze layer.