Previously introduced bundles of hundreds or thousands of microfibers have the potential to extend optical access to deep brain regions, sampling fluorescence activity throughout a three-dimensional volume. Each fiber has a small diameter ( Like single-photon imaging or multiunit electrophysiological recordings, it may not be possible to distinguish or limit recordings to a single neuron. But much as spike sorting enables separating individual neural activity, it is possible to incorporate knowledge of the target neural population and the sensitivity profile of the fibers to gain a more comprehensive understanding of the individual neural dynamics that will contribute to the recorded fluorescence signal.When using a low density of fibers to interface with a low-density cell population , the sensitivity profile of each fiber will be dominated by one or two neurons. Limited overlap between the sensitivity profiles of neighboring fibers will preclude source separation, as there will not be sufficient correlations between the fluorescence signals. Alternatively, when using a higher density of fibers, the sensitivity profiles will have greater overlap, creating the opportunity to reverse the linear mixing of nearby neuronal signals. Under such conditions, the recorded signal is amenable to source separation techniques to extract individual neural traces.In this paper, we model bundles of hundreds or thousands of optical microfibers distributed throughout a target brain region as an optical interface for collecting fluorescence activity. Our model allows us to evaluate the interface properties of the bundle and how variables like the distribution of fibers and neurons effects the interface performance. Our model results indicate that with a low density of fibers, each fiber will collect the optical equivalent of a local field potential, measuring bulk fluorescence from a small number of neurons in close proximity to the fiber tip; as the density of the fibers increases, overlap between the fiber sensitivity profiles will enable collecting sufficient information to separate individual neural activity.22.1,We calculate the fluorescence sensitivity profile for a single fiber inserted into the tissue using Monte Carlo simulations of 10,000,000 photon packets launched from the fiber as described in Refs. The sensitivity profile for a bundle of fibers is obtained by taking the linear superposition of the profile for a single fiber. Based on our prior histology documenting the distribution of bundles of hundreds to thousands of fibers,,By superimposing the sensitivity profile on the tip of each fiber in the bundle based on the splay distribution,2.2,Equipped with this model of the optical interface properties for a bundle of microfibers, we can now evaluate the ability to measure signals from a simulated neural population. We simulate a three-dimensional (3-D) volume of tissue, of sufficient size to ensure that it includes all regions of non-negligible fluorescence sensitivity .The fluorescence traces for each neuron are then combined via the mixing matrix described in the previous section to produce fluorescence traces for each fiber. The fluorescence collected by a fiber can be written 2.3Given the linear mixing process inherent in the fluorescence signal collected by each fiber, the data are well suited to blind source separation techniques to estimate the underlying neural fluorescence signals. To improve the performance of the source separation, we developed an approach that incorporates our knowledge of the mixing process and associated fluorescent indicator dynamics. Similar to existing techniques that use deconvolution to estimate precise spike timing,Next, we apply the non-negative-independent component analysis algorithmHaving separated the independent signals, we can then apply the waveform filter .As a control, we compare correlations between the separated signals with a set of randomly generated traces.2.5https://codeocean.com/2018/11/12/fiber-source-separation/code. The MATLAB modeling and source separation implementation is available on Github https://github.com/nathanntg/fiber-source-separation.The code to generate the fiber sensitivity profile and to run all described models are publicly available at 33.1The relevant experimental technique being modeled, including the optical interface, implant methods, and motivating histology, is fully described in Ref.  exposed .After cutting away a fraction of the fibers in the bundle (based on the desired implant count), the remaining fibers are anchored together 4 to 5 mm from the tip using a bead of light-cured acrylic . The bead is secured to a digital manipulator in a stereotaxic rig. An anesthetized animal is mounted in the rig, and a craniotomy and durotonomy are opened above the target brain region. The fibers can then be slowly lowered into the tissue; the fibers have enough rigidity to enter the tissue without bending. Histology from implants shows that each fiber follows a path of least resistance, spreading through the target brain region .The other end of the fiber (a ferrule and polished imaging surface) can be mounted below an objective in either a traditional fluorescence microscope or a purpose built optical configuration to provide excitation light and collect emission light via a CMOS sensor .3.2In 3.3Calculating the fluorescence sensitivity profiles for various distributions of optical microfibers, consistent with prior histology, we are able to model and evaluate the number and relative brightness of neurons contributing to the detected fluorescence signal. This relationship between the number of fibers and the number of neurons contributing fluorescence is explored further in These results demonstrate that the number of neurons contributing to the fluorescence signal increases more rapidly than the number of fibers. Under low densities of fibers, each fiber collects from a distinct set of neurons and the low-excitation power means that the signal will be limited to the 2 to 3 neurons closest to the fiber aperture. But as the fiber density increases, the paradigm shifts to one where many neurons are contributing fluorescence to multiple nearby fibers [, and higher fiber density results in increased overlap between the sensitivity profiles of neighboring fibers. With this shift, the fluorescent signal from a single cell is more likely to contribute to multiple fibers . These c–Blind source separation techniques are widely used in neuroscience for decoding of neural signals,,These existing blind source separation techniques can be applied directly, as the overlapping sensitivity profiles associated with each fiber act as a linear mixing of fluorescence activity from the surrounding tissue. Yet the physical distribution of the splayed optical microfibers and the mechanics of the fluorescent indicator provide useful constraints that can further inform the source separation process. Specifically, we adopt an approach that incorporates the dynamics of the fluorescent indicators and the non-negativity of the mixing process. First, we use an inverse filter to remove the fluorescence waveform. Our approach assumes consistent and known waveform dynamics, which enables the inverse filtering; should this assumption not apply, source separation techniques designed for convolutive mixtures become relevant and enable estimating both the mixing and convolution steps.Similar techniques and model constraints are frequently used in traditional calcium imaging, using non-negative matrix factorization and deconvolution to extract neural activity and estimate underlying spiking.In exploring source separation techniques, we also looked at what factors impact the source separation performance. Unsurprisingly, increased recording duration improves the performance. More relevantly though, source separation performance is best with sparse signals . Such sparse signals provide discrete, differentiable events that are conducive to estimating the underlying mixing process.To evaluate the performance of the source separation for a realistic analysis task, we compared spike detection analysis on the separated signals and on the true, underlying neural traces. Separated signals are first matched with the underlying neural traces based on correlations; the majority of the separated signals have a strong correlation (Further work can look at incorporating additional information into the source separation process. For example, it may be possible to estimate the initial mixing matrix through empirical measurement of correlations between fibers. Specifically, by shining light down a single fiber and looking at light collected by the other fibers, it may be possible to estimate the relative spatial configuration of the fibers and, as a result, the correlations that we would expect in the linear mixing process.More broadly, our work here has laid a modeling framework for evaluating high-channel count fiber photometry as a means of interfacing with deep brain regions. By estimating the overlap of neighboring sensitivity profiles and hypothetical neural population dynamics, we can describe both the likely composition of the signals collected by individual fibers and the larger mixing process that occurs. One limitation of our study, although likely minor, is that we simulated neural signals as arising from point sources. Further work is needed to explore the full impact of the fluorescent signals arising from large and complex neurons, which may alter both the mixing process and the feasibility of source separation.


The acquisition of translucent objects has become a very common task thanks to the progress of 3D scanning technology. Since the characteristic soft appearance of translucent objects is due to subsurface scattering, the details are naturally left out in this appearance. For objects that have complex shapes, this lack of detail is obviously more prominent. In this paper, we propose a method to preserve the details of surface geometry by adding highlight effects. In generating highlight effects, our method employs a local orthonormal frame and combines, in a novel way, the incoming and outgoing light in approximating the subsurface scattering process. When the incident illuminant direction changes from nearly overhead to nearly horizontal, our method effectively preserves complex surface geometry details in the appearance of translucent materials. Through experiments, we show that our method can store surface features as well as maintain the translucency of the original materials and even enhance the perception of translucency. By numerically comparing the generated highlight effects with those generated by the traditional Bidirectional Reflectance Distribution Function (BRDF) models with different bandwidth parameters, we demonstrate the validity of our proposed method.  Advances in 3D scanning technology have been widely applied to object acquisition due to the abundance of low-cost RGB-D sensors. Translucent objects are very common in realistic domestic environments, and by placing a scanner at strategically selected positions, geometric details can be captured with thoroughness and high fidelity. However, in rendering the complex scanned data of objects made from translucent materials, blurred geometric details become more pronounced as the illuminant direction approaches a horizontal orientation. According to previous works ,2, highlCurrently, many specular highlight component expressions of the BRDF  model employ a local orthonormal frame to calculate the highlight value. The tangential vectors of the local orthonormal frame are important for the shape of the generated highlight. Many methods used tangent vectors to simulate highlight and produce special visual effects.Various approaches ,5,6 usedThe visual appearance of specified materials and the dimensions of the physical structure are related to the chosen local orthonormal frame. Brandt and Scandolo  used spehsbacher  made an These models can describe reflectance on a surface or scattering inside a medium by analyzing specular reflection on specific structure models such as a cylinder, a curved cylinder, or voxels stored in a lattice. However, they are not well suited to surfaces that have complex geometry details. Methods to add highlight effects on translucent materials have seldom been reported in the literature due to light scattering away inside a medium. Fleming and Bülthoff  used fulSome computer vision methods focus on extracting specular components from captured images. The main idea is to separate the global illumination effect and the directional reflection component to obtain the meso-structure of translucent objects and detailed surface normal vectors.Chen et al.  combinedThe drawback of these methods is that they are essentially BRDF-based highlight-generation methods, in addition, they cannot completely separate the subsurface scattering component and the direct-illumination component, and the residual direct component may be too low and noisy compared with the translucent surfaces.Kim and Ghosh  successfThe method of Sun et al.  is basedPhotometric stereo can also be used to recover surface details for translucent materials ,20. HoweOur method simulates the highlight effects on the approximation of subsurface scattering in rendering translucent materials. It can generate highlight effects in the appropriate position for normally shaped surfaces. Moreover, the subtle geometry details are preserved while the translucency is well maintained in the rendering results, and the method can also improve the translucency effects in edge regions.The approach of rendering translucent materials using subsurface scattering is based on the radiative transfer equation. Several BSSRDF  models are available to estimate subsurface scattering, including the standard dipole model , the quaFor generating highlight effects, we employ the specular component expression of the Ward , AshikhmOur method is inspired by the concept of using a minimum enclosing cylinder to model a fiber for approximating outgoing radiance at a surface patch from incoming irradiance at another patch by scattering effects . We emplThe geometry of light transport is shown in We define the highlight effect that one would observe from Our method consists of four steps: (1) defining the local orthonormal frame, (2) choosing the illumination directions, (3) selecting the highlight component expression, and (4) generating highlight effects by a Monte Carlo method for rendering translucent materials. We define half-way vector We employ the local orthonormal frame The definition of In the calculation, We generate highlight effects from different incident illumination directions. When we use the directional dipole model to approximate the diffusive part of the BSSRDF, the calculation process assumes that the diffusive light at the exitant point no longer depends on outgoing direction Based on the microfacet model, the specular highlight component For visualizing different appearance effects, we employ the specular highlight component of the Ward, Ashikhmin, and Lafortune models to generate the highlight effects. The normalized tangent vectors For the Ashikhmin model, we use the highlight expression withFor the Lafortune model, we need Since the subsurface scattering decreases exponentially with distance In our rendering of translucent materials, the BSSRDF is implemented in a Monte Carlo ray tracer. We fix The proper choice of the illuminant direction is important for preserving details when adding highlight effects. We set the upper hemisphere with its origin at the surface incident point and its north pole as vector nd Elder , as the To compare the results of the generated highlight effects by our model and those by traditional models, we used parameters k is 10. We used Stanford bunny object with two materials which are from Jensen et al.  and Nara methods , the expWhen the illumination direction changes from nearly overhead to nearly horizontal , the sofIn The results in As shown in Qi et al.  investigFor methods based on the Ward and Ashikhmin models, we chose the left edge of the bunny leg’s for numerical analysis (red closeups in For chocolate milk (regular) material, the results of the generated highlight show similar effects as those for the marble material. From the “shadow-haze” view described in the work of Fleming and Bülthoff , our metFor methods based on the Lafortune model, we chose the bunny’s leg region for numerical analysis (purple closeups in We select a vertical line of pixels that is in the middle of the red closeup region  to studyWe presented a method to preserve surface details by adding highlight effects in the rendering of translucent materials. The method employs the light paths of the incident and exitant points in the light-scattering process, uses the local orthonormal frame in the incident point, combines three highlight-generation models that employ local tangent vectors, generates highlights in appropriate positions, stores surface details under different conditions while maintaining the original material’s translucency, and enhances the translucency in edge regions. All these features help to prove the validity of our method.We compared the results of the proposed method and traditional models in three highlight expressions. Generally, the proposed method exhibits rich visual effects in regions with complex surface details and geometry features, and it increases the perception of translucency in those regions. Under some conditions, the number and average intensity of the generated highlight effects by the proposed method are smaller than the results generated by the traditional models, i.e., lower values, and the region’s details are less clear than those for higher values. However, the proposed method has the advantage of avoiding excessively bright or metal-like highly specular appearances in the rendering results.Our method works with a single-direction light, and it can be incorporated in applications that need a fast solution. However, due to the restrictions on incident light directions, in some positions, where the angle between the incident light and the normal vector is above 90°, the highlight value is clamped, so it cannot produce highlight effects in these positions. In the future, we plan to study the relation of the generated highlight position and the local orthonormal frame. Consequently, by applying proper tangent vectors, it would be possible to produce specific highlight effects that are in accordance with a specific geometric shape.


This paper presents a concept of an intelligent sensing technique based on modulating the frequency responses of microwave near-field sensors to characterize material parameters. The concept is based on the assumption that the physical parameters being extracted such as fluid concentration are constant over the range of frequency of the sensor. The modulation of the frequency response is based on the interactions between the material under test and multiple sensors. The concept is based on observing the responses of the sensors over a frequency wideband as vectors of many dimensions. The dimensions are then considered as the features for a neural network. With small datasets, the neural networks can produce highly accurate and generalized models. The concept is demonstrated by designing a microwave sensing system based on a two-port microstrip line exciting three-identical planar resonators. For experimental validation, the sensor is used to detect the concentration of a fluid material composed of two pure fluids. Very high accuracy is achieved. Modern technologies such as lap-on-a-chip involve fluidic and microfluidic analysis that require highly accurate sensing systems. These technologies can be implemented in many applications including environmental monitoring, agriculture, aquaculture, biotechnology, food production, and public health and safety . The senIn electrically-small resonators, the physical dimensions of resonators are electrically small compared to the operating wavelength where the electromagnetic (EM) fields are enhanced in small areas . By distonly on a single or multiple resonance frequencies, the utilization of the resonators in the sensing systems are still at the preliminary stage. This is due to the limitations and difficulties in constructing either analytical models or models based on numerical results at multiple frequencies.Planar resonators are unbounded structures, which makes their analytical models difficult to develop for characterizing the MUT. However, as long as the sensing mechanism is based on measuring the changes in the resonance frequencies, full-wave numerical simulations can be utilized to model the responses of the resonators as functions of the properties of the MUT . By using fitting function techniques, the models can then be constructed ,7,8. NevIn this paper, we introduce a concept for designing microwave intelligent sensing (MIS) systems based on modulating the frequency response of multiple electrically-small resonators with the MUT. The sensing mechanism of the systems is based on measuring the changes of the transmission coefficients of microstrip lines. When multiple resonators are used, coupling to the resonators will increase the bandwidth of the bandgap. Thus, if one of the resonators is loaded with the material under test whereas the other resonators are loaded with well reference materials, the coupling between the resonators will be affected and that will add more information about the MUT to the response, which is the main idea of this work. The concept is based on the assumption that the targeted physical parameters such as fluid concentration are constant over the range of frequency of the sensor. The modulation is based on the interactions between the sensors and the MUT. We show that using many points at different frequencies can reduce the need for a large number of MUT samples to achieve higher accurate neural networks. For validation, an MIS is developed based on utilizing three-identical resonators. The concept was tested using numerical simulation and verified experimentally to characterize the concentration of a fluid mixture.In practical sensing applications, the accuracy of the extracted results can be degraded due to many sources of errors including air gaps between the sensor and the MUT and the background noise. Thus, for characterizing a single physical parameter, such as fluid concentrations or the permittivity or permeability, using the information provided by a single frequency point or the shifts in the resonance frequencies are not sufficient to meet the need for highly reliable sensing systems. If the sensing systems are going to be utilized for characterizing many physical parameters simultaneously, the limitations of models  increase. Of course, the sensing mechanism based on measuring only the shifts in the resonance frequencies or the reflection coefficients at a single frequency will make the models’ construction process easier; however, these models will limit the sensitivity due to measurements’ errors or the background noise.To overcome these limitations, the sensing systems can be designed by utilizing electrically-small resonators where the response over many frequencies can be observed. Microwave planar electrically-small resonators that are based on two-port microstrip line systems terminated by load impedances have an important feature compared to one-port resonant systems such as open-ended coaxial resonator probes and evanescent microwave probes ,10. In aElectrically-small resonators such as complementary split-ring resonators (CSRRs) show higher sensitivity to detect the presence of the dielectric materials compared to split-ring resonators (SRRs) . Figure The concept is demonstrated by characterizing the concentration of liquids. For instance, suppose that the MUT .To study the effect of different liquid concentrations, the resonators A and C are loaded by dielectric materials with relative permittivity of 4.806 and 2.015, respectively, while resonator B is loaded by a dielectric material with its permittivity varying from 2.015 to 4.806 with an increment of 0.01. Thus, the response to pure cyclohexane and chloroform will determine the upper and lower frequency limits, whereas the response of the mixture will be presented as a notch in the SYS HFSS  as shownThe response of the MIS system was analyzed and compared to a single CSRR based systems. Both systems were tested to detect materials with dielectric constants of 2.515 and 2.715. The responses of the single-CSRR and three-CSRR sensors are shown in d P4 see b, were cThe MIS was fabricated using printed circuit board (PCB) technology. Three aluminum cavities were added to the resonators to hold liquids. The sensing system is shown in Twenty-one samples were prepared with different concentrations from 0% to 100% with an increment of 5%. The response of the MIS at each concentration is observed as a vector, To find the relationship between the input and the output (the liquid concentration in our case), the multi-layer perceptron (MLP) neural networks with supervisor learning algorithms can be utilized. There are many backpropagation learning algorithms such as Bayesian regulation backpropagation, Levenberg–Marquardt backpropagation, scaled conjugate gradient backpropagation, and resilient backpropagation. Since the datasets are small and limited to only 21 samples, the Bayesian regulation backpropagation algorithm is the best choice to use for the learning process of the neural networks . The BayThe BRANN was tested with different hidden layers since higher number of hidden layers are not necessary to produce higher accuracy. Four and five hidden layers were found to be sufficient for our datasets. In the BRANN, the datasets are divided into training and test (unseen) samples. In particular, the datasets were classified as 1—70/100% training 30/100 test (14 samples for training and six samples for the test) 2—85/100% training 15/100 test (18 samples for training and three samples for the test) 3—90/100% training 20/100 test (19 samples for training and two samples for the test). In the supervisor learning algorithm, the choice of particular samples for training or testing is random to test the performance of the BRANN. Since the datasets are small, we found that training the BRANN with 19 samples can result in an accuracy of approximately 99.9%. The summary of the training and the test samples with different hidden layers and classification of the datasets and the performance in the mean square error (MSE) is presented in This paper introduced a microwave intelligent sensing systems. The system is based on modulating and demodulating the frequency response of multiple closely distanced sensors over a wide frequency range. Under the assumption that the physical parameters to be characterized such as fluid concentration are constant over the frequency band of the sensor, the responses characterized as vectors of multiple dimensions of the transmission coefficients are then utilized for constructing and testing 205 models that were built using neural network techniques. Furthermore, to increase the sensitivity of the system over a larger frequency band, multiple identical electrically-small resonators are used. In this work, three identical CSRRs were used. The system was tested to detect the concentration of fluids. The sensing part of the system was fabricated using PCB technology, and the results were recorded using a vector network analyzer. Then, the results were used to construct models utilizing the Bayesian regulation backpropagation artificial neural network (BRANN). The models were used to characterize the fluids. Although the datasets were relatively small, the utilization of the vectors of many components as inputs provided sufficient information to the BRANN to produce models with an accuracy ranging from 82.8% to 99.97%. An accuracy level of 99.97% was achieved as a result of using 90% of the data to train the BRANN, which shows that larger datasets provide higher accuracy.


In recent years, 3D trans-oesophageal echocardiography (TOE) has become widely used for monitoring cardiac interventions. The control of the TOE probe during the procedure is a manual task which is tedious and harmful for the operator when exposed to radiation. To improve this technique, an add-on robotic system has been developed for holding and manipulating a commercial TOE probe. This paper focuses on the probe adjustment strategy in order to accurately monitor the moving intra-operative catheters. The positioning strategy is divided into an initialization step based on a pre-planning method, and a localized adjustment step based on the robotic differential kinematics. A series of experiments was performed to evaluate the initialization and the localized adjustment steps. The results indicate a mean error less than 10 mm from the phantom experiments for the initialization step, and a median error less than 1.5 mm from the computer-based simulation experiments for the localized adjustment step. Compared to the much bigger image volume, it is concluded that the proposed methods are feasible for this application. Future work will focus on evaluating the method in a more realistic TOE scanning scenario. Catheter-based procedures for minimally invasive surgeries have been widely used in the last two decades as a replacement for open surgery for treating various types of heart disease. In these procedures, miniatured catheters , serving a broad range of functions for different treatment purposes, are slowly moved into the heart. The catheter is inserted into a large vein through a small incision, made usually in the groin area, and then is advanced into the heart. To guide the catheter’s operation within the heart, X-ray fluoroscopic imaging and 2D trans-oesophageal echocardiography (TOE) are typically the modalities of choice. Working collaboratively, fluoroscopy provides high-contrast images of the devices, while 2D TOE provides more detailed information on the soft tissues. With the development in recent years of real-time 3D ultrasound using matrix-array techniques, guiding of catheter-based procedures can be performed using a 3D TOE probe , and thiDespite the advancement of the ultrasound imaging techniques, 3D TOE still remains as an operator-based manual approach. With the operator standing next to the X-ray system and manipulating the TOE probe, it is tedious and harmful for the duration of the longer interventional procedures . Heavy pIn this paper, a new concept using the designed TOE robot to automatically monitor intra-operative catheters and place the catheter in the center of the FOV has been presented. To our knowledge, the device we designed was the first published system for robotically manipulating a TOE probe, and the automatic catheter monitoring with TOE is a unique topic which has not been reported before. Detailed methods employed in this paper are related to research work for other types of endoscopes in different applications, such as described in ,14. In aThe developed add-on TOE robot a holds tWith microcontrollers and Bluetooth modules built inside the handle actuating chamber, the TOE robot can be remotely controlled by the user via Bluetooth communication from a PC. More details of the design, along with the method of operation and safety concerns, can be found in our previous work . The whoFor the use of the proposed TOE robot, an exemplary embodiment is shown in The pre-planning method used for the initialization step intends to locate the global probe position during a defined procedure. In a view-planning platform developed for 3D TOE a, an autF(p), where p is the robotic parameter set, gives the transformation between the patient coordinates and the predicted probe tip coordinates:Using gradient-descent search, an objective function based on the X- and Y-axial distances from the origin of the ultrasound image coordinates to the target can be defined to optimize the robotic parameters based on the forward kinematics . The forPatienttTarget. The position of the target in the ultrasound image coordinates can be calculated using a known calibration obtained from T. The velocity skew of the probe tip ProbeTipV = T, expressed in the probe tip coordinates, is determined by the robotic Jacobian RJ, where ProbeTipV = For the differential kinematics, the robotic joint space parameters are defined as ω can be extracted from the skew-symmetry matrix, which is obtained from the multiplication of the partial derivative of ProbeBaseRProbeTip and the transpose of ProbeBaseRProbeTip:The Jacobian for RJ = T. The image servoing control for adjusting the probe bi-directional bending pointing to the catheter can be further determined by the image Jacobian, which relates differential changes in the image features to differential changes in the configuration of the transducer, i.e., the velocity skew ProbeTipV.The resulting robotic Jacobian is expressed as USp = T be the tracked catheter tip position in the 3D ultrasound image space. The coordinates of the catheter tip in the probe tip coordinates ProbeTipp are decided by the fixed calibration ProbeTipp = ProbeTipTUSUSp. In this study, the catheter tip position is assumed to be tracked either by image-based processing methods for USp or sensor-based tracking methods for both the TOE probe and the catheter, giving ProbeTipp as the result. The following relation describes the coordinates of the catheter tip in the probe base coordinate frame ProbeBasep:Let S(ProbeTipp) is the skew-symmetry matrix of ProbeTipp. The final Jacobian J relating the changes of robotic joint parameters to the changes of the catheter tip position in the probe tip coordinates is therefore obtained. With the differential kinematics, we have considered a simple controller as shown in ProbeTipP* is the ideal location of the catheter in the probe tip coordinates. This can be determined in the same way as described in the previous section so that the X- and Y-components for the USP* are zero, while the Z-component remains at the same value for its current position in the ultrasound image coordinates. The corresponding desired position ProbeTipP* can then be calculated based on the fixed calibration ProbeTipTUS and used for the bi-directional position controller. A simple geometrical relationship then relates the resulting R = T to the desired motor parameters for the bi-directional bending axes.As the probe shaft is not expected to move during the localized adjustment, the catheter tip is considered not moving in the probe base coordinates for a given catheter tip position when bi-directional bending is applied (oesophagus movements are ignored). The rest of the components in (7) will all change when the bi-directional bending is applied. Taking the time derivative of (7), the following relationship is obtained according to the visual servoing method summarized in :(8)ProbeTo verify the correct working of the initialization step, a computer-based simulation experiment focusing on the search strategy itself has been performed. Five hundred random points were defined within a heart–oesophagus segment obtained clinically to simulate the targets defined by the user. The probe positioning method described in To simulate the initialization procedure in a more realistic setup as a proof of concept, an experiment using a custom-made heart–oesophagus phantom was performed . The phaUSp = T in the ultrasound image space. The Z-component USz was assigned within the depth of 5–10 cm, where most structures in a TOE scan are located. For the X- and Y-components indicating the in-plane movement range of the catheter, uniformly distributed simulated positions in the range from −5 cm to +5 cm were assigned to cover the size of a normal heart. Compared to ProbeBasepProbeTip. The experiment is summarized in To verify the correct working of the localized adjustment step, a computer-based simulation experiment was performed by simulating the catheter’s position vector To further simulate the initialization step and the localized adjustment step in a more realistic clinical scenario, selected cardiac interventional procedures using 3D TOE as the monitoring device were simulated with catheter tip positions defined  based on the clinically obtained heart–oesophagus segment. Examples include atrial septal defect repair, left atrial ablation, percutaneous mitral valve repair or replacement, and trans-catheter aortic valve replacement (TAVR). The virtual catheter tip positions were defined in the original MR image of the patient based on the anatomies of the procedure. These points were then imported to the simulation platform to represent the targets to be monitored by the TOE image. Based on the simulated positioning data of the catheter tip, similar experiments as described for the randomly generated targets were re-performed, and the resulting positioning data were recorded. As the heart phantom used in For the simulated initialization method, the error function is defined as the root sum square of the X- and Y-components extracted from the matrix. The Z-component along the penetration depth of the ultrasound beam is not included as it usually has a minimum influence on the visualization. For the simulated experiments with randomly generated target position, the mean error was found to be less than 0.5 mm. To assess the performance of the acquisition for the phantom experiments, the acquired ultrasound images were registered to the pre-scanned MR images based on the method described in our previous work . This caFor the localized adjustment method, the X- and Y-components extracted from the recorded catheter tip positions in the ultrasound image coordinates were compared with the desired values, in which case, both distances are expected to be zero. These are treated as the absolute errors. Since the pyramid FOV of 3D TOE extends with increasing ultrasound penetration depths, centralization of the catheter tip positioning in the X–Y planes at different penetration depths were analyzed, taking the size of the X–Y planes into consideration. This was done by calculating the ratio of the X- and Y-components to the half-length of the X–Y plane, measured along the X- and Y-axis of the live 3D TOE image space. These two ratios are defined as the normalized errors.The median absolute and normalized errors of the X- and Y-components for both the randomly generated catheter positions and the defined clinical procedures were calculated, and are summarized in For the above analyses, the results from the simulation of the initialization step indicate the correct working of the method on its own, with a submillimetre probe positioning error identified. Though the simulation experiments have verified the accurate working of the proposed initialization method, differences between the simulation environments and the real scanning scenarios, as well as the involvements of the operator, could potentially influence the performance of the method. This can be observed from the experimental results of the phantom experiment, where the pre-planning environment is different to the real scanning setup in terms of physical constraints. For this initialization step, though the pre-planning method can accurately find an ideal probe pose, guiding the probe to the exact location without real-time tracking of the TOE probe would be difficult. The issues for probe tracking have been discussed in detail in our previous work ,20 whereThe results from the simulation of the localized adjustment process indicate that all absolute errors are less than 10 mm, and the majority of errors are less than 1 mm for both of the randomly generated positions and simulated clinical procedures. The errors are further interpreted by comparing to the size of the plane perpendicular to the ultrasound penetration direction in the live 3D TOE FOV. The normalized positioning errors indicate that all catheter tip positions are located in the center area of the FOV, when compared with the size of the live 3D volume. These results demonstrate the errors of the method on its own would have little influence on the operator’s visualization when observing the 3D image and the initial goal to automatically keep the catheter at the center of the FOV is achievable using the method, assuming the catheter can be tracked in the image coordinates. Comparing the results from the randomly generated positions to the simulated clinical procedures, lower errors have been identified for the simulated clinical procedures as the more clinically realistic approach in defining the catheter tip position resulted in a smaller required monitoring range of the TOE probe. A similar trend has been identified when comparing the three procedures within the clinical simulation experiment: a smaller monitoring range, e.g., the atrial septum repair procedure, results in smaller errors. This is because the probe’s bi-directional bending axes would have a higher possibility of success when used to cover a smaller required FOV. Therefore, the combined use of the initialization process and localized adjustment process is also important for accurate monitoring. Comparing The tracking of the catheter, either based on an image processing technique or an additional sensor, is assumed to be known in the simulation experiments. In reality, intra-operative instruments are difficult to visualize using ultrasound, due to image artefacts ,22. HoweIt should also be noted that the initialization method and the simulated TOE procedures described in this study focused on trans-oesophageal views, such as upper-oesophageal views and mid-oesophageal views. The remaining trans-gastric views and deep trans-gastric views were not included, as these views were outside the segmented region of the current heart–oesophagus model obtained from the clinical image, as well as not available from the custom-made heart–oesophagus phantom. However, the current method can be adapted to work with the trans-gastric views without too much additional work. For the initialization step, to decide a global location of the probe, the current pre-planning strategy within the oesophagus using the gradient descent search method will be extended to the stomach. In the new scheme, the constraint of the probe’s location will not be limited to the oesophagus center line, but extended to the stomach with a virtual moving trajectory line of the probe generated from the geometries of the oesophagus and the stomach. For the localized adjustments using the proposed differential kinematics, the method will remain the same as the modelled localized space, since the derivation of the differential kinematics has no difference between trans-oesophageal and trans-gastric views.This paper has described a strategy for controlling a recently developed TOE robotic system to adjust the pose of the probe, and therefore, automatically monitor intra-operative catheters used in cardiac interventions. The method has been divided into two stages with an approximate pose of the probe determined using a pre-planning method and an accurate pose of the probe adjusted based on a Jacobian position control loop. Accordingly, all of the four DOFs are employed for the once-only initialization, when human assistance is required for the probe insertion, and only two bending DOFs are utilized for the localized adjustments, when the robot could be controlled automatically. A series of experiments were performed, and the proposed method has been validated for its feasibility for use in this application. Limited by the experimental setup to simulate the complete workflow, our future works will focus on developing a new ultrasound imaging phantom to simulate the real trans-oesophageal approach, allowing both the TOE probe and a catheter to work inside the phantom in order to further test the proposed method.


Inertial measurement units (IMUs) are typically classified as per the performance of the gyroscopes within each system. Consequently, it is critical for a system to have a low bias instability to have better performance. Nonetheless, there is no IMU available commercially that does not actually suffer from bias-instability, even for the navigation grade IMUs. This paper introduces the proposition of a novel fluid-based gyroscope, which is referred to hereafter as a particle imaging velocimetry gyroscope (PIVG). The main advantages of the PIVG include being nearly drift-free, a high signal-to-noise ratio (SNR) in comparison to commercially available high-end gyroscopes, and its low cost. Navigation is a process by which a moving platform determines its state of motion through what is termed as the navigation states. The navigation states for any given platform are the position, velocity, and attitude of such a platform. The navigation states should be able to characterize the motion of any given body in either a two-dimensional (2D) or three-dimensional (3D) space .Navigation is performed through various methods; such methods are characterized mainly into two clusters: position fixing and dead reckoning. Position fixing is the process by which the navigation states are driven through the knowledge and use of a set of fixated well-defined positions. An example of position fixing navigation is the global navigation satellite system (GNSS). Whereas, dead reckoning is a process by which the navigation states are determined through a recursive process of relative positioning with respect to the initial states. An example of dead reckoning navigation is the inertial navigation system (INS) . Inertial navigation is performed by processing the data of inertial measurement units (IMUs). IMUs are an assembly of sensors that are mounted in a geometric form that guarantees capturing the motion of any given platform. Such sensors measure what is referred to as inertial measurements and they include accelerometers, which measure specific forces , and gyroscopes, which measure angular rates. The inertial measurements undergo a mathematical process, known as INS mechanization, that mainly includes integrations and geometric transformations to determine the navigation states of the moving platform with respect to a pre-defined reference frame. A typical IMU constitutes a triad of accelerometers and a triad of gyroscopes, placed along three mutually orthogonal axes.Inertial sensors endure systematic errors which can be calibrated in the laboratory and eliminated from the raw inertial measurements. Such systematic errors include biases, scale factor, scale factor non-linearity, and cross-coupling. The bias is a constant error, whereby the inertial sensor reads out a measurement that is at a constant offset from the actual value. The scale factor is an error that is demonstrated by the existence of a gradient for the input–output relation of the inertial sensor which is not unity. However, if such a gradient is non-linear, then it is said that the inertial sensor suffers from scale factor non-linearity. Cross-coupling error indicates that a given inertial sensor  measures residual inertial measurements from an axis that is orthogonal to its sensitive axes. Typically, cross-coupling occurs due to the non-orthogonality of the sensitive axis of the inertial sensors .Furthermore, inertial sensors suffer from random errors that appear as noises within the inertial measurements from electrical and mechanical sources. The noises are assumed to be white for frequencies below 1 Hz. The order of magnitude of such noises depends on the technology behind the manufacturing of the inertial sensor.Nonetheless, the performance of any given IMU is defined by the order of magnitude of the systematic and random errors. Whereby, IMUs are classified according their performance and accuracy into various categories.Consequently, IMUs are classified into strategic grade, navigation grade, tactical grade, automotive grade, and low-cost consumer grade IMUs. Strategic grade, also referred to as marine grade, is the highest quality IMUs. Strategic grade IMUs can be used in military ships, submarines, some intercontinental ballistic missiles, and some spacecraft. A strategic grade IMU might cost more than $1 million. Navigation grade IMUs are used in military aircrafts and commercial aviation. Navigation grade IMUs cost around $100,000. Tactical grade IMUs can only be used for stand-alone inertial navigation for a few minutes. However, an accurate navigation solution can be obtained by integration of the tactical grade IMU with an aiding positioning system. Typically, tactical grade IMUs are used in guided weapons and unmanned air vehicles (UAVs). Tactical grade IMUs cost around $2000 to $50,000. The automotive and consumer grade IMUs are the lowest grade of inertial sensors. They are typically used in pedometers, antilock braking, active suspension, and airbags. Consumer grade accelerometers costs start around $1, and the gyroscopes costs start around $10 [Nevertheless, IMU grades are analogous to the technologies adopted to manufacture their inertial sensors . Typical commercialized gyroscopes are classified into optical gyroscopes or vibratory gyroscopes. Optical gyroscopes utilize the phase of light waves to determine angular rates. The optical gyroscopes are realized by either ring laser gyroscope (RLG) technology or interferometric fiber-optic gyro (IFOG) technology . VibratoA main characteristic for the inertial navigation system is the bias instability, regardless of the grade of said INS. In other words, the navigation solution that is acquired from inertial navigation systems tends to drift from the actual occupied trajectory due to variations in the bias offset value, which is referred to as bias instability. However, such bias instability is less the higher the grade of the IMU, but it is inevitable. The bias instability tends to be a stochastic error that cannot be evaluated directly using a mathematical model, which makes handling its effects a sophisticated aspect of research in inertial navigation.There are a lot of reported sensors that operate with utilizing fluid as the inertial proof mass of said sensors. Other sensors use fluids as a part of the inertial sensing mechanism. This section provides a brief overview on reported inertial angular motion sensors that use fluids within their process of measuring angular rates.A type of inertial sensors that utilizes fluids as a part of its inertial sensing mechanism is the rate-integrating gyroscope (RIG). The rate-integrating gyroscope originally occurred in the 1960s. The rate-integrating gyroscope relies on the conservation of momentum and the trade-off between spatial rigidity and precession of a high-rate spinning rotor in 3-D space. One embodiment of the rate-integrating gyroscopes includes a high-rate spinning rotating body, with sufficient moment of inertia about its spin axis, should maintain a fixed spin axis in 3D space, unless acted upon by an external torque, which would lead the rotor to precess, such that the spin axis would follow the direction of the applied torque axis . Many UnAnother type of fluid-based angular rate sensor is the magnetohydrodynamic (MHD) angular rate sensor. MHD angular rate sensors deploy the magnetohydrodynamic properties of fluid to measure the angular rate . The magDual axis rate transducer (DART) is another form of a sensor that uses fluid as a part of its inertial sensing mechanism. This type of angular rate sensors first occurred in the 1960s in the United States. The sensor operation concept utilizes the law of conservation of momentum for a fluid body subjected to external inertial forces. The DART deploys a sphere of heavy liquid, such as mercury, as the sensing inertial element. The fluid sphere is contained within a spherical cavity. The spherical cavity is rotated via a driver motor at high speed such that the fluid sphere maintains a high angular momentum. Deflectable paddles are fixed at the periphery of the spherical cavity and passing through the fluid sphere. The paddles are attached to piezoelectric crystals. The paddles are used as a motion transduction mechanism. From the nomenclature of the sensor, it is noted that the device is sensitive to rotation about the two orthogonal axes, which are normal to the spin axis of the spherical cavity .Jet flow gyroscopes are a type of fluid-based inertial angular rate sensors. The development of these type of sensors occurred in the 1960s. The scientific concept behind fluid flow rate sensors is utilizing the fluid to be the inertial sensing element, instead of using the fluid to drive or support a mechanical element. The sensors deploy the concept of temperature or pressure variations on a fluid control volume under the impact of external motion imparted to the control volume. These temperature and pressure variations are modeled by what is known as the Navier–Stokes equations, which describe the law of conservation of momentum for a fluid mass. Nonetheless, mathematical evaluation of the Navier–Stokes equations is not a straightforward problem. Hence, the detection of the external angular rate imparted to these kinds of sensors is dependent on the proportionality between the external motion and the temperature or pressure variations that occur within the sensor internal structure. The popular embodiments of this type of sensors can be found under the title “flueric sensors” in .Vortex rate gyroscopes use a concept near to that of jet flow gyroscopes. However, a vortex rate sensor contains a container in which a 2D sink flow is generated. The generated sink flow comprises a vortex whose pattern and streamlines are well-defined, by design, in the absence of an input angular rate. However, when an angular rate is applied to the sensor an additional vortex flow is superimposed to the initial sink flow, which leads to a combined vortex flow. The variation in the fluid streamlines indicates the magnitude and direction of the input rate. There are different designs that can achieve an operating sensor utilizing the vortex rate concept , and the continuity equations for viscous laminar Newtonian fluid flows. The typical internal structure of these kinds of sensors includes an annular channel where fluid flows. The fluid flows under the impact of external angular motion. The annular channel is built such that it contains an obstruction or float. The obstruction/float are moved under the impact of fluid flow. The motion of the obstruction/float is an indication of external angular acceleration imparted to the sensor. The motion of such obstruction/float is sensed using bridge circuits or other equivalent pick-off mechanisms. However, the angular acceleration values are integrated by deploying a signal conditioning mechanism to measure an integrated angular acceleration sensor. In other words, the sensor measures the applied angular rate. There were numerous attempts that used the same concept with different embodiments and modifications to achieve angular rate sensors. Among these attempts are a series of US patents that were filed mainly in the 1960s and 1970s ,23,24,25A micromachined biomimetic fluid rotor angular rate sensor was introduced by Andreou et al. in 2014 to be used for vestibular prostheses ; wherebyThere are other types of micro-machined fluid inertial sensors that were introduced twenty years ago. In comparison to MEMS-based IMUs, fluid-based inertial sensors use fluid instead of a solid proof mass to detect the inertial forces. Consequently, the fluid-based inertial sensors are advantageous in terms of having a simple structure, low cost, high shock resistance, and large measurement ranges; whereas, the sensitivity and the bandwidth of the fluid-based inertial sensors are less competitive with the MEMS-based inertial sensors .These micro-machined fluid gyroscopes are based on either jet flow or thermal flow. However, micro-machined fluid gyroscopes are less mature. The jet flow gyroscopes use a laminar gas flow driven by a micro pump, whereby an external rotational motion deviates the flow with an amount proportional to the angular rate by which the rotation was applied . The micFluid-based angular accelerometers share most of their technologies and operation concepts with the fluid-based angular rate sensors. However, in their design, fluid-based angular accelerometers lack the signal conditioning mechanism that would integrate their output signal to be an angular rate signal. Nonetheless, fluid-based angular accelerometers can be thought of as the predecessors of fluid-based angular rate sensors; whereby, fluid-based angular accelerometers are usually modified to reach fluid-based angular rate sensors. Consequently, the classification of fluid-based angular accelerometers technologies shares most of its nomenclature with the earlier mentioned angular rate sensors technologies.One of the first fluid-based angular accelerometer technologies was the fluid rotor angular accelerometer. Fluid rotor angular accelerometers are classified in literature into liquid rotor angular accelerometers and gas rotor angular accelerometers. Furthermore, the liquid rotor accelerometers can be further classified in terms of their motion pick-off mechanism. Liquid rotor angular accelerometers with an electric pick-off mechanism were first introduced in 1957 by Statham laboratories . This waWith the revolution of micromachining industries, micromachined and miniaturized versions of the liquid rotor angular accelerometers emerged, such as the one reported by Wolfaardt and Heyns in 2008 . AnotherAnother scientific concept that was used to realize fluid-based angular accelerometers was the heat-mass transfer of fluids. The concept was also utilized for manufacturing angular rate sensors, as stated above. Thermal flow angular accelerometers were reported initially by Benedetto and Linder in 1980 ,38.Recently, micromachined biomimetic versions of the thermal flow angular accelerometers were reported by a research group at the University of Twente in 2014, to use these sensors to provide low cost inertial sensors. The reported thermal flow used water as its fluid mass and used the thermal variations of the fluid flowing in an annular channel to derive the angular acceleration imparted to the sensor through solving Navier–Stokes equations. Nonetheless, the reported sensor provided poor performance which cannot be used for vestibular protheses .The same research group at the University of Twente reported a micromachined biomimetic MHD flow angular accelerometer, which was also intended to be a low-cost sensor that utilized tap water and saltwater solutions as fluid mass flowing in microfluidic annular flow channels. The flow was induced when the sensor was subjected to angular motion, and the flow was detected and measured utilizing the MHD effect. However, the sensor also reported a poor performance. The rate-integrating gyroscopes include various typical error processes in their output. Such error processes include g-insensitive biases, g-sensitive biases, isoelastic biases, scale-factor errors, cross-coupling errors, and zero-mean random biases. Each error process has its own sources, which are discussed thoroughly in . NeverthMagnetohydrodynamic angular rate sensors require proper magnetic shielding for the indication component to ensure proper rate measurement. Otherwise, the sensors suffer from high bias instability due to their dependency on the magnetohydrodynamic effect.Fluid rotor angular rate sensors suffer from low bandwidth and low dynamic range in comparison to other angular rate sensors. Nonetheless, the literature did not provide proper performance assessment for such sensors, especially for navigation applications.In the early developments of these kinds of sensors, the major problems were achieving adequate stability, resolution, and insensitivity to environmental impacts especially temperature variations. Consequently, the fluid-based sensors of this form were not considered as plausible sensors to be used for navigation applications.Micromachined fluid-based gyroscopes suffer from drift due to improper thermal compensation and vulnerability to variations in ambient temperature. Furthermore, micromachined fluid-based sensors suffer from low bandwidth and low sensitivity. Moreover, the main research aspects to be considered for micro-machined inertial sensors are enhancing temperature compensation, developing a full monolithic IMU, and reducing the cross-coupling errors ,28.Angular accelerometers are mathematically incompetent relative to angular rate sensors. The reason lies within the mathematical integration process that is required to retrieve the attitude parameters of a moving platform. Such an integration process over time takes place once for angular rate sensors; whereas, the integration process takes place twice for angular accelerometers. Typically, the integration of the sensor output signal over time acts as a low-pass filter, which eliminates short-term errors such as white noise within the sensor measurements. However, the integration cannot handle nor eliminate long-term errors, such as the bias-instability or in-run variations within a sensor output signal. Consequently, it is preferable to implement the integration process as little as possible to evade long-term error accumulation within the derived quantities and numerical integration truncation errors due to digital signal acquisition from modern sensors.Moreover, what makes angular accelerometers most unsuitable for inertial navigation applications is an inherited error within all angular accelerometers, known as the threshold error . The thrAs established earlier, angular accelerometers are of low performance in comparison to angular rate sensors, especially for navigation applications; however, some fluid-based technologies that are utilized in the manufacture of angular rate accelerometers are prone to errors that degrade the performance of such sensors even more. The main drawback of liquid rotor angular accelerometers is non-linear responses due to changes in temperature and thermal gradients across the sensor. The changes in ambient temperature can be compensated using volume compensators. However, the sensors should be carefully designed and provided with proper thermal shielding to avoid such effects. Moreover, some designs are sensitive to linear accelerations, which induce errors in the measured angular accelerations. Similarly, the gas rotor angular accelerometers suffer from temperatures gradients across the sensor. Furthermore, proper shielding for the electronic components is required to avoid stray capacitances from affecting output signals.To sum up, regardless of the technology used to build the inertial sensor, high bias instability is a characteristic of all low-cost inertial measurement units due to either mechanical fabrication imperfections, poor electric pick-off mechanisms, analog to digital signal converters for mechanical sensors, or thermal drift. Additionally, low cost IMUs are characterized by a low signal-to-noise ratio. On the other hand, inertial sensors of high performance are typically of high cost and not free from bias instability in their output.One objective of the proposed particle imaging velocimetry gyroscope (PIVG) is to eliminate bias instability that is encountered in low-cost inertial sensors. To be specific, all inertial sensors suffer from a drift rate that occurs due to various reasons, regardless of the IMU grade or cost. The main objective of the current invention is to provide a nearly drift-free gyroscope, that provides drift rate values less than the listed values for strategic grade IMUs.An objective of the proposed PIVG is to eliminate errors that typically occur in the analog to digital conversion process that takes place in any IMU, such as quantization errors. Furthermore, the PIVG does not require signal conditioning to derive the measured angular rate from another measured quantity, such as deriving the angular rate from a measured direct current (DC) voltage, as is the case in most inertial sensors.Moreover, one objective of the proposed PIVG is to provide an output of a relatively high signal-to-noise ratio (SNR), in comparison to IMUs that are suited for navigation and stabilization applications.Conforming with fluid-based inertial sensors, the system temperature extremely affects the sensors’ performance. Consequently, it is considered an objective of the current invention to provide a gyroscope that does not include moving parts except for a single particle, which does not cause variation in the temperature of the system and maintains constant throughout the operation time of the sensor, provided that the system assembly considers proper shielding from ambient temperature. Additionally, the proposed PIVG ensures a design that is not affected by variations in the magnetic fields within the environment in which the sensor is operated.As stated earlier, the cost of an IMU is dependent on the technology and the performance of said IMU, such that the cost increases as the IMU performance becomes better. Hence, the proposed PIVG is of a relatively low cost in comparison to typical IMUs that share the same performance and characteristics.Additionally, another objective of the proposed PIVG is to be compact for use in typical navigation applications. The design ensures that the PIVG can be mounted for typical navigation platforms, especially in terms of size; such platforms include ground-based, aerial, and marine platforms. Moreover, the design ensures a simple structure that does not require special precautions or procedures in sensor fabrication and operation processes.The proposed fluid-based gyroscope PIVG depends on fluid dynamics to provide the inertial measurements. The concept is that when an inertial force is imposed upon a control volume of fluid, a flow is generated in response to such force to satisfy Newton’s second law of motion. Hence, once the flow is detected, and through the implementation of fluid dynamics theories, the actual inertial measurement can be acquired. The fluid dynamics theories, which are of interest to design this system, are the law of conservation of momentum implemented for incompressible fluids, alternatively referred to as Navier–Stokes equations, and the continuity equation for incompressible fluids. The concept is applicable for either linear or rotational types of motion. However, the PIVG considers the detection and measurement of rotational motions.From the proposed nomenclature, PIVG employs a version of particle imaging velocimetry to perform the inertial measurements. Particle imaging velocimetry is a branch of fluid dynamics science in which the properties of fluids and fluid flows are determined through tracking particles that are neutrally buoyant and mimic the actual dynamics of the flow. Particle imaging velocimetry implies the use of an imaging sensor that is used to track the particles, whether those imaging sensors are set up in fixed or moving positions along the examined flow. Research has been extensive in particle imaging velocimetry with vast technical advancements in digital image processing to acquire more precise, computationally efficient, and high rates of data. However, in this adoption of the concept within PIVG, only one particle is monitored within a predefined control volume flow channel, and the particle monitoring is done using a fixed imaging sensor with respect to the pre-located and fixed flow channel. The motion of such a particle is determined from the acquired sequence of images via a series of digital image processing techniques.To lay down a basis for particle tracking, the projectivity condition is implemented to determine the relative position of the particle with respect to the image space coordinate system. It is noted that the image space represents a constant view of the flow channel in which the particle is only in motion as result of the fluid flow. Since the object space geometry is known, the use of a single imaging sensor can determine the object space position of any point that is an element of the image space view. Consequently, the position of the particle can be determined directly.The adopted design of the PIVG is an open-loop design, in which the sensor performs the measurement without the need of an external restoring mechanism that restores the sensing element to its original state. Nonetheless, the sensing element within the PIVG is designed such that it assumes its original position after each measurement on its own as per the proper damping mechanism provided within the PIVG design.Any inertial measurement sensor constitutes three main components that are crucial for it to perform the inertial measurement. The components are the motion transduction mechanism, the signal conditioning component, and the sensor readout component. The motion transduction mechanism comprises the sensing element, which is sensitive to the applied external motion to the sensor, and a damping mechanism that eliminates the effects of oscillations and disturbances that occur whenever the external stimulus is unapplied to the sensor. The efficiency of any given damping mechanism is measured by the amount of time required for the sensing element to stabilize after an external disturbing force is removed. The damping mechanism is the mechanism responsible for restoring the sensing element to its stable state . For closed-loop sensors, the damping mechanism takes the form of external input means that apply an appropriate form of restoring force or couple to the sensing element. However, for open-loop sensors, the damping mechanism relies on the physical properties and the system design to apply the proper damping forces/torques that are required to stabilize the system.The signal conditioning component is used to apply a modification or adjustment for the signal acquired using the motion transducer and generating a proper output signal for the readout mechanism. The signal conditioning phase might include applying basic operations to the signal, such as filtering, integration, or differentiation. For some sensors, signal conditioning is applied to increase the amplitude or the power of the signal to be able to drive the readout component.The readout component is a mechanism that transforms the signal into a comprehensive form of information that can be understood by human recognition or can be fed into a controller for mechanical instruments and applications. Hence, any sensor must include a readout mechanism. The readout mechanism for an inertial sensor is usually fed into a controller that can be either used for machine control, platform stabilization, or navigation.1. Motion IndicatorPIVG consists of three fluid flow channels placed in three mutually orthogonal axes, with a camera that is facing each flow channel to track a particle that is placed in each flow channel. The particles are chosen to be of the same density of the fluid filling up the flow channels to be neutrally buoyant.The described PIVG design represents a three-axis gyroscope; however, the PIVG design can be reduced to be a single axis gyroscope. The schematic shows that the PIVG is chosen to be in the form of a cuboid, where there is a flow channel in the shape of a circular torus fixed on each face. There is an imaging sensor facing each flow channel to capture the motion of the particles within each flow channel. For the chosen embodiment of the invention, the imaging sensor is chosen to be a digital camera.Once the PIVG is mounted over a moving platform, the imaging sensors capture the motion of the particles through the circular flow channels. The acquired images are then processed to determine the location of the particle. Consequently, the location of the particle within each image is compared with respect to its initial location before motion. To illustrate, the vector , which is the difference between the slopes of the two vectors  can be adjusted by manipulating the system design components as discussed hereinafter.Nonetheless, when the PIVG is operated, and as soon as the external applied angular rate is removed, the viscous drag enforces the particle to reach back to its initial stable position . However, it should be noted that the design should be adjusted to reach the required damping effect, whereby, the damping effect can be described in terms of the precision of the particle to return to its stable position, and the amount of time that the particles take to restore to a stable position. The proper damping effect implies a high accuracy of position restoration and the least amount of time for the particle to reach stability.One of the main advantages of the PIVG is that the sensor does not require any signal conditioning, as per the proposed sensor design. The sensor does not require any piece off additional hardware that is responsible for modifying the acquired signal. Rather the signal is directly transmitted to the readout component.For the PIVG, the readout mechanism can be any electronic processing unit that acquires the output of the utilized imaging sensor. The electronic processing unit is used to implement the designated image processing algorithms to acquire the position of each particle within its respective flow channel throughout the sensor operation time. The time series of each particle position is directly the output signal that can be used as an indication for the angular rate by applying the proper signal calibration.PIVG performance depends on the temperature and pressure stability such that the driving and damping forces governed by the fluid depends on the fluid properties remaining constant throughout the entire operation of the gyroscope. Hence, it is crucial to provide means for stabilizing the temperature and the pressure of the fluid control volume; the pressure is kept constant by the design of a well-sealed fluid torus compartment.The system design considers a set of design aspects for each component of the system. The design aspects are chosen based on the system components and the fluid dynamics within the chosen control volume for the fluid. These design aspects are interrelated and cannot be separated, rather they should be addressed holistically and are discussed as follows.The current embodiment of the sensor implies the use of an imaging sensor to acquire the output signal for the sensor. Hence, the proper design of the PIVG should propose a set of specifications for the imaging sensors that are used to detect the inertial motion of the particles. Such specifications should include the field of view (F.o.V.), focal length, variable or moving lenses, spatial resolution, and temporal resolution.However, it is clearly stated that any optoelectronic sensor can be used instead of typical imaging sensors to ensure a smaller size and higher data rate for the PIVG.The flow channel was chosen to be in the form of a circular torus, as depicted earlier by the schematic in ρ is the fluid’s density, v is the velocity of the flow, d is the cross-sectional diagonal dimension, and μ is the fluid’s coefficient of viscosity.Moreover, the flow channel should be clear and transparent to go with the chosen sensing mechanism . The design considerations for the flow channel should also include the overall diameter of the torus and its cross-sectional area. The dimensions of the circular torus pipe should follow the limits of Reynold’s number which ensures a laminar flow. Reynold’s number is a unitless quantity that represents in its abstract sense the ratio between the inertial forces with respect to the viscous force acting upon a fluid flow. The formula to compute Reynold’s number .The dimensions should ensure that the particle lies at the point of maximum velocity within the velocity gradient of the cross section of the flow channel and at the same time does not block the fluid flow.Furthermore, the dimensions  of the particle should be chosen such that it is the least likely to be represented and detected within an image as per the spatial resolution of the deployed imaging sensors. It can be stated that the precision of the detection of the centroid of the particle within any given image would affect the PIVG measurement precision. Hence, as the size increases, the uncertainty in the detection of the centroid of the particle increases accordingly. Therefore, the choice of an appropriate size for the particle should balance between being apparent in the image space and being as small as possible to minimize the uncertainty in its detection.Moreover, the shape and size of the particle determines, in association with the fluid flow channel design and the type of fluid, the amount of damping force that can be applied to the particle, and hence, to the system overall. The size  of the particle should be chosen such that the ratio between the cross-sectional area of the particle to the cross-sectional area of the fluid flow channel is optimized to obtain the designated damping effect. Additionally, it is obvious that the shape of the particle is also a basic factor in defining the viscous drag force created upon itself, when interacting with the fluid layers, whereby, the viscous drag force is factored by the impact of the shape around which the fluid is flowing. Nonetheless, the shape of the particle is studied, as well, to optimize the damping effect within the PIVG, as per the design implications.Digital image processing comprises a crucial aspect in defining PIVG performance. The precision by which the chosen digital image processing algorithm can determine the centroid of the particle defines different PIVG performance parameters including the signal-to-noise ratio (SNR) and the angular random walk.The chosen digital image processing algorithm includes two sequential phases. The initial phase is motion detection of the particle. The initial phase depends on the fact that the image space view is constant and still, except for the particle in the case of external motion. Hence, the initial phase implies computation of the sum of absolute differences of any given image during PIVG operation with respect to the initial image at the beginning. Consequently, the approximate location of the particle, at the initial and current epochs, within the vicinity of the image space can be determined by setting a threshold and binarizing the image as per such threshold. Those locations would appear as white blobs within the black background of the binary image. The second phase is based on the output of the initial phase, such that the computed centroid is used to create a region around itself with the dimensions of the bounding box of such a blob, as depicted by As explained earlier, the angle confined between the vector connecting the current particle position and the center of the circular torus projection within the image, and the vector connecting the initial particle position and the center of the circular torus projection within the image space, can be computed.The following section shows the current embodiment of the sensor. It is noted that the current embodiment of the sensor is subject to variations and modifications to reach the optimum sensor design, in terms of cost, structure simplicity, size, and performance. The PIVG, as stated earlier, is introduced as a triaxial gyroscopic assembly, which can be adopted to a single axis, as per the application requirements. Nonetheless, the invention embodiment would assume the triaxial configuration for the sake of generalization, with emphasis on the PIVG being adaptable for inertial navigation applications.As shown in the above figures, the sensor comprises a set of three perpendicular planes in the form of a cube, whereby each plane constitutes a set of three concentric fluid flow toroidal channels. The fluid flow channels are made of transparent material such as glass, acrylic, or an equivalent material that ensures transparency and sufficient material strength to resist vibrations and shocks. Additionally, the material must allow manufacturing tolerances that ensure the fluid flow channel can be hermetically sealed to preserve the pressure of the contained fluid at a constant value. Each fluid flow channel has a fitted valve that is used for filling the channel with fluid, and as an entry passageway to the particle. As explained earlier, each flow channel is filled with a fluid that is chosen to optimize the performance of the PIVG. A color-coded spherical particle is inserted within each fluid flow channel, such that the particle is neutrally buoyant, and with specific known dimensions. The material of each particle is chosen such that the particle manufacturing process should ensure a smooth surface for the particle, such that it minimizes the chance of creating micro turbulences around the particle, which might affect the particle motion within the channel. The particles are color-coded to facilitate and ensure the potency of the digital image processing phase within the PIVG operation.The rationale behind adding another two toroidal flow tubes within each measurement plane of the PIVG is to increase the overall sensor sensitivity for angular rate measurements. In such a case, each fluid flow channel contains a different fluid, has different dimensions, and contains particles with different shapes and size, whereby the system of flow channels provides angular rates around its concentric axis, with each representing a different range of sensitivity. The particles are tracked within each fluid flow channel, and the proper particle to be considered is determined by applying the digital image processing algorithm as explained earlier. The processing algorithm defines the time series of the location of each particle within the set of concentric flow channels. For each particle, the motion rate is determined instantly. Afterwards, a particle reflecting an insensitive reaction to a low dynamic motion or a particle showing excessive hyper-reaction to a high dynamic motion is eliminated from the computation process of the angular rate. It should be noted that a particle is defined as reactive to external applied motion based on a prior set of calibration tests performed on the PIVG.Another attribute of a set of concentric toroidal fluid flow channels for each measurement axis is to provide redundancy in measurements, higher dynamic range, and higher dynamic sensitivity. Moreover, they can act as checks for proper measurements. Hence, the processing algorithm is supplemented with an additional phase by applying a proper estimation algorithm that ensures that the PIVG provides the best estimate of an angular rate.For each a plane within the PIVG that contains a set of fluid flow channels, there is an opposing plane that contains a high-resolution small-sized digital camera module, with a high frame rate and large F.o.V. Preferable digital camera modules have an F.o.V. within 80° to 120°. Adjacent to the digital camera module, on any given face, a pair of light sources is mounted, one on each side of the digital camera module. The light sources are used to illuminate the fluid flow channels. Consequently, the triaxial PIVG should contain a set of three digital camera modules, each placed on a face opposite a face containing the fluid channels. Additionally, the triaxial PIVG contains a set of six light sources. The assembly of digital camera modules and light sources are connected to a wiring block that facilitates the connection of the electronic elements to the on-board electronic processing unit. The on-board electronic processing unit is used to implement the designed processing algorithm in real time mode.The electronic processing unit is then connected to an interface, which is considered as a controller used for providing feedback for a machine, or robot, or can be used for performing automated mobile navigation and mapping. The interface also includes a monitor that provides the measurements for instantaneous visualization and a storage module for data storage. The total assembly of electronic modules are connected to a power supply as shown in The PIVG should be provided with a thermal compensation mechanism that is utilized to account for the change in ambient temperature and internal temperature variations. The thermal compensation mechanism comprises a set of internal ventilation passageways that isolate the fluid-contained compartments, as depicted by To validate and prove the concept of the PIVG, a series of experiments were held to determine the performance of a prototype of the PIVG. The prototype comprised only a single toroidal fluid-filled channel instead of three channels. The preliminary design of the initial PIVG prototype used for this experiment was a single axis gyroscope design. The body of the adopted single axis PIVG was fabricated as a wooden box in the shape of a cube with a side length 30 cm. The circular torus flow channel is made of a transparent vinyl flexible pipe with an overall diameter 18 cm and a cross-sectional diameter of 1.30 cm. The flow channel is fixed to the bottom of the wooden box. On the opposite face to the flow channel, a smartphone’s camera is mounted to be used as the imaging sensor. The smartphone used in this experiment is LG X-power smartphone, and 3.The used fluid is a water sodium-hydrochloride (NaCl) solution with 0.7% concentration. The solution was realized with a mixture of ordinary tap water and table salt. The reason behind using a NaCl solution is to achieve a fluid density that is equal to the density of the particle which was used. The particle used was a plastic hollow sphere of with a diameter of 5 mm and relative density of 1.1 gm/cmThe experiments mainly address the sensor response validity in terms of amplitude and SNR. Furthermore, the experiments address the hypothesis that the PIVG is nearly drift-free, and that the sensor can maintain substantially long periods of time with no induced errors in the PIVG output signal.The objective of this experiment was to test the PIVG response to external angular rates. The experiments were designed to address the amplitude response of the PIVG for a given dynamic range. The amplitude response of a sensor is a measure of system interaction under the impact of the external effect. It is a measure of a system’s ability to faithfully sense and transmit the information required from the sensor to measure.2. Nonetheless, for these experiments, the turntable was used a single degree of freedom (D.o.F.) setup to test a single axis of the PIVG.The experiments were held in a laboratory where the PIVG was placed and centered on a servo driven turntable. The used turntable is a high-end turntable that is used for precise calibration and angular rate tests for angular rate sensors. The turntable has two degrees of freedom (D.o.F.) and is controllable via a PC-connected controller. It can be operated within a wide range of input angular rates, up to 220°/s, and a range of operating angular accelerations, up to 50°/sTo be more conservative, and to validate the PIVG measurements, the experiments were performed simultaneously with another reference gyroscope mounted on the turntable. The reference gyroscope is Xsens MTi-G-710 GNSS-aided IMU, which is a stable commercialized sensor that is used for inertial navigation.2 to 50°/s2. The experiments were held such that each value of angular acceleration was fixed while changing the values of the angular rates from 10°/s to 180°/s with increments of 30°/s apart. Consequently, the angular acceleration value was increased by 10°/s2 increments and the angle rate tests were repeated. The duration of each experiment was about one minute. The turntable was used to test a single axis of the PIVG over a series of experiments at different angular rates that were chosen to test the operability of the PIVG for the dynamic range encountered by most typical navigation and mobile mapping platforms. Moreover, the PIVG was tested for its response for different values of angular accelerations to address the sensitivity of the PIVG angular rate measurements to angular accelerations. The tests for the effect of angular acceleration are referred to as centrifuge tests . Nonethe2. The figure emphasizes the difference between the SNR for the PIVG output signal in comparison to the reference gyroscope output signal.The results show that the PIVG provided substantially accurate results in comparison to the input angular rates from the turntable controller and the reference gyroscope results. Additionally, the PIVG processing model demonstrates substantially a proper response with minimal delay in comparison to the reference gyroscope response and shows that the PIVG is not affected by variations in angular accelerations. Moreover, the results show that the PIVG has a relatively high SNR in comparison to the reference gyroscope.Finally, the results of the experiments have shown the repeatability of the PIVG results under different operating conditions. Hence, the PIVG can be used as a reliable angular rate sensor for various applications, as discussed earlier. However, the PIVG must pass through a series of calibration tests that are crucial to ensure the performance that is shown by the above results. The calibration tests account for the sensor modeling parameters and environmental operation conditions.The main objective of this invention was to have a nearly drift-free angular rate sensor, as stated earlier. The objective of this experiment was to validate that the PIVG is a nearly drift-free angular rate sensor, which can operate for substantially long periods of time without the occurrence of drift rate within the output angular rate signal. Typically, there is no angular rate sensor that does not have a component of drift rate within its output; however, such drift rate is minimal in high-end inertial sensors. On the other hand, these sensors tend to be of an extremely high cost. The PIVG is validated through this experiment to be the first nearly drift-free inertial sensor at an extremely low cost, in comparison to sensors of comparable performance, especially in terms of drift and signal stability.The experimental setup of this experiment is like the experimental setup discussed for the previous set of validation experiments. However, the PIVG was mounted in the turntable for a single axis experiment. The duration of the experiment was nearly five hours of operation, where the turntable was rotated at an angular rate of 60°/s. The data rate of the PIVG was set to 24 Hz through the frame rate of the utilized camera module. Hence, the total number of samples acquired by the PIVG was 472,769 samples.From the signal shown in Consequently, it is obvious from these results that the PIVG has no drift rate in its output signal over an operation period of nearly five hours, which covers most of the requirements of inertial navigation applications at a relatively low cost. It is also noted that the SNR (shown in the results of this experiment), although comparable to the current state of the art angular rate sensors, can be enhanced by using better imaging sensors with higher spatial resolution.To conclude, this paper introduces a fluid-based angular rate sensor that is nearly drift-free with a high SNR. Such a fluid-based inertial sensor is realized at a relatively low cost in comparison to systems of the same performance. The introduced fluid-based angular rate sensor is referred to as PIVG. PIVG utilizes fluid as the inertial mass and operates on PIV concepts.A prototype for the PIVG was designed and fabricated as a proof of concept. The prototype was tested against a commercial industrial grade IMU. The results showed the repeatability and accuracy for rate transfer tests. In addition, the results also showed that the PIVG is nearly drift-free; whereas, the PIVG recorded a zero-mean error signal with a standard deviation of 0.0254°/s for an operation period of nearly five hours.Research is ongoing to better model the PIVG response and calibrate the various systematic errors and model any stochastic errors that are encountered with the use of the PIVG.This research entails a pending patent; however, research is currently ongoing to achieve a fully functional prototype for the PIVG .


Following a substantial increase in the utilization of extracorporeal membrane oxygenation (ECMO) during the last decade, its associated benefits and complications, including acute kidney injury have become more apparent. Acute kidney injury requiring dialysis during the ECMO treatment is very common and is associated with adverse outcomes. Cross talk between ECMO and dialysis equipment has been debated in the literature in order to enhance the quality of dialysis and avoid its potential adverse events.Na et al. recently published the results of a prospective experiment by using three different methods for integration of the continuous renal replacement therapy device into the ECMO circuit. In this experiment, the investigators showed that by using three different connection strategies between continuous renal replacement therapy device and ECMO and the utilization of three separate structures of pressure control lines, the dialyzer lifespan could be optimized.In this commentary, following a brief review of the ECMO and dialysis devices history and cross talk, we discuss the findings by Na et al. and provide additional insights for future investigations. Extracorporeal membrane oxygenation (ECMO) was first tried in patients with respiratory failure in the early 1970s by Donald Hill, a surgeon in the San Francisco area . SubsequAcute kidney injury (AKI) is a common complication in patients receiving ECMO due to multiple injurious mechanisms, including underlying inflammation, hypoperfusion, and exposure to nephrotoxins , 6. The The pressure within the ECMO circuit is variable depending on the location before or after the centrifugal pump. The pump generates a negative pressure in the inlet arm − 20 to -100 mmHg) in order to pull a predefined blood volume from the patient . After the pump, however, the intra-circuit pressure is positive to ensure adequate blood flow into the oxygenator and eventually into the patient. In comparison, CRRT devices are set up to be connected to venous pressure 0 to + 20 mmHg   is essential to avoid potentially life-threatening complications. Connecting the CRRT device to the ECMO circuit may result in high- or low-pressure alarms depending on where the CRRT connections are made in relationship with the ECMO pump. To avoid high access pressures in the CRRT inlet line, the options are to connect the access line to the ECMO circuit pre-pump or to decrease the CRRT blood flow rates . SimilarN = 16). Among patients in the other two groups, the CRRT device was integrated into the ECMO circuit using Luer Lock connections with an additional pressure control line on the inlet (N = 36) or both the inlet and outlet lines (N = 118), respectively. While the investigators did not find any difference in platelet count, lactate dehydrogenase, and hemoglobin values among the 3 groups, they reported significantly higher CRRT circuit lifespan when a pressure control line was applied to both, the inlet and outlet of the CRRT circuit (avoiding high positive or negative pressures within the CRRT circuit) in comparison with the other two groups. In addition, they observed significantly lower mortality among patients with double pressure control lines compared to the other two groups.Na et al.  analyzedThis study provides substantial knowledge regarding the practical aspects of ECMO-CRRT cross-talk and shows the impact of simple interventions on the cost of treatment and filter lifespan, and potentially their effects on the clinical outcomes. Importantly, the authors observed that patients who required RRT following initiation of ECMO started RRT shortly after initiation of ECMO, i.e., median of 1 (0–2) day. Naturally, patients who require ECMO have significant organ failure (at least respiratory or cardiovascular) and potentially a high incidence of volume overload. In this patient cohort, the gap between demand and endogenous capacity of kidney function widens fast, which may warrant the initiation of RRT very early in the course.Na et al. showed that simple interventions to reduce the pressures within the dialysis circuit were associated with an increased lifespan of the dialyzer, and potentially lower costs. However, it is not clear how to explain the reduction in mortality among patients who had pressure control devices on both sides of the CRRT machine. In addition, a higher incidence of metabolic acidosis at the initiation of RRT was noted in the first group compared to the other two groups, which suggests that overall clinical practice may have changed during the course of the study.While the study by Na et al. provides excellent insights regarding the impact of ECMO-CRRT cross-talk on the quality of care and potential patient outcomes, it does not offer a generalizable approach for the integration of ECMO and CRRT devices as there is significant variability between the programs regarding the type of devices, patient populations, indications of extracorporeal life support and CRRT, and finally the prescriptions used for each of the device. However, the analysis offers a path to a better understanding of this cross-talk and provides options on how to overcome challenges related to intra-circuit pressures in order to optimize treatment.