Cyprinus carpio) and tilapia (Oreochromis niloticus), that water tracers displayed higher curvatures than food tracers, indicating an inertia-driven filtration. In addition, tilapia also exhibited a ‘central jet’ flow pattern, which aids in quickly carrying food to the pharyngeal jaw region. When the food was trapped at the branchial basket, it was resuspended and carried more centrally by periodical bidirectional waterflows, synchronized with head-bone motions. By providing a complete picture of the suction-feeding process and revealing fundamental differences in food transport mechanisms among species, this novel technique opens a new area of investigation to fully understand how most aquatic vertebrates feed.Virtually all fishes rely on flows of water to transport food to the back of their pharynx. While external flows that draw food into the mouth are well described, how intraoral waterflows manage to deposit food at the esophagus entrance remains unknown. In theory, the posteriorly moving water must, at some point, curve laterally and/or ventrally to exit through the gill slits. Such flows would eventually carry food away from the esophagus instead of toward it. This apparent paradox calls for a filtration mechanism to deviate food from the suction-feeding streamlines. To study this gap in our fundamental understanding of how fishes feed, we developed and applied a new technique to quantify three-dimensional (3D) patterns of intraoral waterflows in vivo. We combined stereoscopic high-speed X-ray videos to quantify skeletal motion (XROMM) with 3D X-ray particle tracking (XPT) of neutrally buoyant spheres of 1.4 mm in diameter. We show, for carp ( How organisms acquire food is largely determined by the medium in which they feed . AquaticA wide range of aquatic vertebrates across a large size range employ suction feeding: from larval fishes  and frogIn virtually all fishes, waterflows generated by suction are essential for transporting food inside the buccal cavity . During In contrast to this relatively straightforward and well-resolved mechanism of food capture toward the mouth, food transport mechanisms inside the buccopharyngeal cavity are not as simple  and are To date, there is no evidence that the esophagus expands in concert with buccopharyngeal suction generation to directly draw in a significant amount of water together with the food. Therefore, other structures need to be involved in separating food from the waterflows. Ideally, in bony fish, the food that is sucked into the mouth could be intercepted by the pharyngeal jaws located just anterior to the esophageal sphincter . The mobAlternatively, two other intraoral flow patterns are theoretically possible, which may facilitate interception by the pharyngeal jaws . ComputaCyprinus carpio) and tilapia (Oreochromis niloticus). These species were chosen from the large diversity of suction feeders because their adult head size is small enough to fit the imaging volume of the biplanar X-ray setup, yet large enough to avoid hindrance from an intake of tracer particles for waterflow inside the buccal cavity using X-rays. We use this data to describe the intraoral waterflow and determine the intraoral mechanisms of aquatic feeding that have remained hidden thus far.Despite this critical role during feeding in nearly all fishes, the spatiotemporal pattern of intraoral waterflow has still not been quantified in vivo due to the difficulty in gaining optical access to the buccopharyngeal cavity. This implies a fundamental gap in our knowledge regarding how more than half of all vertebrates feed. Here, we develop a new technique based on biplanar high-speed X-ray videography to simultaneously quantify the 3D path lines of both food and intraoral water, as well as skeletal kinematics. We apply this technique to two distantly related omnivorous suction feeders, carp , the water tracers moved alternatingly backward and forward, interspersed with phases of stasis . At the Then, we observed a rapid increase in the posterior velocity of the water tracers, reaching up to 92.1 ± 59 cm/s for the carp and 97.8 ± 23 cm/s for the tilapia. We named this rapid increase "intake" , correspAfter the intake, a phase of apparent stasis occurred, with a steady, low-velocity, anteroposterior water tracer trajectory, leading to a nearly complete stop of the food.Then, the water tracers started to move again, corresponding to the beginning of the intraoral food-handling phase. This phase was characterized by lower velocities compared to the capture phase (the maximum during food capture was 32.9 ± 11 cm/s in carp and 24.7 ± 12 cm/s in tilapia) and the occurrence of bidirectional motion: reverse flows (rfs), defined by negative anteroposterior velocities of the water tracers  . We founAmong the 73 trials recorded, no water tracers passed the esophagus; however, the food was successfully ingested in most cases . We were able to fully analyze  seven trials for carp and six trials for tilapia. Among these trials, the total distance traveled by the food tracers during the pooled intake, the rf, and the bf phases was significantly lower than the distance covered by the water tracers , demonstThe 3D radio-opaque particle-tracking method allowed us to export the 3D trajectory of all the water tracers during a suction-feeding sequence. This provided us with a visualization of the water tracer paths for both the lateral and dorsoventral views during the intake , the rf During the intake, the food item usually followed a trajectory that was more dorsal in comparison to the water tracer trajectory in both species . In addiDuring the other phases, the water tracers spread throughout the entire buccal cavity, but their trajectory remained predominantly anteroposterior . After tEspecially for tilapia, at the end of the intake, many flow tracers moved anteriorly after starting to curve in the lateral direction . Such anWe found no correlation between the moment a tracer was ingested and its final location in the buccal cavity. For both species, the Kendall test of correlation between the location at the moment of ingestion (tz at the beginning of intake) and the final location of the particle (tz at the end of intake) was not significant . However, visually, it seems that early-sucked particles tend to stay in the medial plane of the buccal cavity, participating to the central jet, whereas the late ones tend to end up on the lateral parts of the buccal cavity .Generally, the particles tend to stay on the same side as their initial location, but there can be a change of side before the particles arrive close to the esophagus . A KendaThe 3D kinematic analysis of the head bones revealed a stereotypical sequence of motions of the upper and lower jaws (gape), hyoid depression, and opercula abduction . During the intake  can be easily combined with 3D motion reconstruction protocols for skeletal elements  developing a "central jet" flow pattern inside the buccal cavity  and (2) Secondly, the path curvature from a dorsoventral viewing perspective was significantly higher for the water tracers compared to the food tracers , indicatAs shown in previous work , the feeOur data provide the first empirical evidence of bidirectional waterflows being synchronized with motions of the upper and lower jaws, hyoid, and opercula to create anterior-to-posterior waves during bfs and posterior-to-anterior waves during rfs. The anterior-to-posterior wave of head expansion and compression is already well known in fish . This isIn conclusion, our study highlights the biomechanical complexity of the processes occurring inside the mouth cavity during aquatic feeding in fishes. We showed how high-velocity suction flow patterns during the initial food intake phase can have an effect on the food’s initial deposit site, and how centralized suction streams may help to direct food toward the esophagus. The omnivorous species studied here during benthic feeding followed this phase by repeated, hydrodynamically driven repositionings as part of their food sorting and tasting abilities. Future research using our X-ray particle-tracking technique could further reveal how these processes, and their intraoral hydrodynamic patterns, may vary between feeding modes , between animals with varying head morphology and size, etc. Such studies would further advance our currently limited insight into the optically invisible processes of aquatic feeding in vertebrates.C. carpio), carp 01  and carp 02 , purchased from a pond shop , and two Nile tilapia (O. niloticus), tilapia 01  and tilapia 02 , from a farm , were housed in two large water tanks at the University of Antwerp and provided with food ad libitum.Two carp , the number of trials , and the consistency of the results imply that the mainly qualitative conclusions on overall patterns in waterflow and skeletal motion are well supported.Prior to videography, the fish were anesthetized with 50 mg/L MS222 (ethyl 3-aminobenzoate methanesulfonate) and implanted with 0.35-mm-diameter beads made of a tin and silver alloy. Using hypodermic needles, four markers were implanted in the neurocranium, and three on the upper jaw, lower jaw, hyoid, and left operculum . The lefThe water tracers had to fulfill two typically incompatible requirements: (1) being radio-opaque to be trackable on the X-ray videos and (2) being small enough and neutrally buoyant to passively follow the water trajectory as closely as possible. An easy-to-execute and cost-efficient procedure was developed to make spherical waterflow tracers of minimal size to be used with biplanar high-speed X-ray video setups. Like the existing tracer designs , we surrEPS foam spheres with diameters between 1 and 2 mm were purchased  and sieved to retain only those with a diameter of 1.4 mm. To do so, the foam spheres that were loosely stuck in between the mesh holes of a precision test sieve were extracted. Next, 0.4-mm-diameter brass rods  were cut into approximately 0.1 m pieces in length, and their tips were sharpened by holding the rod at a sharp angle against a grinding stone rotating on a Proxxon MF70 micro-milling machine . Next, the sharp tips were dipped in petroleum jelly to reduce friction. A foam sphere was then gently immobilized between two fingertips while being pierced (as close as possible through its center) by the sharp tip, which was gently rotated by the fingertips of the other hand . The oneAlthough these particles are large compared to traditional (non-X-ray-based) particles used in particle-tracking velocimetry, the volume of our 1.4-mm-diameter tracers is more than five times smaller than the smallest tracers used so far for X-ray-based particle tracking . In rela3.To estimate the volume of the food items used in the experiment, we laser-scanned 10 food items (Faro Laser ScanArm V2 system). We obtained a volume  of 24 ± 2 mm3.The food items implanted with radio-opaque markers were weighted to obtain their dry mass and were soaked in water for 15 min to estimate their wet mass. We obtained a density  of 1845 ± 170 kg/mhttps://www.xromm.org/xmalab/; terminal) of the tracer (tracer) using Stokes’ law ρtracwater corresponds to the dynamic viscosity of water – that is, 1.002 mPa·s at 20°C  density of the produced particles from the test sample (N = 19) was 994 ± 33 kg/m3 .where µ at 20°C  – g to t33 kg/m3 . Since o33 kg/m3 , the mea2) in a transient, laminar flow model using the 6 DOF dynamic mesh functionality to update the position of the spherical particle to the next time step (800 time steps of 0.125 ms solved with 40 iterations per time step). The inertial properties of spheres filled with uniformly dens material, as well as the hydrostatic lift force, were provided by user-defined functions. A second user-defined function set the profile of the pressure drop at the interior base plane of the cylinder (3) at two initial positions  of the current sample of particles  and density  set-up at the University of Antwerp  and extracted CT marker coordinates. We generated the rigid-body transformations in XMALab and imported them into Autodesk Maya to animate the bones. We used scientific rotoscoping  to positTo obtain a common framework between trials, we created an anatomical coordinate system (ACS) attached to the neurocranium. The origin of this ACS was offset to be positioned close to the location of the pharyngeal jaws as an approximation of the esophagus entrance location . The pharyngeal jaws are known to move during the food capture and transport e.g., , but we Among the 73 trials recorded, the entire buccal cavity and tracers were not always visible throughout the entire sequence; therefore, we focused our analysis on seven trials among the two individuals of carp and six trials among the two individuals of tilapia. For these trials, we imported the 3D trajectories of the food and water tracers into Maya . We computed the relative 3D trajectory and relative 3D velocity in the coordinate system previously described for the water and food tracers. We computed the mean and standard deviations of these parameters among trials within each species .To divide the suction-feeding sequences into phases, we used the anteroposterior trajectory and velocity of the water tracers . We exclThe 3D trajectory of all the water tracers during each suction-feeding sequence provided a visualization of the water tracer paths for the different views of a representative trial and of all the trials of a given species during the intake , the rf We registered the number of successful suction-feeding sequences among all the recorded trials . For this set of data, we noted the number of food and water tracers reaching the entrance of the esophagus that were effectively trapped in the digestive tube when the tracer was at the level of the pharyngeal jaws. We also measured the number of water tracer oscillations (backward and forward motions) before the food tracer reached the area of the pharyngeal jaw’s axis.In addition, we compared the global distance traveled by the food tracers and the water tracers. We computed the relative contribution of each component of the trajectory during the intake, the rf phases, and bf the phases among the studied trials .To quantify the deviation of the water tracers relative to the food tracers, we calculated the curvature of a given tracer based on three consecutive frames in the dorsoventral plane . To do so, we standardized the position of the tracers on the anteroposterior axis by calculating the x-trajectory percentage for each tracer, then we computed the curvature values over these intervals for the food tracers and the water tracers. As the distribution was not normal, we performed a Kruskal–Wallis rank-sum test to evaluate if the differences in the maximum curvature per trajectory of each interval were significantly different between the water tracers and the food tracers .We plotted the tracer coordinates in a dorsoventral plane  and assigned a color based on their location at 25% of the intake phase duration for all the tracers of all the trials of each individual . This prTo explore the relationship between the initial position of the tracers at the beginning of the first intake to their final position at the end of the first intake, we plotted the tracer coordinates  and assigned a color based on their initial location for all the tracers of all the trials of each individual . We testThe exclusion criteria were even more drastic for this part of the analysis as we needed not only the buccal cavity and tracers visible throughout the entire sequence, but also the bone markers. We had to withdraw the data for one individual of tilapia, consistently out of the field of view at the end of the first intake. Therefore, we ended up with two individuals of carp and seven trials, one individual of tilapia and three trials.We exported the 3D trajectories of additional locators we positioned on the anterior tip of the upper jaw, lower jaw, and hyoid, as well as on the posterior tip of the operculum on the animated Maya scenes . This al How do fish suck food underwater? Using new artificial food particles that are radio-opaque and naturally buoyant, Provini et al. imaged the roller-coaster ride that food particles make being sucked in from outside to inside the fish using 3D stereo high-speed fluoroscopy. The recordings show fishes to have an intriguing ability to generate flows that center the food particles as they enter the muscular tube that carries them from the mouth to the stomach. Remarkably, the flow patterns in the mouth that accomplish this seem to differ between the two species of fish studied, although samples sizes are very small at present and results should be interpreted cautiously. These new insights will be of interest to biologists working on suction-feeding mechanisms ranging from millimeter-sized carnivorous water plants, tadpoles, and fish larvae, to large fish and marine mammals, and even gigantic whales. Bioinspired engineers designing rapid underwater suction apparatuses may benefit from harnessing the new insights to elegantly center items of interest. public reviews designed to be posted alongside the preprint for the benefit of readers; (ii) feedback on the manuscript for the authors, including requests for revisions, shown below. We also include an acceptance summary that explains what the editors found interesting or important about the work.Our editorial process produces two outputs: (i) Decision letter after peer review:eLife. Your article has been reviewed by three peer reviewers, including David Lentink as Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Christian Rutz as the Senior Editor.Thank you for submitting your article "in vivo intraoral waterflow quantification reveals hidden mechanisms of suction feeding in fish" for consideration by The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted this decision letter to help you prepare a revised submission. Please carefully address the comments and suggestions, mark all changes in the revised manuscript in coloured font, and provide a point-by-point response. This will help the Reviewing Editor evaluate your revision. Please add line numbers in case another round of review is required.Essential revisions:Overall, the editors and reviewers all greatly appreciate the method used, the experiments performed, and the data visualization presented. This has persuaded us to give the authors the opportunity to resolve several weaknesses in a revised manuscript.– High-level feedback based on editorial discussioneLife. Hence, we believe the manuscript would be much improved if it succinctly addressed the following points.While this work on fish suction feeding is impressive and informative, it is currently too narrowly focused for a broad interest journal like (1) A broader introduction on suction feeding in its ecological and evolutionary context is needed, covering a wider range of suction feeding studies, from tadpoles, fish larvae and microscopic carnivorous waterplant suction feeding apparatuses up to those of the largest marine vertebrates . The point needs to be made more compellingly that, while suction feeding has been studied across scales externally to the degree possible, the internal fluid mechanics remain poorly understood at any of those scales due to the experimental challenges of imaging flow at high-speed internally. This requires a paragraph in the Introduction, to enable the general reader to understand why this research matters and what challenge was solved for which reason; especially, if the evolutionary context of "why suction feeding matters" is given in the broadest sense. For this the following review article may be helpful: Deban, Stephen M., Roi Holzman, and Ulrike K. Müller. "Suction feeding by small organisms: performance limits in larval vertebrates and carnivorous plants." Integrative and Comparative Biology 60.4 (2020): 852-863.(2) With this broader background in place, the authors have a solid jump-off point at the end of the Introduction (before they describe the methods used) to introduce the model species selected. This is likely based on factors such as their size matching the optimal bi-planar (stereo) fluoroscopy setup imaging volume.eLife, it would help to translate the biological implications to movement ecology or evolutionary biology or any other biological domains the authors believe will benefit from the new scientific insights. This section does not need to be speculative; it can be strongly connected to the actual findings, for example, on how they could inform other studies in which suction feeding is a key factor. For example, an effective broader impact closeout could include discussion of any scaling laws derived or confirmed based on the research, helping explain how these findings could advance our understanding of suction feeding across a wider range of species. Or, it could be a very specific and focused paragraph on the mechanism for the species tested written in a fashion that makes it more broadly accessible, with associated clear schematics connecting internal and external perspectives: a helicopter perspective completing our understanding of suction feeding (one that could go into a biology textbook). In conclusion, we encourage the authors to write a closing paragraph that is scientifically rigorous while serving a broader audience beyond the small suction feeding community. Some minor speculation is acceptable if the wording communicates this.(3) The text about the potential impact of the method itself in the last paragraph of the Discussion can stay , but it isn't an effective closing paragraph for a broad audience. In the Discussion, it would be good to conclude with a paragraph with a broader perspective, one that connects to the introductory biological context added under (1). Currently the last paragraph is very dense and technical, written for a specialist audience. To better serve the broad readership of – Actionable feedback summary from the peer review outcomes(4) Please clarify the number of species, individuals and trials for which data have been processed and integrated in the figure captions, Methods, Results and Discussion. The number of individuals per species (n = 2) is very small and this has been brought up as a concern. Another concern is that the number of individuals and trials analyzed seems to be variable throughout the manuscript, depending on the particular analyses made. To address this, we would like to see all data processed for all individuals and species reported in the manuscript. In the Methods, the rationale for using just 2 individuals, as compared to the more typical 3-4 as a minimum, is missing. In the Discussion, a few sentences should clarify how the low number of individuals may limit the interpretation and generalisability of the data. Given the total number of individuals across both species is 4, the overall conclusions supported across both species and both individuals are considered strongest. Finally, please keep in mind that the calculation of a biologically meaningful average and standard deviation representative for a species requires a minimum of three individuals. The major limitations remain  and impact the ability to conduct statistical analyses, inferences and conclusions that are biologically meaningful. Yet, this context is not sufficiently integrated in the presentation of the experimental design and discussion of the results.(5) The tracking of particles into the mouth should provide qualitative data that can highlight the hydrodynamics of the flows, but the speed of intra-oral flows is not fully examined. The manuscript is missing a quantitative estimate of the volume of water that passes near the esophagus, and of the kinematics of bone movements and how they correspond to fluid flows. Therefore, we would like to see quantitative mapping of the relationship between the distance from the mouth at the beginning of the strike to where the particle ends, and other quantitative measures. The authors also mention efficiency but make no attempt to quantify it. For the reviewers, it is the quantitative insight into these and other suction feeding parameters that has the potential to make this article particularly impactful.(6) The beautifully illustrated tracks are a strength of the manuscript, but also mask some mechanistic details we would like to see dissected. The tracks in the current plot format confound space and time, meaning that the particles potentially started at different distances under different angles from the mouth and may have also started to move substantially at different times. We therefore believe additional figures are needed to quantitatively analyze this. Quantification will also enable the authors to make statements such as "food items that are initially somewhere close to the mouth…" as specific as possible based on the dataset.(7) We are curious if the suction effort of the fish may have varied across the trials (videos). Such variation has been shown to be primal in determining the speed of the external flows, hence we believe it is worthy of inclusion in the analyses.(8) The manuscript somewhat oversells the study's novelty; please try to better delineate the contribution. (i) Briefly clarify the exact contribution of the new method in the context of existing literature; what existed and has been done, which aspects are added / new etc. For example, the water markers seem to represent a modified design of previously reported particle and food item tracking approaches. Neither the XROMM technique nor particle tracking nor neutrally buoyant particle tracking are entirely new; for example, tracking marked prey items (with radio opaque markers) into the mouth and esophagus using XROMM has been done previously. Both particle tracking and particle image correlation are established for a wide range of applications. XROMM is also established for 3D imaging and tracking in the body. The new advances should be presented in the context of the relevant experimental literature. (ii) Discuss previous CFD studies more extensively, and how the limitations of those studies precluded the insights achieved by the present study. Considering these internal flows were modeled extensively by the authors (others), there is a model and theoretical framework that predicts how the internal flow behaves. This rich CFD and literature context is missing in the presentation of Figure 1, the Introduction and Discussion.(9) Please discuss the tracking particle physical properties consistently and correctly: there is confusion about the particles being neutrally buoyant versus (page 16) negatively buoyant; this should be clarified throughout. The PIV literature provides a useful tracking error framework for relating the effect of particle size and density ratio to the ability of particles to track the flow. These established error analyses should be integrated into the Methods section in which the method is outlined. This should include estimated limitations due to particle size and buoyancy imperfections limiting the ability of the particles to trace out the flow pattern of interest. With this context the reader can fully comprehend the significant advance the current study achieves, within the acceptable limitations given the challenges involved in this tracking approach. The technical discussion paragraph could comment on how future tracing particles could/should be improved. Essential revisions:Overall, the editors and reviewers all greatly appreciate the method used, the experiments performed, and the data visualization presented. This has persuaded us to give the authors the opportunity to resolve several weaknesses in a revised manuscript.The authors want to thank the editors and reviewers for the valuable feedback they provided and for the opportunity to submit a revised version of the manuscript.– High-level feedback based on editorial discussionWhile this work on fish suction feeding is impressive and informative, it is currently too narrowly focused for a broad interest journal like eLife. Hence, we believe the manuscript would be much improved if it succinctly addressed the following points.(1) A broader introduction on suction feeding in its ecological and evolutionary context is needed, covering a wider range of suction feeding studies, from tadpoles, fish larvae and microscopic carnivorous waterplant suction feeding apparatuses up to those of the largest marine vertebrates . The point needs to be made more compellingly that, while suction feeding has been studied across scales externally to the degree possible, the internal fluid mechanics remain poorly understood at any of those scales due to the experimental challenges of imaging flow at high-speed internally. This requires a paragraph in the Introduction, to enable the general reader to understand why this research matters and what challenge was solved for which reason; especially, if the evolutionary context of "why suction feeding matters" is given in the broadest sense. For this the following review article may be helpful: Deban, Stephen M., Roi Holzman, and Ulrike K. Müller. "Suction feeding by small organisms: performance limits in larval vertebrates and carnivorous plants." Integrative and Comparative Biology 60.4 (2020): 852-863.We have added a paragraph in the introduction to broaden the scope of our work. We now mention other aquatic organisms, from larval fish to whales and carnivorous plants.We also cited the work mentioned in your comment as well as the following references:- Herrel, A., Van Wassenbergh, S. and Aerts, P. (2012). Biomechanical studies of food and diet selection. In: eLS, pp. 1-9. Chichester: Wiley.- Alexander, R. McN. (1969). Mechanics of the feeding action of a cyprinid fish. J. Zool., Lond. 159, 1-15.- Drost, M. R., Muller, M. and Osse, J. 1988 A quantitative hydrodynamical model of suction feeding in larval fishes: the role of frictional forces. Proc. R. Soc. B 234, 263–281. doi:10.1098/rspb.1988.0048- Deban SM, Olson WM. 2002. Suction feeding by a tiny predatory tadpole. Nature 420:41–42.Werth AJ. 2004. Functional morphology of the sperm whale  tongue, with reference to suction feeding. Aquat Mamm 30:405–418.- Müller, U.K., Berg, O., Schwaner, J.M., Brown, M.D., Li, G., Voesenek, C.J., van Leeuwen, J.L. (2020). Bladderworts, the smallest known suction feeders, generate inertia-dominated flows to capture prey. New Phytologist 228, 586-595.- Wainwright, P.C., McGee, M.D., Longo, S.J. and Hernandez, L.P. (2015) Origins, innovations, and diversification of suction feeding in vertebrates. Integr. Comp. Biol. 55, 134-145.- Werth, A. J. (2004). Functional morphology of the sperm whale  tongue, with reference to suction feeding. Aquatic Mammals 30, 405–418(2) With this broader background in place, the authors have a solid jump-off point at the end of the Introduction (before they describe the methods used) to introduce the model species selected. This is likely based on factors such as their size matching the optimal bi-planar (stereo) fluoroscopy setup imaging volume.We added a sentence at the end of the introduction to present the model species selected:“These species were chosen from the large diversity of suction feeders because their head size as adult is small enough to fit the imaging volume of the biplanar X-ray setup, yet large enough to avoid hindrance from an intake of tracer particles for waterflow inside the buccal cavity using X-rays.”(3) The text about the potential impact of the method itself in the last paragraph of the Discussion can stay , but it isn't an effective closing paragraph for a broad audience.We streamlined the last paragraph of the Discussion of the previous version of the manuscript.In the Discussion, it would be good to conclude with a paragraph with a broader perspective, one that connects to the introductory biological context added under (1). Currently the last paragraph is very dense and technical, written for a specialist audience. To better serve the broad readership of eLife, it would help to translate the biological implications to movement ecology or evolutionary biology or any other biological domains the authors believe will benefit from the new scientific insights. This section does not need to be speculative; it can be strongly connected to the actual findings, for example, on how they could inform other studies in which suction feeding is a key factor. For example, an effective broader impact closeout could include discussion of any scaling laws derived or confirmed based on the research, helping explain how these findings could advance our understanding of suction feeding across a wider range of species. Or, it could be a very specific and focused paragraph on the mechanism for the species tested written in a fashion that makes it more broadly accessible, with associated clear schematics connecting internal and external perspectives: a helicopter perspective completing our understanding of suction feeding (one that could go into a biology textbook). In conclusion, we encourage the authors to write a closing paragraph that is scientifically rigorous while serving a broader audience beyond the small suction feeding community. Some minor speculation is acceptable if the wording communicates this.We added a concluding paragraph that (1) summarizes the biological implications of our work, and (2) provides a future perspective of research following up on our work. We feel that it should be accessible to a broad audience. The newly discovered flow type was highlighted. It has the potential to become textbook knowledge on the basic functioning of suction feeding.– Actionable feedback summary from the peer review outcomes(4) Please clarify the number of species, individuals and trials for which data have been processed and integrated in the figure captions, Methods, Results and Discussion. The number of individuals per species (n = 2) is very small and this has been brought up as a concern.We have clarified the number of species, individuals and trials for which data have been processed and integrated in the figure captions, Methods, Results and Discussion.Another concern is that the number of individuals and trials analyzed seems to be variable throughout the manuscript, depending on the particular analyses made. To address this, we would like to see all data processed for all individuals and species reported in the manuscript.The number of individuals and trials were consistent throughout the hydrodynamic analysis , but we chose to only include the data of one individual per species in the Figure 3 and in the illustration Figure 4, to ease the readability. We added a figure in the supplementary material of Figure 3, to illustrate the equivalent of Figure 3 for the other individual of both carp and tilapia . We see that the results are consistent with what was reported in the previous version of the manuscript.For the Skeletal kinematics analysis, one individual of tilapia had to be dismissed because most of the bone markers were consistently out of the field of view at the end of the food intake. This was specified in the text.Please, note that all the data are available in the R code provided as a package in the supplementary material.In the Methods, the rationale for using just 2 individuals, as compared to the more typical 3-4 as a minimum, is missing. In the Discussion, a few sentences should clarify how the low number of individuals may limit the interpretation and generalisability of the data. Given the total number of individuals across both species is 4, the overall conclusions supported across both species and both individuals are considered strongest.A sentence was added to the Method section, explaining that two individuals per species are deemed sufficient, and therefore an appropriate choice when taking ethical considerations into account to reduce the number of experimental animals to a minimum (3R framework). Quantifying individual variability would require a higher number of individuals per species and is, therefore, not a goal of our study. The fact that both show the same result in the framework of the hypotheses from Figure 1 confirms this choice.We added a sentence mentioning this limitation, yet adding that the consistency of the results tends to strengthen our conclusions:“We reduced the number of experimental animals to a minimum for ethical considerations. As quantifying individual variability would require a higher number of individuals per species it was not a goal of our study, therefore intraspecific variation is not covered sufficiently by our analysis to allow generalizations within each species. Additionally, the limited number of species studied do not allow us to generalize our findings to other adult, omnivorous, teleost fish. Even if the number of analyzed individuals was limited to two per species, which may limit the interpretation of the data in the context of the STRANGE framework, the total number of individuals across both species , the number of trials , and the consistency of the results, imply that the mainly qualitative conclusions on overall patterns in water flow and skeletal motion are well supported.”Finally, please keep in mind that the calculation of a biologically meaningful average and standard deviation representative for a species requires a minimum of three individuals.We pooled the data from the two individuals of each species, but standard deviations are “among trials within species” and not “among individuals”. We made that clearer in the manuscript.The major limitations remain  and impact the ability to conduct statistical analyses, inferences and conclusions that are biologically meaningful. Yet, this context is not sufficiently integrated in the presentation of the experimental design and discussion of the results.We added a sentence in the Material and Methods to highlight the small number of individuals per species and the potential limitations it can cause in terms of generalizability of the results.(5) The tracking of particles into the mouth should provide qualitative data that can highlight the hydrodynamics of the flows, but the speed of intra-oral flows is not fully examined. The manuscript is missing a quantitative estimate of the volume of water that passes near the esophagus, and of the kinematics of bone movements and how they correspond to fluid flows.To quantify the volume of engulfed water that passes by the pharyngeal jaws, unfortunately, we would need more particles distributed over a larger volume in front of the mouth. Currently, the particles are distributed over a circular area surrounding the food at the bottom of the aquarium. This does not allow a quantification of the full water volume entering the mouth, and therefore also not the volume of water passing near the oesophagus. Future studies may further optimise our protocol to get a more complete view of what happens intra-orally with the entire water parcel entering the mouth. This will, however, require X-ray-visible particles that are in suspension for a long time, which would be technically very challenging to build.Therefore, we would like to see quantitative mapping of the relationship between the distance from the mouth at the beginning of the strike to where the particle ends, and other quantitative measures. The authors also mention efficiency but make no attempt to quantify it. For the reviewers, it is the quantitative insight into these and other suction feeding parameters that has the potential to make this article particularly impactful.We added a figure in the supplementary material , showing the tracer trajectories, coloured according to their initial position. Generally, the particles tend to stay on the same side as their initial location, but there can be a change of side before the particles arrive close to the oesophagus. A Kendall test of correlation revealed a significant correlation in carp but not in tilapia .As we did not find a solution to clearly define a metric of efficiency for suction feeding that can be quantified with our data, we no longer mention it.(6) The beautifully illustrated tracks are a strength of the manuscript, but also mask some mechanistic details we would like to see dissected. The tracks in the current plot format confound space and time, meaning that the particles potentially started at different distances under different angles from the mouth and may have also started to move substantially at different times. We therefore believe additional figures are needed to quantitatively analyze this. Quantification will also enable the authors to make statements such as "food items that are initially somewhere close to the mouth…" as specific as possible based on the dataset.Indeed, as the number of particles, their initial location relative to the individual, and the fish behaviour  and anatomy  differed between trials and individuals, we chose to use a representation of the data using space instead of time, as a standardization technique.We added a Figure showing the particle distribution for each trial of each individual, based on the moment they start to move toward the buccal cavity . We found no correlation between the moment of ingestion and the distribution of the tracers in the buccal cavity. However, visually, it seems that early-sucked particles tend to stay in the medial plane of the buccal cavity, participating to the central jet, whereas the late ones tend to end up on the lateral parts of the buccal cavity.(7) We are curious if the suction effort of the fish may have varied across the trials (videos). Such variation has been shown to be primal in determining the speed of the external flows, hence we believe it is worthy of inclusion in the analyses.Considering our effort to standardize the set-up  our goal was to minimize the variation between individuals and trials. Therefore, it would not be relevant to quantify the variability in term of suction effort in the context of this study. We agree, however, that further studies using the same methods should try to quantify this.Note that since we are aware that flow velocities are depending on suction effort, which depends on the type of food that is offered, we focus on the spatial patterns in the flow in the light of the hypothesis that were put forward in Figure 1, and not on the velocity magnitudes.(8) The manuscript somewhat oversells the study's novelty; please try to better delineate the contribution. (i) Briefly clarify the exact contribution of the new method in the context of existing literature; what existed and has been done, which aspects are added / new etc. For example, the water markers seem to represent a modified design of previously reported particle and food item tracking approaches. Neither the XROMM technique nor particle tracking nor neutrally buoyant particle tracking are entirely new; for example, tracking marked prey items (with radio opaque markers) into the mouth and esophagus using XROMM has been done previously. Both particle tracking and particle image correlation are established for a wide range of applications. XROMM is also established for 3D imaging and tracking in the body. The new advances should be presented in the context of the relevant experimental literature.We agree this was confusing in the original version of the manuscript, especially in the sentence from the introduction. We rewrote it to:“Here, we develop a new technique based on biplanar high-speed X-ray videography to quantify the 3D pathlines of intraoral water and combine it with existing methods to track food and quantify 3D skeletal motions.”This should clearly separate the new contribution from the existing methods and make it clear that we see the tracking of water as a separate method in addition to the tracking of food. The latter has indeed been done previously in many 2D x-ray studies, and in more recent 3D biplanar X-ray studies. The revised paragraph in the discussion now reverts back to the x-ray particle tracking protocols that have been used in industrial settings .(ii) Discuss previous CFD studies more extensively, and how the limitations of those studies precluded the insights achieved by the present study. Considering these internal flows were modelled extensively by the authors (others), there is a model and theoretical framework that predicts how the internal flow behaves. This rich CFD and literature context is missing in the presentation of Figure 1, the Introduction and Discussion.We added a paragraph in the Discussion, a new figure , and a supplementary video  summarizing the results from previous CFD studies as requested. Note, however, that to our knowledge, 3D CFD models including an outflow of water through the gill slits do not exist currently. Consequently, to study patterns of intra-oral flow, the CFD literature is not rich. We included results from the currently most advanced, dynamically expanding 3D model of the head of a sunperch from Thompson et al., 2018 in Figure 6a. We clearly noted the limitations of this work . In Figure 6b, the non-expanding model from Provini and Van Wassenbergh 2018 is presented. Both CFD studies are discussed in the light of the current experimental results.(9) Please discuss the tracking particle physical properties consistently and correctly: there is confusion about the particles being neutrally buoyant versus (page 16) negatively buoyant; this should be clarified throughout. The PIV literature provides a useful tracking error framework for relating the effect of particle size and density ratio to the ability of particles to track the flow. These established error analyses should be integrated into the Methods section in which the method is outlined. This should include estimated limitations due to particle size and buoyancy imperfections limiting the ability of the particles to trace out the flow pattern of interest. With this context the reader can fully comprehend the significant advance the current study achieves, within the acceptable limitations given the challenges involved in this tracking approach. The technical discussion paragraph could comment on how future tracing particles could/should be improved.We maintain our statement concerning the negatively buoyant particles, in the Material and Methods section of the previous version of the manuscript. We measured the density of 19 particles (figure 7d), but the particles we used in the experiment were the ones that could remain at the bottom of the water tank, waiting for the fish to swallow them. By definition they were negatively buoyant.We had previously evaluated the effect of size and density on tracking performance of our particles using CFD but have now worked this out in detail and added this CFD analysis to the manuscript. The results are presented in a new Figure , which is further illustrated by a video . It serves to quantify the limitations due to particle size and buoyancy imperfection.The CFD results reassure that the finite size (1.4 mm diameter) and density (up to about 1050 kg m-3) of the current sample of particles  does not hinder a realistic assessment of the suction flows by fish. A small deviation of the path in the direction of gravity can be expected , but this should be smaller than 1 mm even for the heaviest particles of 1050 kg m-3. Lag during flow acceleration and overshoot during deceleration for the slightly negatively buoyant particles was relatively small  and therefore acceptable to describe general patterns of flow during suction feeding.As suggested, we added a discussion on how to improve future tracer particles in terms of achieving less variability in density. Miniaturising the manufacturing protocol by Drake et al., 2011 has this potential, in theory.


Spoofing attacks are one of the severest threats for global navigation satellite systems (GNSSs). This kind of attack can damage the navigation systems of unmanned air vehicles (UAVs) and other unmanned vehicles (UVs), which are highly dependent on GNSSs. A novel method for GNSS spoofing detection based on a coupled visual/inertial/GNSS positioning algorithm is proposed in this paper. Visual inertial odometry (VIO) has high accuracy for state estimation in the short term and is a good supplement for GNSSs. Coupled VIO/GNSS navigation systems are, unfortunately, also vulnerable when the GNSS is subject to spoofing attacks. The method proposed in this article involves monitoring the deviation between the VIO and GNSS under an optimization framework. A modified Chi-square test triggers the spoofing alarm when the detection factors become abnormal. After spoofing detection, the optimal estimation algorithm is modified to prevent it being deceived by the spoofed GNSS data and to enable it to carry on positioning. The performance of the proposed spoofing detection method is evaluated through a real-world visual/inertial/GNSS dataset and a real GNSS spoofing attack experiment. The results indicate that the proposed method works well even when the deviation caused by spoofing is small, which proves the efficiency of the method. Global navigation satellite systems (GNSSs) are some of the most important absolute positioning infrastructures, especially for unmanned air vehicles (UAVs) and other unmanned vehicles (UVs). GNSSs can provide precise position information for long periods of time, and the cost of the navigation receiver as a user-end device is low, which greatly expands the applications of GNSSs .Although the GNSS has incomparable advantage, its defects also should not be ignored. GNSS navigation relies on navigation signals transmitted by satellites from Earth orbits. The transmission power of the navigation signal is limited by the volume, mass, and payload of the satellite, and the signal attenuation after passing through the ionosphere, troposphere, and atmosphere is extremely serious. The receiver is vulnerable to interference or attack. The safety of GNSSs has attracted more and more concern in recent years. GPS spoofing is one of the most important threats affecting the positioning process in GPS receivers .There are two main types of GNSS interference: suppression (jamming) interference and spoofing interference. Compared with suppression interference, spoofing interference is more concealed and harmful, and anti-spoofing interference methods are more complicated. The purpose of spoofing interference is to deceive the receiver. The spoofing signals are usually created by imitation satellite navigation signals or delayed real navigation signals. The characteristics of the spoofing signals, like phase, frequency, Doppler frequency shift, and others, can be similar to the real signals when the spoofer has been synchronized with real signals. The signal-to-noise ratio (SNR) of the spoofing signal can be set slightly higher than that of the real one, so the receiver can easily lock the spoofing signals and output incorrect positioning solutions. Most UVs and UAVs utilize GNSSs as the only absolute positioning method, so spoofing attacks can damage the whole navigation system without the user being aware. To handle this kind of threat, the application of anti-spoofing technology in modern GNSSs is acquiring more and more attention .In December 2011, Iran announced the capture of a state-of-the-art US RQ-170 “Sentinel” stealth drone. According to an Iranian engineer who claimed to have participated in the capture mission , they deSpoofing detection methods are generally divided into three categories according to the different detection principles. The first kind is based on detecting abnormal changes in signal characteristics, such the carrier-to-noise ratio , signal The second kind of detection method is based on the encrypted information in the signal. Previous authors ,14,15 haThe third kind of spoofing detection method is based on utilizing supplementary measurements from other navigation or positioning sensors of the system. One previous study  mentioneThe purpose of a spoofing attack is to mislead the navigation system by inducing the system to estimate an incorrect positioning result. The supplementary measurement methods focus on monitoring the abnormal deviation and the credibility of positioning solutions, highlighting the most important point of the spoofing detection problem. Although the IMU method is used most extensively, it also has some disadvantages restricting its detection performance in some spoofing scenarios. Due to the serious cumulative error, the IMU only has good short-term accuracy. Over time, the positioning solution output of the IMU develops a large deviation. This restricts performance in the detection of induced spoofing signals , which dThe IMU and cameras are two of the most widely used sensors for UVs and UAVs. Spoofing detection methods based on visual measurements have also been introduced. Qiao  proposedIn order to improve the performance of the aided spoofing detection method using supplementary measurements and protect visual/inertial/GNSS integrated systems from GNSS spoofing, a novel spoofing detection method based on a coupled visual/inertial/GNSS algorithm for UVs/UAVs is proposed in this paper. The coupled visual/inertial/GNSS positioning algorithm has been widely discussed in simultaneous localization and mapping (SLAM) and other fields. In this paper, a well-known, open-source, optimization-based coupled VIO/GNSS positioning algorithm VINS-Fusion is referred to. The KITTI dataset  is used A novel spoofing detection method using visual/inertial estimations as supplementary measurements is proposed, with the spoofing detection factor calculated by residuals and the error covariance matrix;The spoofing detection and the coupled visual/inertial/GNSS positioning estimation are both considered as part of the coupled positioning system. The vulnerability of the coupled visual/inertial/GNSS algorithms under spoofing attacks is discussed;The least squares solution for GNSS measurement under spoofing interference is analyzed in order to represent the mechanism of spoofing errors. Suddenly changed and induced spoofing are discussed and modeled;The experiments for the proposed spoofing detection method and the modified coupled positioning algorithm were conducted both with a dataset and real GNSS spoofing scenarios. The effectiveness of the detection and the superiority of the modified coupled positioning algorithm under GNSS spoofing attacks are shown.The main contributions of this paper can be summarized as follows:The rest of the paper is organized as follows. In The GNSS receiver locks and processes positioning signals transmitted by GNSS satellites. It calculates its own position from the pseudo-range measurements and positions of multiple satellites. The pseudo-range measurement can be expressed as:Equation , the pseWhen solving the least squares solution of Equation (3), there are errors caused by spoofing included in the solution as follows:The GNSS receiver uses the wrong pseudo-range measurements to calculate incorrect positions and clock errors as follow:Depending on the different types and purposes of spoofing interference, For most spoofing attack scenarios, high-power spoofing signals, aligned with real signals, gradually take the place of the real signals in the GNSS receiver. When the spoofer believes that the spoofing signals have already “taken over” the GNSS receiver, the next step in the spoofing procedure is undertaken. Two different kinds of spoofing split from here. In the first, a significant deviation is immediately added into the GNSS measurement, and it looks like a step response when The performance of GNSS spoofing detection based on visual/inertial supplementary measurements is highly dependent on the precision of the visual/inertial positioning solution. The visual/inertial fusion algorithm is often used in the field of simultaneous positioning and mapping (SLAM). The filtering-based method and optimization-based method are two of the most commonly used methods. The filtering-based method only uses the current measurements to estimate positioning results and poses. It does not have an awareness of the whole positioning course. The optimization-based method can utilize measurements from a long period, so it has the advantage of higher accuracy. The nonlinear optimization method was selected for this paper. Based on previous work the literature ,26, a noIn a sliding window containing Acquiring solutions from the VIO is a process of bundle adjustment . In everSince the GNSS positioning measurement and the estimation of the VIO have already been calculated, the output of the coupled navigation system is a combination of these two kinds of measurements. In this condition, GNSS spoofing is not a concern and the navigation system cannot distinguish spoofed GNSS measurements from integral GNSS measurements. The nonlinear optimization method is also used to construct the coupled navigation algorithm. The parameter to be estimated is a set of global estimations essed as :(13)zkl−The cost function of the navigation algorithm can be constructed from the local factor and GNSS factor as follows :(15)ψ*=aThe solution of the coupled positioning algorithm is a graph-based optimization problem. This kind of optimization is a maximum likelihood estimation (MLE) problem, which consists of joint probability distributions of poses . CoupledHowever, in spoofing interference circumstances, if the GNSS receiver locks the spoofing signals and the spoofing error The GNSS spoofing detection method proposed in this paper is of the same type as the supplementary measurement methods introduced in Chi-square  includes two separated parts. The residual between the GNSS measurement and the VIO estimation In The spoofing deviation By providing an evaluation of GNSS quality from spoofing detection, the GNSS confidence factor When the result of hypothesis verification is 0, it indicates that the GNSS data is normal and the optimal estimation of GNSS and VIO can be operated normally. However, if the result of hypothesis verification is 1, there is a high probability that the GNSS has been spoofed and the GNSS confidence factor To evaluate the performances of the proposed spoofing detection method and the VIO/GNSS coupled algorithm under spoofing circumstances, both validation with an existing dataset and experiments under real spoofing scenarios were carried out. For the dataset experiment, the KITTI  dataset The KITTI 0_03_drive_0027 sequence includes 4504 pairs of pictures and the same number of GPS positioning results. The coupled visual/IMU/GNSS positioning algorithm was operated on a Robot Operating System (ROS) on an Ubuntu 16.04 system to simulate the real-time property. To figure out the accuracy level of the VIO, an estimation experiment without GPS positioning results was carried out. The coordinate system of the VIO is a body coordinate system that does not acknowledge the relative relationship to world coordinates. Therefore, an initialization of the VIO to align it with the world coordinates must be done. The authentic GPS positioning results can be treated as the initial guesses of the positioning solutions Another modification of the GPS positioning results should be undertaken to simulate the spoofed GPS positioning results. As analyzed in To simulate this most difficult condition of inducing spoofing, the spoofed GPS positioning results were set to deviate from the real position by several meters per second and this was increased with the time. These spoofed GPS positioning results can function as a good and appropriate simulation of the GPS positioning results under strategic inducing spoofing interference.The entire coupled visual/IMU/GNSS positioning algorithm was operated in ROS and fed these spoofed GPS positioning results and the corresponding visual pictures to simulate the estimation in the real world and in real time, analyzing the positioning results of the algorithm. x-axis increases during the spoofing period and finally reaches about 200 m. As for the deviation injection rates of 2 m/s and 3 m/s, as shown in The positioning results of the coupled algorithm indicated that the deviation from the spoofed GPS positioning results had a great influence on the optimal estimation. The spoofed GPS positioning results successfully induced the positioning results to the wrong place. According to the positioning results with the spoofed GPS positioning results, the coupled VIO/GPS algorithm obviously becomes fatally corrupted. This is because of the unconditional trust in the GPS positioning results. The influence of the spoofing deviation is significant enough to lead the positioning estimation hundreds of meters away in several minutes. This conclusion is applicable to most of the coupled visual/IMU/GNSS algorithms in SLAM and the visual navigation field that have no robustness to inducing spoofing attacks. It is useful to deploy spoofing detection to stop the coupled visual/IMU/GNSS algorithms being deceived by inducing spoofing attacks.The GNSS spoofing detection program introduced in First, a verification of the online spoofing detection was implemented. The detection part and the optimal estimation part of the coupled positioning algorithm were run in parallel. The detection program output results individually and did not influence the estimation of the coupled positioning algorithm. The positioning estimation had undoubtedly been spoofed, just like in The frequency of the GPS positioning results was downsampled from 10 Hz to 2 Hz and the frequency of the visual pictures was maintained as 10 Hz to simulate the real situation. The optimal estimation of the coupled positioning algorithm was activated every 2 s to balance the computation burden and the positioning performance. Before each optimal estimation, the spoofing detection procedure was activated using the last 10 groups of VIO estimations and the GPS positioning results. However, the detection program and the positioning estimation were totally independent. This indicated that the online spoofing detection program could successfully trigger an alarm during inducing spoofing.Equation (16) can be used to execute the optimal estimation. The GNSS confidence factor Compared with the ground truth, the deceptive influence of the spoofed GPS positioning results is clear in the huge deviation of the blue line in When the deviation injection rate of the inducing spoofing was reduced to 1 m/s, which meant that it was more deceptive and harder to detect, a similar result was obtained, as seen in A comparison of the positioning errors of the coupled GPS/VIO algorithm with spoofing detection and the coupled GPS/VIO algorithm without detection is shown in To evaluate the spoofing detection algorithm in real GNSS spoofing circumstances, a GNSS signal generator was used to generate spoofing signals. This kind of GNSS signal generator is specially made for experimental purposes as a spoofer. The spoofer is capable of receiving real GNSS signals and aligning the parameters of the spoofing signals to the real signals. The spoofing signals are too similar to the real ones to be distinguished by the receiver. The spoofer can transmit the spoofing signals with a SNR slightly higher than the real one. Under these conditions, some real low SNR signals that cannot be tracked by the receiver can be replaced by the similar spoofing ones. Further, the locked signals can also be replaced by the spoofing ones because of the strategy of searching for the highest correlation peak. Therefore, even if the receiver has already locked the real GNSS signal, the spoofing signal can easily take over the receiver once the spoofer has started to transmit spoofing signals. An integrated positioning system with cameras, an IMU, a GNSS receiver, and a microcomputer was set up. The real-time coupled visual/IMU/GNSS positioning algorithm and the spoofing detection algorithm were operated on this platform. The spoofer and the integrated positioning system are shown in In this experiment, the integrated positioning system remained stationary. The GNSS receiver locked real GNSS signals at the beginning with no spoofing signals. The output of the GNSS receiver initialized the coupled visual/IMU/GNSS positioning algorithm. The positioning solutions of the coupled algorithm would obviously stay close to the origin point with Gaussian noise when no spoofing signals existed. The positioning results and spoofing detection results were calculated in real-time in ROS. The spoofer was set to generate signals according to a certain strategy after aligning with the real signals. This meant that the positioning results calculated from the pseudo-range measurements and ephemeris from the spoofing signals would follow a certain trajectory. In this experiment, the spoofing signals were set to lead to a trajectory of uniform linear motion or uniform circular motion. To show the consequences of the spoofing attack, the spoofing detection module was not operated at first. The coupled algorithm estimates positioning solutions following Equation (15). The trajectory estimated by the coupled positioning algorithm in the real-time system under spoofing attack is shown in As shown in As a comparison, the spoofing detection module designed with Equation (21) was initiated to execute the optimal estimation with the spoofing detection algorithm in the next experiment. As explained in Equations (21)–(24), the GNSS confidence factor When spoofing signals led to a trajectory of uniform circular motion, the spoofing position periodically returned to close to the authentic position. After the spoofing detection module started to work, the GNSS confidence factor GNSS spoofing has been proven to be one of the most dangerous threats to navigation systems. If they are unaware of the existence of GNSS spoofing interference, coupled navigation systems will be corrupted by the damage caused by spoofed GNSS data. In this paper, a spoofing detection method based on a coupled visual/inertial/GNSS positioning algorithm to alert the integrated system of spoofing attack was proposed. The least squares solution of the GNSS measurement under spoofing interference was analyzed to represent the mechanism of the spoofing error. To achieve high-level positioning accuracy, an open-source, optimization-based, coupled VIO/GNSS positioning algorithm was introduced. Then, the damage caused by the spoofed GNSS measurements to the coupled positioning algorithm was analyzed. A modification through the addition of a GNSS confidence factor was made to adapt to the spoofing scenario.The spoofing detection method proposed in this paper was based on Chi-square detection. The residuals between the GNSS measurements, VIO estimations, and real position were analyzed separately. Spoofing detection statistics were constructed from the residuals and error covariance matrixes.The KITTI dataset was introduced and modified to simulate a spoofing scenario in this paper. The operating results of the coupled positioning algorithm with the spoofed GPS positioning results indicated that the positioning estimation had been deceived about the authentic location with a spoofed location. The experiment involving the coupled positioning algorithm with spoofing detection indicated that, even with a deviation injection rate in inducing spoofing as small as 1 m/s, the spoofing detection program worked and prevented the positioning result from being induced to a deceptive position. Similar results were found when the deviation injection rate was increased. The positioning errors were compared in two conditions, with and without spoofing detection, and it was found that the former was constrained and at least an order of magnitude smaller than the latter.The real spoofing experiment was undertaken using a GNSS spoofer. A spoofing attack was executed after the GNSS receiver of the integrated positioning platform had already locked real GNSS signals. The positioning results of the coupled algorithm were induced to a certain deceptive position or moved following the spoofing trajectory. The spoofing detection algorithm and spoofing detection module were verified to be good supplements to the coupled positioning algorithm against spoofing attacks.In the results from this paper, the vulnerability of the coupled visual/inertial/GNSS system under inducing spoofing interference was apparent. The spoofing detection method and the modification made to the coupled positioning algorithm proposed in this paper were able settle this problem up to a point. This system could potentially be used in any coupled navigation system that is equipped with a camera, IMU, and GNSS receiver, such as most UVs and UAVs.


Professional cleaning and safe social distance monitoring are often considered as demanding, time-consuming, repetitive, and labor-intensive tasks with the risk of getting exposed to the virus. Safe social distance monitoring and cleaning are emerging problems solved through robotics solutions. This research aims to develop a safe social distance surveillance system on an intra-reconfigurable robot with a multi-robot cleaning system for large population environments, like office buildings, hospitals, or shopping malls. We propose an adaptive multi-robot cleaning strategy based on zig-zag-based coverage path planning that works in synergy with the human interaction heat map generated by safe social distance monitoring systems. We further validate the proposed adaptive velocity model’s efficiency for the multi-robot cleaning systems regarding time consumption and energy saved. The proposed method using sigmoid-based non-linear function has shown superior performance with 14.1 percent faster and energy consumption of 11.8 percent less than conventional cleaning methods. The rampant COVID-19 has brought global crisis worldwide by affecting almost all the continents, infecting more than 103 million people and 2.2 million death reports as of 31 January 2021. With the ongoing COVID-19 pandemic situation, following safe social distancing norms, frequent cleaning, and sterilization of the environment has become an indispensable safety measure to mitigate the spread of the virus. The current pandemic situation has already shown numerous instances of how service robots are being used to help and control the virus’s spread [Over the past few decades, numerous researches have been done in the development of service robots. Robots have been applied in the domestic and industrial workspaces, such as collaborative robots robotic painting, robotic welding, material removal, part transfer, and machine tending. Recently, the robots, such as AGV and drones, have made great strides in fighting the COVID-19 pandemic . these rIn cleaning applications, area coverage is seen as one of the emerging problems, and exploration for coverage path planning has been studied in depth under various aspects: design, control, autonomy, perception techniques, and multi-robot strategies. For instance, in the design aspects of cleaning robots, Wu et al., in Reference , presentTo overcome speed constraints and control complexity, various types of locomotion strategies have been proposed. For instance, Gao et al.  presenteAnother critical aspect of cleaning robots is to understand the surroundings and build synergy with their surroundings. So, perception technique is an important aspect that determines the robustness and overall efficiency of the robot. Maryam et al.  proposedWhile AI and perception techniques are popular in many domains and are an essential aspect to improve the efficiency of the cleaning system, adapting to a multi-robot system is another critical aspect in cleaning robots to improve productivity with fast cleaning approach. Luo et al.  proposedIn the COVID-19 pandemic situation, other than frequent cleaning and sanitation of the environment, following safe social distancing measures is also crucial to slow down the virus’s spread. According to the Centre for disease control (CDC) for social distancing measures, a physical distance of at least 6ft is necessary to be maintained between each individual . A surveFor instance, Reference  presenteGiven a large number of applications in the domain of cleaning and safe social distancing surveillance, existing robotic innovations and technologies provide a wide range of solutions to improve productivity and quality. Even though the works mentioned above discuss various aspects of safe distancing surveillance and cleaning robots to improve productivity and quality, they still are primarily limited with performance constraints due to their application-specific design and their capability to handle one specific application. This militates to the use of the different robots for each application. The major hurdles in using individual robots to perform these tasks are that each robot requires expensive and multiple sensors, communication frameworks, navigation hardware, and high-end processors installed throughout the facility. To this end, this work mainly focuses on the development of an intra-reconfigurable robot WASP with a virtually connected multi-robot cleaning system. In this work, we present an adaptive cleaning method for the virtually connected multi-robot cleaning system. The cleaning system works in synergy with the WASP platform’s surveillance module. The surveillance system’s role is to monitor and identify the contaminated regions based on human interaction levels and generate a zig-zag-based coverage path along with motion commands based on adaptive velocity behavior models. Thus, the multi-robot systems proposed in this approach do not require high computational resources and high-cost intuitive sensors for perception. The proposed adaptive cleaning methodology algorithm is simulated in a MATLAB environment based on the surveillance system’s actual test data. In this work, we evaluate the proposed cleaning strategy’s efficiency in terms of energy and time consumption. This paper is organized as follows: Logistics transportation of different kinds of trolleys that are available in hospital settings.Safe distancing surveillance system to monitor the human interactions inside hospital environment to control the spread of the virus.Efficient cleaning and disinfection in the hospital environment.In this paper, we demonstrate the concept of multi-robot cleaning system on the Intra-reconfigurable logistic platform WASP, as seen in The intra-reconfigurability of the robot aims to handle logistics carrying operations and safe distancing surveillance in hospitals, wherein the robot is capable of switching between trolley lifting, trolley towing, and human safe distance monitoring applications. The multi-robot system acts as a virtual extension to the WASP robot to handle cleaning tasks based on the inputs from the human safe distance surveillance system. One of the significant advantages of the WASP design is its intra-reconfigurability along with the multi-robot system, which unifies the model of the robot. Therefore, instead of having different communication frameworks and repetitive use of the same hardware units on different robots’ models, this design requires fewer resources.a and b are the distance from the robot’s center in the corresponding x and r is the radius of each wheel.The locomotion unit of WASP robot is a four-wheeled mechanism drive system where a series of free moving rollers are attached about the perimeter of the wheels with an angle of 45 degrees. This meccanum wheel design enables the robot to perform motion in eference , the forAfter calculating the velocity of the robot based on inverse kinematics equations, Equations –10) are are10) aThe estimated position and velocity of the robot are published in ROS topic as wheel odometry messages.In this paper, we propose the design idea of using the multi-robot system to handle cleaning and disinfection tasks. In contrast with the fixed cleaning strategies in the works proposed on the multi-robot cleaning systems ,33,47,48In this approach, an adaptive cleaning strategy is developed to improve the cleaning and disinfection performance by maximizing area coverage, able to identify contaminated regions in the environment, and clean the environment by spending more time and increasing the power of cleaning modules on the contaminated regions. The control commands for the cleaning system are generated by the WASP platform after processing the heat map generated from the surveillance module. The control model for the adaptive cleaning is generated by adapting to zig-zag-based coverage path planning for maximum area coverage. An adaptive velocity-based behavior model is integrated along with the path generation to enhance the cleaning performance and efficiency by reducing cleaning time and energy consumption. The WASP robot’s ROS framework establishes a virtual wireless communication with the cleaning subsystems over the ROS server through wifi. The control signals generated from the WASP platform are published in a ROS node and exported to the cleaning subsystem to perform the tasks through ROS client-server. The cleaning system sends the position feedback to the WASP robot’s surveillance system to update the position on the map.WASP is an Intra-reconfigurable robot designed to handle logistics, safe distance surveillance, and cleaning applications. The platform’s software system is categorized into autonomous navigation, surveillance system, and multi-robot cleaning system, as shown in For the autonomous navigation of the platform, the robot is equipped with a 2D laser scan range sensor, RGB-depth camera, and an Industrial grade IMU sensor. Robot localization is one of the critical things required for stable and effective autonomous navigation where the pose of the robot is estimated by the sensor fusion from wheel odometry and IMU data through an extended Kalman filter. For dynamic and static obstacle avoidance, the robot uses both local and global planners. While navigating, the robot generates a global path to reach the goal position, and a local path is generated to avoid dynamic obstacles in the environment by aligning towards the global path. For the safe distancing surveillance system, an intel realsense RGB-depth camera is manually connected on top of the WASP robot with the camera fixed at the height of 150 cm above the ground level. The camera has a field of view of 120 degrees and is used for human detection and tracking to monitor safe distancing. Based on the pose estimates of detected humans and the overlapping of each human’s personal space zones, human interaction levels are measured and plotted as a heat map on the grid map.For the case of a multi-robot cleaning system, the heat map generated from the WASP robot’s surveillance module is processed to generate a motion trajectory. The objective of the proposed framework for human detection and monitoring human interaction levels is to ensure safe social distancing norms in public places to control the spread of the virus during pandemic situations, especially with the ongoing COVID-19. As the WASP robot has the intra-reconfigurable capability of lifting and towing the trolleys for logistics applications and handle safe social distancing surveillance in large spaces, like hospital settings, good and high-quality perception and navigation sensors are required. Therefore, the robot is mounted with a RealSense D435 RGB-depth sensor as the perception unit and Sick 2D lidars to collect the data from surroundings. In this work, we present a vision-based perceptive framework for safe social distancing surveillance, as shown in As surveillance is one of the keys that focuses on the software module used on this robotic platform, we use visual feedback from an RGB-Depth sensor to detect and estimate the human’s pose and position in the environment. The vision sensor publishes raw color image topics and depth information through the ROS node. We further extend our software framework with state-of-the-art external perception library OpenPose  as an unTo measure interaction levels between humans, estimation of human attributes, like shape and size, along with their position and orientation in the environment is crucial. Oppose library gives 25 joints positions of the body in 3 dimension array. The first two columns are the X and Y joint positions, while the third one is detection accuracy. We further filter out the false detections based on the threshold accuracy for each detected joint. In this work, we use the detected human’s joint positions to measure the geometrical attributes of a human. For example, the detected human width can be calculated by measuring the distance between right and left shoulders from the skeleton joint array. Another essential information required to measure the human interaction levels is to know the detected humans’ position and orientation. The raw image frame from the RGB-d camera node, which is used as input for Openpose, provides 2d image frames of detected humans with coordinates of the detected humans. However, the coordinates  estimated are concerning the pixel coordinates and changes within the space of the image frame. So, the pixel coordinates from the raw color image frame further need to be transformed to coordinates in the world frame for estimating human positions in the map of the robot environment.I. The extrinsic parameters of the RealSense camera refer to the estimation of orientation that includes translation vector K and a rotation vector Q of the camera with respect to world coordinates. The extrinsic matrix S can be derived from p and the depth value z, we can convert the pixel coordinates from the image plane to real-world coordinates W using the Equation d=∑Equation .(12)p=I package , the traz and x coordinate positions of both the right and left shoulders of the detected human. Using the Equation  plane.f=exAlgorithm 1: Asymmetric Gaussian-based-heat map generationThe occupancy grid map of the environment is discretized into grid cells with each cell of size 0.05 × 0.05 m. Each cell is assumed to have values ranging from 0 to 1 based on the Asymmetric Gaussian distribution model for human density and interaction levels. The map in the figure shows the environment’s structure where every black pixel shows the space occupied by obstacles, and the white pixel represents the free space in the environment. The other colors in the grid cells show the human-occupied space with the intensity of the distribution ranging from 0 to 1. The maximum peak value of the distribution is represented with the red pixel, and the minimum value in the distribution is represented with the blue pixel on the grid-based heat map. So, the distribution intensity is represented as a variation in the heat map color. For the case of multiple people standing closer, the distribution values will be accumulated over the grid space and are normalized. So, the regions that are overlapped by the spatial zones of multiple persons will have a peak value of the grid cells, representing an area of highly contaminated space. The tracking id of the detected human is stored in the memory. While navigating, if the robot detects a person, an id will be created for the person, and if the same id is detected again on the same position of the grid space, the heat distribution is not generated. The new distribution is generated if the new id appears or the tracked id moves to different grid cells. The generated distributions are accumulated on the grid-based heat map. The heat map density on the distribution is estimated based on the level of interactions between humans.The WASP platform, being the brain/master unit of this system, processes the heat map generated from the surveillance module to generate an adaptive velocity behavior model based on a zig-zag path planner for the cleaning system. The heat map distribution result is used as input to the velocity behavior model for disinfection robots to efficiently clean the areas having peak values on the generated heat map. The robot calculates the velocity with which the robot should traverse to sterilize the environment efficiently. The higher the heat map’s value, the slower it to disinfect the region. On the other hand, the conventional cleaning systems use a uniform velocity model; thus, there are chances of leaving the regions uncleaned. So, this work demonstrates the implementation of an adaptive velocity behavior model based on both linear and non-linear functions to evaluate the performance of cleaning in terms of energy and time consumption with respect to the heat map distribution values.To model linear adaptive velocity behavior function, we use parameters, such as maximum velocity and minimum velocity, required for the cleaning robot to clean and the probability of the contaminated grid cell. For this approach, we considered maximum velocity to clean the environment as 0.2 m/s and minimum velocity to clean the peak contaminated regions as 0.05 m/s and modeled a linear equation, as shown in Equation .(16)vceSimilar to the linear function approach, we model the non-linear adaptive velocity behavior model by considering a maximum velocity of 0.2 m/s and minimum velocity greater than 0 m/s to clean the environment. We considered two non-linear functions based on tanh and sigmoid function to multiply with the linear function in Equation  to add nBased on the velocity models, we further implement zig-zag-based path planning strategy to perform selective cleaning over the contaminated regions by inducing more cleaning power and low velocities, unlike the other conventional cleaning robots. We evaluate the proposed adaptive velocity behavior model and path planning strategy with respect to time consumption and energy conserved.I is the current consumed.The energy consumption during the area coverage for the virtually connected floor cleaning system is directly related to the navigation trajectory and adaptive velocity behavior model generated by the WASP platform. To calculate the total energy consumed based on the motion control function generated by the adaptive velocity behavior model, we measure the energy consumption for a series of robot actions encountered during cleaning. We use the battery management system on the robot to record the current reading of the robot for each action performed by the robot. ns as in –21) whe wheX−axiThis linearly approximated energy vs velocity equation is used to estimate the total energy consumption for the proposed adaptive velocity behavior model.Based on each robot action’s energy consumption, we estimate the robot’s total energy consumption in the experiments and results section for each function model based on adaptive velocity behavior functions.In this work, the algorithms implemented to solve the performance issues of area coverage path planning on the minimization of time and power consumption of cleaning robots are validated. As part of this research work, we evaluated the WASP cleaning robot’s performance in MATLAB simulation and validated the proposed adaptive velocity behavior model for cleaning by organizing experiments under four different scenarios. We used different map environments for each scenario and conducted experiments by gradually increasing human density and interaction levels for each scenario. The total grid space for each map environment is defined with a resolution of 0.05 m. The generated occupancy grid map is decomposed into a mesh-grid with each cell of size 0.05 m. After performing human detection and estimating interaction levels, the heat map is generated, and the values are plotted on the map’s grid cells. The values accumulated on each grid cell range from 0 to 1, depending on the human interaction levels. With the generated heat map distribution on the grid cells, the robot produces an adaptive velocity behavior model to navigate and perform area coverage for scenarios I to IV. We compare the adaptive velocity behavior model results using the linear function and non-linear functions (tanh and sigmoid) with a uniform velocity behavior model similar to conventional cleaning robots. We validated the experimental results by simulating the Adaptive velocity behavior model with time consumption and energy consumption on the WASP cleaning platform. For scenarios I-IV, the experiments were run on a predefined workspace with dimensions and number of persons as listed in the The experimental trials were initiated by running the robot in the testbed environment to monitor the human interactions and safe distancing. The robot generates a heat map concerning the camera frame and records the detected humans’ pose. This data is then processed in Python to perform the transformation from camera frame to map frame and generate heat map plotting on grid map, as shown in The second phase of experimental trials was simulated in MATLAB software to generate an adaptive velocity behavior model for a zig-zag-based area coverage approach. The estimated time taken to cover the region with a uniform velocity model with a velocity of 0.15 m/s is around 712.3 s. The energy consumption for the entire area coverage process for the conventional cleaning method is 1.4958 Ah of the total battery capacity of 36 Ah. Similarly, for scenario II, we increased the number of humans interacting inside the testbed environment. After completely monitoring the human interactions in the defined space using the surveillance module, the generated heat map is processed to generate an area coverage path plan for both uniform and adaptive velocity models, as shown in Figure 13With uniform velocity model, the estimated time for area coverage is 497 s and the estimated energy consumption for the cleaning task is 1.0436 Ah, which is 2.89 percentage of the total battery. With the experiments in scenarios III and IV, we considered a larger testbed of dimensions 27 × 20 and 13.5 × 9 m, respectively. Moreover, the increased number of humans in the space (8 in scenario III and 11 in scenario IV) to validate the efficiency of the proposed adaptive velocity behavior model. This paper presents a novel multi-robot-based adaptive cleaning model in synergy with the human interaction heat map generated by a human safe distancing surveillance system on WASP intra-reconfigurable robot. This cleaning methodology is demonstrated to sterilize large space environments, like hospitals, shopping malls, or high-rise construction buildings, in response to the global pandemic caused due to the COVID-19. We introduced the mechanism design and software modules of the WASP robot and successfully demonstrated the ability to handle a safe distancing surveillance system along with a multi-robot cleaning method. In addition, we introduced adaptive velocity behavior models based on linear and non-linear functions to compare the cleaning performance with the uniform velocity model used in conventional cleaning robots. Experiments were performed in four different map environments by gradually increasing map size and human density to evaluate the performance of the proposed zig-zag path-based adaptive cleaning approach in terms of time and energy consumption. In all the experimental scenarios, the adaptive velocity behavior model based on the non-linear sigmoid function has shown superior cleaning performance in energy and time consumption. Moreover, the results using linear function it is similarly to results based on sigmoid function. In future works, we aim to implement the proposed adaptive cleaning approach for different coverage path planning algorithms and demonstrate the real-time cleaning performance on multi-robot cleaning systems.